So I got a interesting new piece of hardware I didn't have to buy. I got new new Dell HBA355 Adapter.  I got all the firmware updated on it 2 months ago. About all I could do on that. Anyway uses SFF-8654 Connectors. So I had to get these break out cables.
https://www.amazon.com/dp/B0B5CRTG52?psc=1&ref=ppx_yo2ov_dt_b_product_details

Bad thing is I don't have a board that I can do PCIE 4.0 or a CPU for my server to do that. I just recently got switched over to a Xeon E-2226g
and Running a ASRock Rack Server board 64 gigs ECC too. Anyway using this Controller I think it down stepped to PCIE 3.0. But any idea what performance I should expect with it on my spin drives? And or should I put my Sata SSD's on it too from my cache array? I'm just trying to get the best I/O I can expect. I was very shocked I couldn't setup any sort of raid array at all with this controller. Not like I want to do that. I want IT mode anyway. But I found it really weird that Dell locked it down to be just a HBA. Watch the days of just getting a Dell controller and re-flashing it could be going away.

 

 

9 hours ago, Bizquick said:

it down stepped to PCIE 3.0. But any idea what performance I should expect with it on my spin drives?

Assuming an x8 slot it's still good for around 16x275MB/s, so have enough bandwidth with spinners.

 

9 hours ago, Bizquick said:

And or should I put my Sata SSD's on it too from my cache array?

You can if TRIM is supported.

 

I would hope so its the LSI SAS3816 chipset. Also there was no way I could even configure a raid array all drives were seen as just drives. I didn't have to do any pass thru crap or attempt to make an array. if even wanted the raid functions restored so I could make a raid 0 or a raid 1. I would have to figure out how I could cross flash it to LSI retail mode.

I think too many bad users tried to flash their old cards to IT mode and Dell got a bunch of dead parts. So it looks like they are trying to lock that down. If you want HBA mode you buy the HBA mode controller. If you want IR mode you get the Raid mode controller. They sell this HBA controller for Azure MS servers apparently. we use it for Storage Space Direct on a few small MS clusters. 

I'm getting really good Speeds on it. I got 4 drives on 1 channel. and the other 2 drives on B channel. I got 215Megs avg when I did a PreClear on a 12TB drive.  I usually get 160 to 175megs on my SAS3008-16i card. And that was putting putting 3 drives on port1 and 2 drives on Port 4 and parity on the Motherboard. But ZFS kills my speeds though. I have to get my one ZFS drive out of my main array. I like having it for making a place to clone my snap shots to. But Not worth it right now. I'll come up with another Idea. Over all I'm shocked to see the speeds above 200 when it runs for more than 10 plus hours.

 

42 minutes ago, Bizquick said:

I would hope so its the LSI SAS3816 chipset.

Most LSI HBAs only support TRIM for devices with read zeros after TRIM support (deterministic TRIM support).

 

43 minutes ago, Bizquick said:

I usually get 160 to 175megs on my SAS3008-16i card.

That would be a different issue, that HBA has the same bandwidth as the new one when used with a PCIe 3.0 slot.

 

44 minutes ago, Bizquick said:

I have to get my one ZFS drive out of my main array

Currently there's a known write performance issue with zfs on the array, single or multi device pools are not affected.