HBA + Expander advice needed.

[TeCH-Guruz]

Hey guys looking for some advice on which HBA card and expanders to go for. I plan on using my motherboards data ports for my parity drives and some ssd cache drives, so the HBA + expanders will be used to run the main bulk of storage. The drives I will be using are 12TB Ironwolf drives. I plan on running 20 of them eventually will just be adding them as and when I get low or storage or see them for sale cheap. 

 

Due to the case layout it would be best if the HBA card has the ports on the back of it externally since it will make the athletics of my build much nicer. 

 

As far as the expander/expanders I need something that is not pci based and am struggling to find anything. The reason for this is they will be mounted in the back of the case away from the motherboard which again will keep everything looking much cleaner in the front of the case.

 

I'm guessing I am looking at an 2 port HBA with 2x expanders one for each port? Any help would be great I've ordered the case and trying to find a suitable motherboard/cpu combo but would be great if I can get the HBA and expanders ordered.

[JorgeB]

On 3/21/2020 at 8:12 AM, TeCH-Guruz said:

HBA card has the ports on the back of it externally

Not clear if you mean really external ports or just horizontal ports on the right edge like this:

 

[TeCH-Guruz]

4 hours ago, johnnie.black said:

Not clear if you mean really external ports or just horizontal ports on the right edge like this:

 

No I mean really external since I will be using the Tower 900 case so the external ports go out via the top which is covered and I can then route the cables to the back of the machine which isn't visible. 

here are a couple of pics might help you see what I'm talking about.

Front view

Top View 

Rear View where the radiators are mounted will be hdd's in my use case 

 

Edited March 22, 2020 by TeCH-Guruz

[JorgeB]

Then I would suggest something like the 9207-8e or 9300-8e, for expanders you can use Intel SAS2, there's a 24 and 36 port model, both can be molex powered.

[TeCH-Guruz]

4 minutes ago, johnnie.black said:

Then I would suggest something like the 9207-8e or 9300-8e, for expanders you can use Intel SAS2, there's a 24 and 36 port model, both can be molex powered.

Thanks @johnnie.black will look into getting them ordered. Since it seems like your the guy to ask about these things am I going to be bottlenecked by using the sas expander? also are there any other options for the sas expanders it would be better for cable management to use 2 separate ones that could each power 10 drives? 

[JorgeB]

With PCIe 3.0 controllers like the ones I recommended you'll get around 2200MB/s usable per port, then divide that by the number of connected disks, you can see some more performance numbers here.

[TeCH-Guruz]

10 minutes ago, johnnie.black said:

With PCIe 3.0 controllers like the ones I recommended you'll get around 2200MB/s usable per port, then divide that by the number of connected disks, you can see some more performance numbers here.

Thanks for the link great info there so looks like around 220MB/s while the drives are rated at 250MB/s so not too bad thanks again for all your help @johnnie.black

[TeCH-Guruz]

On 3/22/2020 at 1:42 PM, johnnie.black said:

With PCIe 3.0 controllers like the ones I recommended you'll get around 2200MB/s usable per port, then divide that by the number of connected disks, you can see some more performance numbers here.

I have now ordered 1x LSI SAS9207-8e and 2x expanders RES2SV240. 2x SFF8088 to SFF-8087 cables to go from the hba to the expanders. then 2x SFF-8087 to Sata breakout cables to get me started does that look about right @johnnie.black.

[JorgeB]

Yes, just make sure the breakout cables are forward breakout, reverse breakout look the same but won't work for this.

[TeCH-Guruz]

2 hours ago, johnnie.black said:

Yes, just make sure the breakout cables are forward breakout, reverse breakout look the same but won't work for this.

Yeah I think I got the right ones did notice some of them where reverse. Any idea how difficult it is to custom make these cables I know this probably isn't the best place to ask but seen as you seem to know a lot about these things I figured it was worth a try.

[JorgeB]

6 minutes ago, TeCH-Guruz said:

Any idea how difficult it is to custom make these cables

No, sorry.

[TeCH-Guruz]

2 minutes ago, johnnie.black said:

No, sorry.

Thanks for all your help @johnnie.black hopefully I will be pushing the limits of my new case very soon.

[TeCH-Guruz]

@johnnie.black got my hba and my expander card but seem to be having an issue with the hba the one I got was LSI SAS9207-8e as per your recommendation but when I load up the bios it is show it as only running a 4x speed. I have put it in the 16x slot since I only have 2 on my motherboard msi z270m mortar the first slot should run at x16 and the second slot at x4. To test things out I stuck my yestron rx 550 in the first slot and that shows up as x8 so not sure what the problem is right now any ideas? I don't have any cables yet to test it properly ordered them from china and with the current situation I'm sure it will take a while for me to get a hold of them.

[JorgeB]

LSI BIOS doesn't usually show PCIe link width, at least don't remember seeing that, check with lspci to confirm.

 

lspci -vv > /boot/lspci.txt

 

Then upload the txt here.

[TeCH-Guruz]

1 hour ago, johnnie.black said:

LSI BIOS doesn't usually show PCIe link width, at least don't remember seeing that, check with lspci to confirm.

 

lspci -vv > /boot/lspci.txt

 

Then upload the txt here.

root@FTE-Tower:~# lspci -vv > /boot/lspci.txt
pcilib: sysfs_read_vpd: read failed: Input/output error

 

and the log 

lspci.txt

[JorgeB]

Don't know what the error is about but it does confirm x4 link width:

 

LnkSta:    Speed 8GT/s (ok), Width x4 (downgraded)

 

It's either a board issue or a bad HBA, do you have another board you can test with?

[TeCH-Guruz]

13 minutes ago, johnnie.black said:

Don't know what the error is about but it does confirm x4 link width:

 

LnkSta:    Speed 8GT/s (ok), Width x4 (downgraded)

 

It's either a board issue or a bad HBA, do you have another board you can test with?

I don't have another board to test with but I did put my rx 550 in the first slot and that shows up as x8 

 

any chance it could be because no cables are plugged in yet?

 

Edited April 7, 2020 by TeCH-Guruz

[JorgeB]

8 minutes ago, TeCH-Guruz said:

any chance it could be because no cables are plugged in yet?

Nope.

[TeCH-Guruz]

1 hour ago, johnnie.black said:

Nope.

firmware? just updated my bios but no change.

Edited April 7, 2020 by TeCH-Guruz

[JorgeB]

12 hours ago, TeCH-Guruz said:

firmware?

Shouldn't make any different, but it won't hurt, and it should be on latest firmware anyway: 20.00.07.00

[TeCH-Guruz]

3 hours ago, johnnie.black said:

Shouldn't make any different, but it won't hurt, and it should be on latest firmware anyway: 20.00.07.00

Unfortunately is on firmware 20.00.00.00 and when I tried to update it in windows I get the following message 

No LSI SAS adapters found! Limited Command Set Available!

Finished Processing Commands Successfully.

Exiting SAS2Flash.

[JorgeB]

It's probably not using LSI firmware, which wouldn't surprise me since AFAIK LSI firmware never shows link speed/width at boot, though still likely unrelated to x4 link.

[TeCH-Guruz]

6 minutes ago, johnnie.black said:

It's probably not using LSI firmware, which wouldn't surprise me since AFAIK LSI firmware never shows link speed/width at boot, though still likely unrelated to x4 link.

It doesn't show the link speed at boot but I can look in my motherboard bios to see the link speed. Any chance you can point me in the right direction for refreshing this to lsi firmware?

[JorgeB]

Just now, TeCH-Guruz said:

It doesn't show the link speed at boot but I can look in my motherboard bios to see the link speed.

Ahh, that makes more sense, in that case flashing should work correctly, and firmware 20.00.00.00 has known issues, though link speed is not one of them.

 

If you can't flash with Windows try booting with a DOS flash drive, or try flashing it in a different board, older boards are better for DOS flashing, Windows flash should work basically everywhere, never had issues.

[TeCH-Guruz]

2 hours ago, johnnie.black said:

Ahh, that makes more sense, in that case flashing should work correctly, and firmware 20.00.00.00 has known issues, though link speed is not one of them.

 

If you can't flash with Windows try booting with a DOS flash drive, or try flashing it in a different board, older boards are better for DOS flashing, Windows flash should work basically everywhere, never had issues.

Ended up downloading megaRAID to flash the card updated the bios while I was there but still x4 downgraded.