Advice on cabling new unRAID system required...


DavidInDE

Recommended Posts

Hi

I was wondering if I could get the assistance of those more knowledgeable than me (ie most of you) in cabling up a new system:

My hardware - a 12 bay Supermicro SC826BE2C with SAS3 dual-expander backplane, an X9-DRF EATX motherboard and 2 x Xeon E5 2650l V2 CPUs ('controller').

A second Supermicro CSE826 JBOD 12-bay chassis with BPN-SAS2-826EL1 backplane ('shelf').

My questions are:

a) What HBA will be best (I was looking at the LSI 9211-8i, but am completely open to suggestions)

b) What cabling will I need? I was thinking HBA to backplane of 'controller' and then downstream SAS connection from 'controller' to 'shelf', or am I making things way too straightforward? 

I'd appreciate your help, as although I am likely to learn from my mistakes, I'd rather get it right first time than waste money needlessly!

I am referencing the EL1 manual, which can be found here. If I've left any pertinent information out, let me know and I will supply it. Thanks again, and Merry Christmas everyone!

Edited by DavidInDE
Included explanation of which chassis was the controller and which the shelf
Link to comment
8 minutes ago, DavidInDE said:

would I be better off going with something like this, as it is also PCIe 3.0, but has the four internal connections (for the controller), plus four external (which would go to the shelf). Am I thinking correctly?  

Yep, that's even better, so you don't need an internal to external adapter.

  • Like 1
Link to comment
22 minutes ago, DavidInDE said:

since the backplane of the controller is SAS3, and the backplane of the disk shelf is SAS2, will I get higher IO sppeds from those drives connected to the controller backplane?

Only if you get a SAS3 HBA like the 9300-8i (or 4i4e) and your devices are SAS3, but for 12 disks SAS2 is good, though if you can get a 9300 for not much more money it's worth getting.

 

A 4 lane SAS1 link has a maximum theoretical bandwidth of 1200MB/s, SAS2 doubles that to 2400, double again for SAS3 to 4800MB/s, there's always some overhead, I tested with SAS1 and SAS2 expanders and real world max speeds are 1100 and 2200MB/s respectively, never tested SAS3 but assume overhead would be similar, so around 4400MB/s usable.

 

So, if you use a SAS2 HBA you'll have 2200MB/s usable for each backplane, the previously mentioned 185MB/s per disk if all 12 bays are in use, 4400MB/s for both backplanes total, which is above the x8 PCIe 2.0 link, hence why I suggested a PCIe 3.0 HBA.

 

If you get a SAS3 HBA you'll have around 4400 + 2200 max bandwidth, which is likely a little above what a x8 PCIe 3.0 can handle, but don't forget only if you were to use SAS3 devices, and very fast ones at that, like SAS3 SSDs, if using SAS2 or SATA3 disks link will still be 2400MB/s, so probably not worth getting one unless you can find a cheap 9300 and/or you're using SAS3 SSDs of very fast SAS3 disks, like 12 or 14TB 7200rpm disks.

  • Like 1
Link to comment
6 hours ago, johnnie.black said:

but don't forget only if you were to use SAS3 devices

Let me correct myself, since I never had one I sometimes forget, LSI SAS3 HBAs and expanders have a feature, called DataBolt, that is supposedly able to use full SAS3 bandwidth with SAS2/SATA3 devices, never tested so don't know how well it works, but good to keep in mind.

  • Like 1
Link to comment

Btw, let me also post this, I was looking at some LSI pdfs because I didn't remember the name of the DataBolt feature, and found these real world bandwidth values that line up well with my experience, especially the SAS bandwidth, PCIe might be a little optimistic, at least I was never able to get more than 3000MB/s from a x8 PCIe 2.0 slot:

 

image.png.b3dbdb0ad137c0193b9bbc916049f447.png

 

 

Edited by johnnie.black
  • Like 1
Link to comment

I am the first to admit my ignorance on this topic, so...

 

Can one hook up a disk shelf (as we discussed) and use both the original storage server AND the shelf to store 12 SATA disks on each, or is this locking the user in to the use of SAS drives?

Edited by DavidInDE
I can't type worth two cents
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.