DavidInDE Posted December 22, 2018 Share Posted December 22, 2018 (edited) Hi I was wondering if I could get the assistance of those more knowledgeable than me (ie most of you) in cabling up a new system: My hardware - a 12 bay Supermicro SC826BE2C with SAS3 dual-expander backplane, an X9-DRF EATX motherboard and 2 x Xeon E5 2650l V2 CPUs ('controller'). A second Supermicro CSE826 JBOD 12-bay chassis with BPN-SAS2-826EL1 backplane ('shelf'). My questions are: a) What HBA will be best (I was looking at the LSI 9211-8i, but am completely open to suggestions) b) What cabling will I need? I was thinking HBA to backplane of 'controller' and then downstream SAS connection from 'controller' to 'shelf', or am I making things way too straightforward? I'd appreciate your help, as although I am likely to learn from my mistakes, I'd rather get it right first time than waste money needlessly! I am referencing the EL1 manual, which can be found here. If I've left any pertinent information out, let me know and I will supply it. Thanks again, and Merry Christmas everyone! Edited December 22, 2018 by DavidInDE Included explanation of which chassis was the controller and which the shelf Quote Link to comment
JorgeB Posted December 22, 2018 Share Posted December 22, 2018 I would use a 9207-8i instead, since it's PCIe 3.0 and if you use all bays with fast disks you can use the extra bandwidth, this would get you around 185MB/s per disk with all slots occupied, as for cabling, one cable from the HBA to each backplane, no need for cascading, 1 Quote Link to comment
DavidInDE Posted December 22, 2018 Author Share Posted December 22, 2018 Thanks for your advice johnnie.black....would I be better off going with something like this, as it is also PCIe 3.0, but has the four internal connections (for the controller), plus four external (which would go to the shelf). Am I thinking correctly? Quote Link to comment
JorgeB Posted December 22, 2018 Share Posted December 22, 2018 8 minutes ago, DavidInDE said: would I be better off going with something like this, as it is also PCIe 3.0, but has the four internal connections (for the controller), plus four external (which would go to the shelf). Am I thinking correctly? Yep, that's even better, so you don't need an internal to external adapter. 1 Quote Link to comment
DavidInDE Posted December 22, 2018 Author Share Posted December 22, 2018 Quick question...since the backplane of the controller is SAS3, and the backplane of the disk shelf is SAS2, will I get higher IO sppeds from those drives connected to the controller backplane? or will the bottleneck be at the PCIe slot? Thanks for the benefit of your knowledge! Quote Link to comment
JorgeB Posted December 22, 2018 Share Posted December 22, 2018 22 minutes ago, DavidInDE said: since the backplane of the controller is SAS3, and the backplane of the disk shelf is SAS2, will I get higher IO sppeds from those drives connected to the controller backplane? Only if you get a SAS3 HBA like the 9300-8i (or 4i4e) and your devices are SAS3, but for 12 disks SAS2 is good, though if you can get a 9300 for not much more money it's worth getting. A 4 lane SAS1 link has a maximum theoretical bandwidth of 1200MB/s, SAS2 doubles that to 2400, double again for SAS3 to 4800MB/s, there's always some overhead, I tested with SAS1 and SAS2 expanders and real world max speeds are 1100 and 2200MB/s respectively, never tested SAS3 but assume overhead would be similar, so around 4400MB/s usable. So, if you use a SAS2 HBA you'll have 2200MB/s usable for each backplane, the previously mentioned 185MB/s per disk if all 12 bays are in use, 4400MB/s for both backplanes total, which is above the x8 PCIe 2.0 link, hence why I suggested a PCIe 3.0 HBA. If you get a SAS3 HBA you'll have around 4400 + 2200 max bandwidth, which is likely a little above what a x8 PCIe 3.0 can handle, but don't forget only if you were to use SAS3 devices, and very fast ones at that, like SAS3 SSDs, if using SAS2 or SATA3 disks link will still be 2400MB/s, so probably not worth getting one unless you can find a cheap 9300 and/or you're using SAS3 SSDs of very fast SAS3 disks, like 12 or 14TB 7200rpm disks. 1 Quote Link to comment
JorgeB Posted December 23, 2018 Share Posted December 23, 2018 6 hours ago, johnnie.black said: but don't forget only if you were to use SAS3 devices Let me correct myself, since I never had one I sometimes forget, LSI SAS3 HBAs and expanders have a feature, called DataBolt, that is supposedly able to use full SAS3 bandwidth with SAS2/SATA3 devices, never tested so don't know how well it works, but good to keep in mind. 1 Quote Link to comment
DavidInDE Posted December 23, 2018 Author Share Posted December 23, 2018 I am impressed - I actually learned more from those two posts than several hours of web searching (with an unhealthy fixation on Supermicro's web site) yielded... Quote Link to comment
JorgeB Posted December 23, 2018 Share Posted December 23, 2018 (edited) Btw, let me also post this, I was looking at some LSI pdfs because I didn't remember the name of the DataBolt feature, and found these real world bandwidth values that line up well with my experience, especially the SAS bandwidth, PCIe might be a little optimistic, at least I was never able to get more than 3000MB/s from a x8 PCIe 2.0 slot: Edited December 23, 2018 by johnnie.black 1 Quote Link to comment
DavidInDE Posted December 23, 2018 Author Share Posted December 23, 2018 Thank you. You've been really helpful; it's always good to know you've learned something new. Quote Link to comment
DavidInDE Posted December 26, 2018 Author Share Posted December 26, 2018 (edited) I am the first to admit my ignorance on this topic, so... Can one hook up a disk shelf (as we discussed) and use both the original storage server AND the shelf to store 12 SATA disks on each, or is this locking the user in to the use of SAS drives? Edited December 26, 2018 by DavidInDE I can't type worth two cents Quote Link to comment
JorgeB Posted December 27, 2018 Share Posted December 27, 2018 You can use SAS or SATA on any of the shelfs, you can even have mixed SAS and SATA on the same shelf, only ting to be aware is speed will be limited if using slower linking devices, like SAS1/SATA2 devices in any of them. 1 Quote Link to comment
DavidInDE Posted December 27, 2018 Author Share Posted December 27, 2018 Thanks Johnnie and Happy New Year! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.