Server setup - optimal drive connections across 2 SAS cards

[tiwing]

Hi all, I recently picked up a "new to me" supermicro SuperStorage 6047R-E1R36N. I was tired of fiddling with cables and a hacked together solution in an old old case. It has lots of power and room for more drives than I'll throw at it for a long long time. It has 2 LSI SAS cards. I believe one is connected to the front 24 drive bays, the other to the back 12. I currently have 3 SSD cache drives in a BTRS pool, 2 SSD in unassigned devices - one as a download scratch disk and one as a VM host for 2 lightly used VMs. There is a 2.5" old laptop spinner that's used to store seeding files before they're deleted from the downloader. The actual array consists of 10 WD Reds of varous sizes.

 

For ease of access I put all the drives in the front 24 spots. And it seems to work fine. But is there a more "optimal" way to do this? Perhaps to put the SSDs and 2.5" spinner in the back and leave the Array all up front? Or split the Array and cache both between front and back?

 

Looking for recommendations form your immense collective experience please!

 

Tiwing

[HSEDuster]

I don't know if its "optimal" but I have an R710 with an H200 for the front drives and an LSI card for drives in an external DAS. I personally have my array drives, and most unassigned drives, in the external DAS and I have 2 SAS SSD cache drives in the 710 itself, along with 4 470GB unassigned SSDs I have yet to use for anything (my old cache drives). 

 

My DAS is SAS 1, that is 3Gbps. The H200 is SAS 2 (6Gbps). While I haven't tried, I'm pretty sure if I put my cache drives in my DAS, that would slow things down. In your case though, if both cards are the same generation, I don't think it would matter too much. I mean, a spinning drive, or even a few, aren't going to saturate a SAS link. A bunch of SSDs might, but only in heavy use situations. 

 

FWIW there is a docker you can get to benchmark your drives and adapter cards. If it's not a huge PITA you could swap things around, do some benchmarks and/or transfer tests and see what works best? 

[tiwing]

"Diskspeed" docker- I have used that just after starting up the new server. DIdn't see anything out of the ordinary. Both LSI cards are new, and the backplanes from what I've read are both SAS2. I will play around with it and benchmark... but based on your comments about 6Gbps I can't see any of my usage saturating the links... unless there is some IOwait being caused by the cards themselves or something that I hven't really experienced yet. 

 

thx for the comments

[JorgeB]

You can also see here for some numbers, with single and dual link.

[tiwing]

Awesome thread, thanks!