Hi, I couldn't find this topic addressed here.

 

3 PCIe 4.0 slots on MOBO

1 Intel RAID Expander Card (RES2SV240)

1 GPU

20 7200 RPM 6gb/s disks

 

I have a lot of options of how to wire things up given my LSI SAS cards and I'm not sure the best way.

 

I can EITHER use 2 LSI 9211-8i (6gb/s) and the expander card (molex powered) or 1 LSI 9300-8i (12gb/s) and the expander card (pci powered).

 

A couple of questions / thoughts.

 

1. I think the molex powering the expander card may not provide enough max watts to support max power draw causing drives to power bump -- hence I'm wondering if just using the pcie power directly to power the expander may be more stable. 
2. Question - LSI bandwidth per port -- can the LSI 9300-8i (12gb/s) offer all 12gb/s through a single sas port or is it just 6 gb/s per port?
   1. This is important as it raises the question, should I feed the expander card with 2 inputs from the HBA or is one the same? Does adding 2 inputs help the bandwidth through the adapter at all or should I just dedicate 1 port to expander and 1 port to a sas rail of drives (4)?
3. Question - Any thoughts as to if there's any difference between having 2 6gb/s HBAs vs 1 12 gb/s HBA? 

 

I'm trying to figure out the most efficient way to plug in the cables to maximize throughput. I'm not a SAS expert and I know networks have bandwidth limitations per port as do switches...just curious as to where the bottlenecks would be in this setup with HBAs and the expander.

 

The reason I need the expander is because I have 20 drives and only 2 free PCIe slots (don't want to pay for a 24 port HBA). Furthermore, the power concern in #1 above is why I'm thinking it may be more stable to have just a single HBA and plug the expander directly into PCIe rather than rely on the molex connector for power.

 

Looking for help, thanks in advance.

2 hours ago, srfnmnk said:

may be more stable. 

No issue, if all connect well.

 

2 hours ago, srfnmnk said:

LSI 9300-8i (12gb/s) offer all 12gb/s

All port / lane were 12gb.

 

2 hours ago, srfnmnk said:

Question - Any thoughts as to if there's any difference between having 2 6gb/s HBAs vs 1 12 gb/s HBA? 

No big difference, 12gb in general only useful for 12gb expander, because device often 6gb and haven't use up that bandwidth.

 

6gb single cable have ~2GB/s throughput, so slightly bottleneck for 20 disks. You can go to dual cable or 12gb, actually depends which you like.

Edited January 10, 20233 yr by Vr2Io

4 minutes ago, Vr2Io said:

All port / lane.

Can you say more about this? Does this mean a 12gb/s card with 2 ports has only 6 gb/s per port? Does port == lane? If so, would plugging both ports into the expander grant the full bandwidth the expander (i.e. 2 ports to HBA and 4 to drives)

 

4 minutes ago, Vr2Io said:

because device usually 6gb and haven't use up that bandwidth

I'm thinking about local, data-intensive tasks -- perfect example is parity check / rebuild. If each disk can real-world seq write that's (120MB/s * 20) 2.4GB/s. If my math is write with a single 12gb/s card I shouldn't expect total array throughput to be any greater than 1.5GB/s, is this theory correct?

Edited January 10, 20233 yr by srfnmnk

1 hour ago, srfnmnk said:

Can you say more about this? Does this mean a 12gb/s card with 2 ports has only 6 gb/s per port?

All lane is 12gb, i.e. 8i card means support 8x 12gb max.

 

1 hour ago, srfnmnk said:

If my math is write with a single 12gb/s card I shouldn't expect total array throughput to be any greater than 1.5GB/s, is this theory correct?

 

All depends on the device speed i.e. 6 or 12gb.

 

For 6gb, single cable also provide over 1.5GB/s.

 

In simple say, 12gb only useful between controller and expander when both is 12gb.

Edited January 10, 20233 yr by Vr2Io

Does your motherboard have sata ports? You can also do 2 LSI 8i for 16 drives and put the remaining 4 on motherboard slots. This allows you to not think too much about link bottlenecks 

 

I have a 24 drive setup, with 16 connected to LSI-9300-16i and 8 connected to the motherboard. I skipped any expanders for this one. 

 

Also a previous thread covering this topic with some numbers: 

 

 

Edited January 11, 20233 yr by apandey
Added previous thread link

That’s a decent idea but my case has a back plane with sas connectors that allow the drivers to be hot swappable. Not sure if I could rig it to accept sata and maintain hot swap.

 

I have a NORCO RPC-4220 4U

Edited January 11, 20233 yr by srfnmnk

50 minutes ago, apandey said:

Also a previous thread covering this topic with some numbers: 

This is exactly what I was looking for! Thank you! 

@apandey been doing some reading and I’m wondering if some sata sas reverse breakout cables would work. mobo sata (x4) to mini sas on the back plane. Anyone else got this to work?

 

Cable Matters Internal Mini SAS to SATA Reverse Breakout Cable (SFF-8087 to SATA) 1.6 Feet https://a.co/d/inC8lZf

@srfnmnkno real experience with reverse breakout cables myself

As for hot plug, that is possible with SATA. It needs to be supported by the controller and most modern sata controllers do. But hard to say for your specific case. Best bet is to try, the expense is just a cable here

10 hours ago, srfnmnk said:

been doing some reading and I’m wondering if some sata sas reverse breakout cables would work. mobo sata (x4) to mini sas on the back plane. Anyone else got this to work?

If connecting to an enclosure with a SAS expander it will work if the board has a SAS controller, it won't work with SATA, it *should* work with SATA if the enclosure has no expander.

I bought a reverse breakout cable, I figured I'd give it a try and see if it works. I'll report back.

 

Enclosure has no expander. Planning to just connect sata to mobo and sas to 4-drive rail in the Norco case.