Best option to connect more hdd's?

[FreakyUnraid]

Hi,

 

Currently running a fujitsu D3643-H (4x sata) with 2x IBM M1015 (2x 8 sata). All ports are populated with 2,5" drives. With 20 drives I'm at capacity and I need more storage. No more ports left, so looking for a good solution to expand the number of ports. Less money and less power consumption is much better.

 

Options I considered but deemed not a good option:

- Changing HDDs for larger ones. Already running 5TB drives, so can't go bigger in 2,5". Going 3,5" would mean I have to change out at least 3 drives (I have dual parity) to go bigger. With at least 16TB being the best option this would set me back at least $800. Too expensive and also 3,5" make too much noise for my taste. Server is located in a room I often work and sit in so it's a big deal. The 2,5" are so quiet, I can't even hear them. And I have a couple spares laying around.

- Changing 1x M1015 for a 16 port HBA. Again, this would cost a lot. You can get one for around $170 on ebay, but I would like to get 2 so I have a spare laying around just in case. Ebay and shipping can take weeks if not months and I don't want my server to be down so long. With $340 this would be too costly in my opinion.

 

The only real option I found was to change out 1x M1015 for an expander. Looking around the Intel RES2SV240 looks to be the best option? For around $60 not a bad deal? The Lenovo is real cheap at just $30 but I think I can't get enough ports with that option? 

 

After reading the performance topic on throughput I'm a bit worried that the PCIe 2.0 M1015 is going to bottleneck my drives quite a bit. They start parity at around 135MBps. With (in theory) 20 drives on the expander this would mean they would be bottlenecked to around 113MBps if my math is correct? Could this be damaging in some other way than just parity taking a bit longer? Perhaps I should change out the M1015 for a PCIe 3.0 card like the 9207-8i and sell my M1015's? 

 

So in short:

 

NOW

2x M1015

 

Option 1

1x M1015 (PCIe 2.0 bottleneck with 20 drives on expander?)

1x Intel RES2SV240

 

Option 2

1x 9207-8i PCIe 3.0 (Dell or a different one?)

1x Intel RES2SV240

 

Option 3

? Suggestions are welcome.

 

 

Would love to hear your thoughts on this.

 

Thanks.

 

Edited November 7, 2022 by FreakyUnraid
113MBps max with 20 drives if I understand these results correctly https://forums.unraid.net/topic/41340-satasas-controllers-tested-real-world-max-throughput-during-parity-check/

[Vr2Io]

You won't got benefit change to 9207, because it still 6gb HBA same as M1015, bottleneck not on PCIe2.0. You need concurrent upgrade PCIe3.0 and HBA and Expander to 12Gb.

 

For 2.5 vs 3.5, this also apple and orange, 3.5 easy got high capacity CMR drive. If in terms of cost and power usage, it will endup no benefit using 2.5.

 

For example, three 5TB 2.5 vs one 16TB 3.5 disk, 3.5 would all win.

 

PS : Last mount I buy 18TB disk just 257usd, it even cheaper then shuck disk a lot. This only need one port instead 3 or 4 port.

 

The drawback also as your mention, hard for cooling and noise ( cooling fan not disk itself ) and much longer parity check time.

 

My current config are 9300-4i4e, 4i connect expander to 16disk and 4e connect expander to 12disk. All expander are 12Gb. This setup still have room to add one more expander for more disks.

Edited November 7, 2022 by Vr2Io

[FreakyUnraid]

10 hours ago, Vr2Io said:

You won't got benefit change to 9207, because it still 6gb HBA same as M1015, bottleneck not on PCIe2.0. You need concurrent upgrade PCIe3.0 and HBA and Expander to 12Gb.

 

Are you sure about that? These 2,5" Seagates only do about 135MBps max. 

 

HBA has 8x 6 Gbps

With 1 link to the expander; that's 4x 6 Gbps = 24 Gb

With max 20 drives that comes to 24/20 = 1,2 Gb per drive = 1,2 / 8 = 150MBps

 

PCIe 2.0 does 500 MBps per lane. HBA is x8 with 2 ports which comes to x4 per port?

So 500 MBps x 4 = 2000 MBps / 20 drives = 100 MBps max theoretical. In practice this will be lower is my guess. 

So PCIe 2.0 is the bottleneck here.

 

Changing out the PCIe 2.0 card for a PCIe 3.0 card result in:

 

PCIe 3.0 does 985MBps per lane x 4 = 3940 MBps / 20 drives = 197 MBps

I read that in practice this is more like 3200 MBps / 20 drives = 160 MBps

Still well over the max these drives can do. Around 20% headroom.

 

If, in the future, I would switch to 3,5" drives which can do much higher speeds I will also need fewer drives. So I think I'm good when I switch to something like a 9207-8i, right?

 

10 hours ago, Vr2Io said:

For 2.5 vs 3.5, this also apple and orange, 3.5 easy got high capacity CMR drive. If in terms of cost and power usage, it will endup no benefit using 2.5.

 

For example, three 5TB 2.5 vs one 16TB 3.5 disk, 3.5 would all win.

 

PS : Last mount I buy 18TB disk just 257usd, it even cheaper then shuck disk a lot. This only need one port instead 3 or 4 port.

 

The drawback also as your mention, hard for cooling and noise ( cooling fan not disk itself ) and much longer parity check time.

 

My current config are 9300-4i4e, 4i connect expander to 16disk and 4e connect expander to 12disk. All expander are 12Gb. This setup still have room to add one more expander for more disks.

 

The 2,5" drives need more HBA/Expanders so the power consumption difference between a 3,5" setup is indeed negligible at this point. I just really like these little drives for being so quiet. Even in 100% operation (all drives) I can barely hear it. I remember my old Synology with 4x 3,5" drives which was in the same spot the current server is, and the noise was really unbearable.

 

The power consumption 'demand' was for the new hardware. Power is getting expensive so I don't want a HBA or Expander that consumes a ton of power when there are (far) better and more efficient once out there. 

 

 

 

 

Edited November 7, 2022 by FreakyUnraid