SATA Port Multiplier Recommendation?

[werkkrew]

As the subject says I am looking for the highest performance SATA 3 port multiplier I can find.

 

I am completely new to unRaid so please don't hesitate to let me know if my concerns are misplaced, specifically as to how the parity check speed relates to real-world performance of unRaid.

 

Here is my current unRaid system:

 

• HP z820 Workstation, Dual Xeon E5-2630v2's with 128GB ECC Memory
• 13 spinning disks - 4x 8TB WD Reds, 6x 4TB Seagate, 3x 3TB HGST
• Cache is a 512GB M.2 NVMe card on a PCIe 3.0 adapter
• The tower itself has room for 8 drives, which are on the on-board LSI SAS 2308 SAS controller which is on the PCIe 3.0 bus (4x 8TB WD Reds, 4x 4TB Seagate)
• 5 drives are in a SANS Digital 8-bay TowerRaid enclosure, 3 drives on 1 expander and 2 on the other which is connected to a HighPoint RocketRaid 622 PCIe 2.0 x1 Adapter 

 

I got the z820, the TowerRaid and the RocketRaid for dirt cheap from a friend, so I am inclined to try to use it before I go spending a bunch of money trying to set up a proper SAS HBA with expanders and a separate enclosure - or better yet, a 24+ bay rackmount case.  Although I will likely go down that road eventually, money is a bit tight at the moment so I'm trying to get the most out of what I have for the time being.

 

The RR622 is flashed with the Quick BIOS and is in full JBOD mode - I had to use the "enable_ahci" script to get unRaid to recognize the drives which I don't really like.

 

When running DiskSpeed against the RocketRaid 622 the controller was able to max out the PCIe 2.0x1 slot at 500MB/s using 3 drives on an expander.  Each drive tested individually showed identical performance to when they were on the LSI SAS adapter.  Really, this isn't too bad and was better than I expected.

 

However, parity checking is only able to peak out at about 70-72MB/s when the system is completely idle, generally around 40-50MB/s when the system has activity on it such as Plex streams, which is pretty much all the time.  With an 8TB parity drive this puts my parity check/rebuilt time at over 50 hours which is a huge concern.  Another concern is what day to day performance will be like once the initial parity check is completed - when things are not going to cache.

 

Luckily I have a MicroCenter nearby so I thought I would test out some other adapters.  

 

I tried a Syba SI-PEX40057 which is a Marvell 88SE9230 based PCIe 2.0x2 card thinking I would at least double the bandwidth and get 1 dedicated PCIe lane per eSATA port with it for <$50.  Good news is it worked right out of the box with unRaid (no special BIOS or enable_ahci script needed).  The bad news is that it can only address 4 drives maximum, no matter if its 1 on each port or 4 multiplied on 1 port - so this wouldn't work for my needs.

 

I returned that and got two MediaSonic HP1-SS3 cards, which are AsMedia ASM1062 based PCIe 2.0x1 cards.  They were <$20 each so I figured with two it would be equivalent to what I attempted above.  The also worked out of the box with no BIOS flash or enable_ahci script needed.  However, in DiskSpeed one drive would work at full speed (~180MB/s) but each additional drive would drop that total down linearly.  2 drives would peak out at about 160MB/s (80 each) and 3 drives would peak out at about 120MB/s (40 each).  I am really suprised at how slow these cards are even though they state they have FIS based switching, the performance is really bad.

 

So now I'm here. 

 

I'm leaning toward spending the money on this: https://www.shopaddonics.com/ProductInfo.aspx?code=AD4ES6GPX4

 

It's PCIe 2.0x4 (~2000MB/s) with full (advertised) support for all the SATA features I need, and potentially enough bandwidth to not be a bottleneck.  Before I do though, I want to see if it's worth it as opposed to other options, or even just living with the slow parity checks.  That would make the bottleneck the 2x 6Gb/s eSATA ports.  Giving each drive about 1.5Gb/s or about 185MB/s, theoretically.  Which is about the peak performance I see out of any one drive in my system.

 

Since I plan to keep the z820 for a while before I consider a proper rack mount storage chassis I want to get a solution for externally attached drives which are not bottle-necked by anything other than the speed of the disks themselves.

 

I suppose I am also open to what is essentially the best route to have an a cost-effective external enclosure  of disks that will leave the individual drives as the bottleneck and not anything in between but I would like to see if I can make it happen with the SATA multipliers.

 

[ken-ji]

The general consensus here is that Port Multipliers tend to be bad news  waiting to happen.

If it works out for you than great because its a relatively cheap solution (as it was meant to be)

That said I'm fairly sure the ASM1062 do not support FBS, rather CBS only.

 

The card looks great and I would have gotten that if I was comfortable with testing stuff out. (buying abroad for testing after getting burned a few times is a calculated risk)

 

I used to run the PM setup but have migrated to SAS + Port Expander. Its just a matter of purchasing a new enclosure now if I want to expand from my 6+1 array right now.

[werkkrew]

Yeah, the more I thought about it the more I just decided to live with what I have while I get the stuff I need to do it with SAS.

 

I think I'm going to follow this guide and use a LSI SAS 9206-16e HBA.

 

https://www.reddit.com/r/DataHoarder/comments/89jqfu/my_300_diy_16_drive_das_build/

 

I think I can build a really nice, bottleneck-free DAS to attact to my z820 for around $300 or so and it would give me 16 (11 empty) external bays to grow into vs the 8 (3 empty) I have now.