This is a bit convoluted, but here goes...
I'm migrating my Unraid server to a new box with all NVMe drives (16 x Intel P4510). The drives are hooked up to several HBA cards - two HighPoint 1580 and one Broadcom 9600-24i.
12 drives are connected directly to the two HighPoint cards, and 4 remaining drives are inside an Icy Dock enclosure, which is connected to the Broadcom card via Oculink cables.
For some reason, the drives inside Icy Dock read/write at around 3 GB/s, while the drives connected to the HighPoint card max out at 750 Mb/s. I'm using DiskSpeed docker for benchmarks.
Any ideas on where to begin figuring this out? Diagnostics file attached.
roach-diagnostics-20240325-1522.zip