"Slow" PCI-E 4.0 NVME Performance using UnRAID.

[Outcasst]

Hello there.

 

I believe I am having an edge case issue regarding Gen 4 SSD performance in the UnRAID OS. I have two PCI-E 4.0 drives, a Sabrent Rocket 1TB and a Samsung 980 Pro 1TB.

 

My original intention was to use these in a RAID0 cache pool, however I noticed performance much worse than I was anticipating, even in individual drive modes. I decided to benchmark the drives individually incase of the RAID0 not functioning correctly.

 

Using the DiskSpeed docker, along with some file transfers, on the Sabrent drive I was only seeing read performance up to a maximum of 4.4GB/s along with a maximum of 4.1GB/s on the Samsung. I originally thought that they may not actually be running at PCI-E 4.0, however I double checked that they were. The Sabrent is rated at 5GB/s and the Samsung at 7GB/s read speeds. I have only benchmarked read speeds so far in the UnRAID OS since it is easy to do with the DiskSpeed docker.

 

I decided that it could be one of two things causing these bottlenecks; either the filesystem (I tried both BTRFS and XFS with the same results) or how the OS itself is handling these drives.

 

To test this theory, I decided to pass through the drives bare metal to a VM and measure their performance in a Windows environment using NTFS with CrystalDiskMark sequential. The benchmark results show that in this environment, the drives perform to their full potential whereas in the UnRAID environment they are quite far off. Below are the comparisons in read speeds:

 

Sabrent Rocket in UnRAID: 

 

 

 

Sabrent Rocket in Windows VM:

 

 

Samsung 980 Pro in UnRAID:

 

 

 

Samsung 980 Pro in Windows VM:

 

 

Quite frustrating. I know that unRAID is not really built for this but I can't really see why the OS is holding back performance so much.

 

Edited March 30, 2021 by Outcasst

[mikeyosm]

That's interesting. I just upgraded to PCIE4 motherboard and noticed the same thing. Got a Sabrent Rocket 4.0 2tb in my x570s and benching in UNRAID is in line with your speeds.

Have you figured out a fix yet?

[Outcasst]

On 10/27/2021 at 12:28 PM, mikeyosm said:

That's interesting. I just upgraded to PCIE4 motherboard and noticed the same thing. Got a Sabrent Rocket 4.0 2tb in my x570s and benching in UNRAID is in line with your speeds.

Have you figured out a fix yet?

Hi!

 

Unfortunately I never figured this out. I ended up moving to Windows Server which (for me anyway) is working much better for raw performance with NVME drives. Copying files between shares on the same SSD is literally twice as quick as when using unRAID. And when you're transferring 200+ GB at a time, that becomes valuable.

 

If this was ever addressed in unRAID (which I doubt, it's a very niche use case) I'd move back in a heartbeat.