I have 9 disks for zfs format data disk and 2 parity disk for unraid array.When I read data from 9 data disks simultaneously through /mnt/diskx, the overall throughput is very low, only 300M/s, slightly higher than the throughput of a single disk (260M/s). At the same time, I can see that my 32-core 7d12 CPU is almost fully loaded in the background. By using the "top" command, I can see that processes like unraidd1 and unraidd2 are consuming a significant amount of CPU.

I have confirmed that my hard disk bandwidth is sufficient to support all my disks being fully loaded for reading simultaneously. During parity check, all 11 hard disks can operate at speeds above 250M/s.

I suspect that, at least in part, it's caused by the checksum verifications, I had already noticed this with btrfs, though zfs appear to be a little slower, this is with 3 simultaneous reads on a 15 year old 4 core Xeon:

 

ZFS:

 

BTRFS:

 

XFS:

 

I think your CPU should be able to handle more than 300MB/s, how much do you get for 3 reads?

1 hour ago, JorgeB said:

I suspect that, at least in part, it's caused by the checksum verifications, I had already noticed this with btrfs, though zfs appear to be a little slower, this is with 3 simultaneous reads on a 15 year old 4 core Xeon:

 

ZFS:

 

BTRFS:

 

XFS:

 

I think your CPU should be able to handle more than 300MB/s, how much do you get for 3 reads?

I test three disk，It more faster

  

9 disk read：

1 hour ago, JorgeB said:

I suspect that, at least in part, it's caused by the checksum verifications, I had already noticed this with btrfs, though zfs appear to be a little slower, this is with 3 simultaneous reads on a 15 year old 4 core Xeon:

 

ZFS:

 

BTRFS:

 

XFS:

 

I think your CPU should be able to handle more than 300MB/s, how much do you get for 3 reads?

I don't believe that checksum verifications are the main culprit in this situation. Nowadays, even single-core CPUs have computing power measured in GOPS (Giga Operations Per Second), and a data flow of less than 1G/s should not cause significant CPU usage. Additionally, based on the "top" command, it is evident that the CPU-consuming processes are kernel threads of Unraid. It is highly likely that Unraid has assigned a separate kernel thread for each disk to handle IO requests,miuse spin locks make that happen(more disk,slow thorughput,but higher cpu usage.

.

Edited January 30, 20242 yr by competent-bailout3425

It came to me that there's an easy way to test if the Unraid driver is affecting simultaneous performance, test the same devices in separate pools:

 

 

These are the same 3 devices as before, zfs formatted, and the difference is quite clear, 785MB/s total vs 450MB/s when used in the array, so looks like the Unraid driver is limiting simultaneous read performance, and possibly the more devices you try to read from at the same time, the more it limits it.

 

This is not something that most users will run across, but it is a valid use case, I would suggest creating a bug report, it probably won't be considered high priority, but it's something that LT can try to optimize in the Unraid driver.