Read/Write Performance Comparison - Test Results

[erikhume]

Hello,

 

I am new to the forum and am building a new home server to replace my Synology DS-918+. I originally started with a ProxMox setup using a TrueNAS Scale VM and an Ubuntu server VM, which worked but was a little too complex for my level. I started watching some YouTube videos on Unraid and really began to admire the customization options along with the great community support. I really want a system faster than the Synology, though, so I have struggled with the transfer speeds of Unraid. One area I have had trouble finding objective information is read and write performance comparisons between the traditional Unraid array and a ZFS pool. Since I don't have any critical data on my new server, I thought now would be a good time to experiment. I want to share these results for the benefit of others who may be curious.

 

Server hardware: Ryzen 3600X CPU, 32GB DDR4 3200 ECC RAM; 10GBE NIC, GTX 1660 Super GPU; 2x 1TB Samsung 980 Pro NVME drives (both in PCIE4 m.2 slots); 4x 8TB Seagate Ironwolf HDDs

 

Testing PC: Ryzen 5800X, 10 GBE NIC; 2TB Samsung 980 Pro NVME, Windows 11 Pro

 

Test file: 42,320.41 MB MKV file

 

Test 1: Array, xfs, no cache, 1 8TB drive, 1 8TB parity disk, turbo write enabled:

To server - 6m21.8s, 110.79 MB/s

From server - 2m51.3s, 247.49 MB/s

 

Test 2: Array, xfs, single 8TB drive, no cache, no parity

To server - 2m40.1s, 264.5 MB/s

From server - 2m46.3s, 254.94 MB/s

 

Test 3: NVME cache, ZFS mirror, 4GB ARC:

To server - 42.6s, 984.2 MB/s

From server - 39.5s, 1085.14 MB/s

 

Test 4: 2x 8 TB HDD, ZFS mirror, 4GB ARC:

To server - 3m12.7s, 219.28 MB/s

From server - 2m42.6s, 259.63 MB/s

 

Test 5: 2x 8TB HDD, ZFS mirror, 8GB ARC:

To server - 3m04s, 230 MB/s

(Did not run from server)

 

Test 6: 3x 8TB HDD, RaidZ1, 4GB ARC:

To server - 1m44.7s, 403.5 MB/s

From server - 1m36.3s, 440.84 MB/s

 

Test 7: 3x 8TB HDD, RaidZ1, 8GB ARC:

To server - 1m41s, 419.01 MB/s

From server - 1m36.4s, 440.84 MB/s

 

My comments and conclusions:

 

On the HDD ZFS mirror, with the 4 GB ARC, when writing to the server the performance was odd - there was a fast write until the ARC was filled, then the transfer took a sawtooth pattern, going all the way down to 0 MB/s, then up to 300-400 MB/s, then back to zero, repeat. When I upped the ARC to 8GB, the sawtooth pattern disappeared and the transfer rate was steady, but the actual performance gain was only 5%. I did not observe the "sawtooth" issue nearly as much with the RaidZ1 configuration with a 4GB ARC, and it was not seen at all on the NVME drives. 

 

Regardless of ARC size, I was getting the same write speed with a ZFS mirror as I was without a parity drive.

 

There appears to be no real difference in read speed between an array and a ZFS mirror, at least when it comes to a large file. I wonder if a group of small files would be different.

 

The real speed increase seems to come from a RaidZ1 configuration, with almost 4x write speeds and 1.8x read speeds. I imagine with more drives in that pool, it would be even faster.

 

I hope this is of help to someone in a similar situation trying to choose between ZFS and the traditional array. Any feedback is welcome!

[trurl]

Test1 with single data and parity is a special case and Turbo is irrelevant.  Unraid array with more than one data drive will not write that fast though Turbo will help in that case.

[erikhume]

9 hours ago, trurl said:

Test1 with single data and parity is a special case and Turbo is irrelevant.  Unraid array with more than one data drive will not write that fast though Turbo will help in that case.

Great point - I did not think about that. Good to know the performance difference would be greater in a multi-drive array. 

[Yock]

While very much faster than the normal array, RaidZ1 seems to top out at around 1.2GB/s for me with my pool of 3 NVME SSDs and a Ryzen 3900x.

I know there are reasons i can't get more than that, just wanted to add to this thread in case people wanted to make a RaidZ1 for speed reasons.