erikhume

Great point - I did not think about that. Good to know the performance difference would be greater in a multi-drive array.

Hello, I am new to the forum and am building a new home server to replace my Synology DS-918+. I originally started with a ProxMox setup using a TrueNAS Scale VM and an Ubuntu server VM, which worked but was a little too complex for my level. I started watching some YouTube videos on Unraid and really began to admire the customization options along with the great community support. I really want a system faster than the Synology, though, so I have struggled with the transfer speeds of Unraid. One area I have had trouble finding objective information is read and write performance comparisons between the traditional Unraid array and a ZFS pool. Since I don't have any critical data on my new server, I thought now would be a good time to experiment. I want to share these results for the benefit of others who may be curious. Server hardware: Ryzen 3600X CPU, 32GB DDR4 3200 ECC RAM; 10GBE NIC, GTX 1660 Super GPU; 2x 1TB Samsung 980 Pro NVME drives (both in PCIE4 m.2 slots); 4x 8TB Seagate Ironwolf HDDs Testing PC: Ryzen 5800X, 10 GBE NIC; 2TB Samsung 980 Pro NVME, Windows 11 Pro Test file: 42,320.41 MB MKV file Test 1: Array, xfs, no cache, 1 8TB drive, 1 8TB parity disk, turbo write enabled: To server - 6m21.8s, 110.79 MB/s From server - 2m51.3s, 247.49 MB/s Test 2: Array, xfs, single 8TB drive, no cache, no parity To server - 2m40.1s, 264.5 MB/s From server - 2m46.3s, 254.94 MB/s Test 3: NVME cache, ZFS mirror, 4GB ARC: To server - 42.6s, 984.2 MB/s From server - 39.5s, 1085.14 MB/s Test 4: 2x 8 TB HDD, ZFS mirror, 4GB ARC: To server - 3m12.7s, 219.28 MB/s From server - 2m42.6s, 259.63 MB/s Test 5: 2x 8TB HDD, ZFS mirror, 8GB ARC: To server - 3m04s, 230 MB/s (Did not run from server) Test 6: 3x 8TB HDD, RaidZ1, 4GB ARC: To server - 1m44.7s, 403.5 MB/s From server - 1m36.3s, 440.84 MB/s Test 7: 3x 8TB HDD, RaidZ1, 8GB ARC: To server - 1m41s, 419.01 MB/s From server - 1m36.4s, 440.84 MB/s My comments and conclusions: On the HDD ZFS mirror, with the 4 GB ARC, when writing to the server the performance was odd - there was a fast write until the ARC was filled, then the transfer took a sawtooth pattern, going all the way down to 0 MB/s, then up to 300-400 MB/s, then back to zero, repeat. When I upped the ARC to 8GB, the sawtooth pattern disappeared and the transfer rate was steady, but the actual performance gain was only 5%. I did not observe the "sawtooth" issue nearly as much with the RaidZ1 configuration with a 4GB ARC, and it was not seen at all on the NVME drives. Regardless of ARC size, I was getting the same write speed with a ZFS mirror as I was without a parity drive. There appears to be no real difference in read speed between an array and a ZFS mirror, at least when it comes to a large file. I wonder if a group of small files would be different. The real speed increase seems to come from a RaidZ1 configuration, with almost 4x write speeds and 1.8x read speeds. I imagine with more drives in that pool, it would be even faster. I hope this is of help to someone in a similar situation trying to choose between ZFS and the traditional array. Any feedback is welcome!