Confused XFS, BRTFS, or ZFS for a moving Unraid build.

[grumpy]

Current Unraid Server a 10 core Intel® Xeon® W-2155 CPU @ 3.30GHz with 16 GB ram, 4 16TB spinning rust and 4TB SSD cache
Moving to:
MB X399 AORUS XTREME-CF, AMD Ryzen Threadripper 2950X 16-Core @ 3500 MHz with 32 GB ram, 2 new 16TB spinning rust plus 4 current 16TB spinning rust, 
current 4TB SSD cache could add another 2TB SSD to cache
Both 10GB

Only reason of changing hardware is the case of the Dell does not support more drives. Also runs hot and does not support 6.12 Unraid (last I checked).

I tried ZFS on the replacement board Cpu and it works; but I'm not impressed. If I run more than one task to drives I get CPU_IOWAIT errors.
Caused by memory shortage (1/8) I'm sure as the CPU has room to spare and no other tasks running. So I do not understand why ZFS is so in demand
will it really be a benifit or just more headaches. I'm all for cutting edge but do not see the benifit.

My usage is media/document server with VM's and Docker containers running 24-7.

Is Mover really that slow, 9 hours about; in each direction: to move about 30GB from ZFS SSD to XFS HD, ZFS HD to ZFS SSD
Unbalance is slow but way faster then this. Network copy starts of great as expected to array but during the move it eventually times out at 0 bytes from Windows machine.

So do I stay with the devil I know XFS spinning rust, BRTFS SSD cache or move to ZFS Full or a Hybrid? I would like to achieve greater transfer speeds more akin to my
current Synology which uses BRTFS. 

If if I move away from XFS how do I go about that?

[itimpi]

When you tried ZFS was that in the array or a pool?   ZFS in the main array is know to be slow - it needs to be in a pool to get any decent performance from it.

[grumpy]

My experiment was in an array with each disk being a standalone ZFS drive.
Hard to do a pool with 3*3TB, 1*2TB, 2*1TB

 

I could do a pool with the original 4*16TB plus the 2*16TB, just not sure of the game plan needed and how to move the original data around. Obviously in place will not work as there would not be enough free HD's for any kind of a real pool.