ZFS Array transfers very slow

[905jay]

thanks for all the help @MAM59 and @JorgeB

Just to round things off...

 

The official suggestion from @MAM59 is that the unraid array should NOT be ZFS and should remain as XFS (or BTRFS if you're brave)

I will begin the arduous process or converting all my array disks back to XFS.

 

I have turbo write enabled, and in the Disk Settings, I have it set to reconstruct write

I will perform the following actions.

1. Remove 1 parity disk
2. Allocate the removed parity disk to the array as Disk 5 with XFS
3. Let parity sync complete
4. Evacuate ZFS Disk1 to XFS Disk5
5. Format Disk 1 to XFS
6. Continue the disk evacuation and format process until all array disks are back to XFS

 

Aside from this painfully long process is there anything else I should do or tune?

Am I missing anything? 

Would there be a faster way for me to complete the process of moving all that data around and reformatting disks and moving the data back onto them?

 

Right now my transfer speeds have dropped to 15mb/s. This is going to be an extremely painful process....

 

Any suggestions? 

Edited August 29, 2023 by 905jay
added context and will mark this as the solution per @mam59 recommendations

[JorgeB]

Array to array copy/move (with parity) is always extra slow, you'd need to unassign parity for speed to improve a lot, after the conversion is done try turbo write for much better writing speeds a the expanse of all disks spinning up for writes.

[JonathanM]

5 hours ago, 905jay said:

Evacuate ZFS Disk1 to XFS Disk5

Copy instead of move will speed things up considerably, since move is a copy followed by a delete, it thrashes parity extra hard as the deletes are processed on the source disk at the same time the new data is accounted for on the destination disk.

 

Since you will be formatting the source disk anyway, forgo the move and just copy. You also can compare the result to be sure the copy is complete before doing the format if desired.

[MAM59]

I guess, he can't live without parity... Anyway, better have a full external backup (turned off when done) and get rid of all parity drives during the copy.

You can restore parity after all is back to xfs. It wont take longer than the extra time that would be spend during the copy / move process.

 

If copy or move is your choice, both have advantages and disadvantages. I prefer move mainly because you can then see which files are done already and where it will start again if you cancel because of any reason.

 

One more hint: UNBALANCE worked sometimes "strange" to me when I've ordered it to move full (very full) disks. Although I've told it "move everything to disk 5" it created a long list (aka "plan") what to do and the last few dozen entries were WRONG (wrong target drive, never found out how this could happen). I've ended up to plan only about 10000 files (a few dirs) and repeated the moves until the disk was totally copied (had the nice side effect that a single plan did not take so long, a full disk was expected to run over 8 days here and I had 4 of them...). This is just a warning. I suggest you create your normal plan but before you hit the "doit" button, scroll down the list to the last entry and check if it is still what you want. Then you can run the job.

 

 

Edited August 30, 2023 by MAM59