Jump to content

Moving a large amount of data into unRAID


Recommended Posts

The time has come for me to move a large amount of data from my old freenas box to my new UNRAID box.


I have to move 13TB.


I have precleared the drives for 3 full cycles over the past week so my array is currently 18T 2x8t, 2x1t) with no parity or cache disks- yet.


The plan is to move over all the data and then add a parity drive. It's on order and will also be getting a 3-cycle full preclear to stress-test the drive.


All my dockers are set up and ready. I just now need to move (copy) the data.


I was thinking of using rsync or sftp,  both are fault tolerant due to compression. It's over a local 1gbps network so encryption is not required.


My freenas box can saturate the network in read mode (disks are striped) and has rsync server/client included.


What's the fastest way to do this? I don't want to have to keep watching it, I just want it to do the copy then maybe email me when done, or something... it would be useful if it auto-restarts in case there is network interruption etc. 



Link to comment

A quick one line script example:

rsync command && mail command if good || mail command if error

Basically, if your rsync (or sftp) command ends with a good return code then the good mail command runs. If it returns something other than good, it sends a error mail.


Not sure if freenas has a mail command and on my unraid box it doesn't. You may have to mount each server onto a VM or other box and sync that way.


Hope this helps.

Link to comment

Thanks. I didn't try this but I did manage to transfer 13T of data over the network successfully with rsync. 


I used 'screen' to allow multiple rsync sessions at the same time to xfer different folders etc.


Without a parity drive, the transfer rate was ~ 120MB/sec (full speed of the network), so i'm happy with that.



Link to comment


This topic is now archived and is closed to further replies.

  • Create New...