There doesn't seem to be a universally agreed upon way to transfer all data from an old unRAID box to a new unRAID box.

 

I built a new server and want to move all of my data from they old server to the new. I want it to be a 1:1 copy keeping all permissions and file modified/created dates. I want to do it all over the network, not removing drives from the old server.

 

I've searched the forums for hours, looked on the wiki and there is no clear cut command to run. I do know that I'll be using rsync (unless something else is better suited but from my understanding rsync is the answer)

 

Found a couple examples throughout the forum.

 

I use rsynch, like so:

rsync -avAXP -e ssh root@source-ip-address:/mnt/user/Source-share/* /mnt/user/Dest-share

/usr/bin/rsync -avzh /mnt/user/media [email protected]:/mnt/user/media

rsync -aruv /mnt/user/share1/dir1 192.168.1.xxx:/mnt/user/share1

rsync -a --progress  --dry-run /path/to/source/ /path/to/destination/

 

I'm not saying any of those examples are incomplete but each have a few different options, with a few more choices listed in their respective threads.

 

For example, one of the examples shows the use of -z and the other do not. Should -z be used on a LAN? (I assume no?)

 

Couple options I'm not sure if they should be used or would be helpful, ie:

-h is for human readable for file sizes

--stats give some file-transfer stats

--progress show progress during transfer

--log-file=FILE log what we're doing to the specified FILE

 

-s, --protect-args

This option sends all filenames and most options to the remote rsync without allowing the remote shell to interpret them. This means that spaces are not split in names, and any non-wildcard special characters are not translated (such as ~, $, ;, &, etc.). Wildcards are expanded on the remote host by rsync (instead of the shell doing it). [...]

 

 

 

I know those aren't critical but would be nice to have a coherent or logged output.

 

Since rsync has so many run options it would be helpful to have well setup command that everyone needing to do a large transfer could reference. Which would probably be good to put into the wiki.

 

Also, should I "push" the data or "pull" I know that it doesn't really matter but is there a preference?

 

Another point of discussion that always comes up if while doing a large initial transfer is it better to leave parity unassigned to increased write speed? Generally, it looks like the answer is yes. But now with turbo write, is having parity enabled with turbo write just as fast (generally speaking)?

 

With the newer server built and ready to go for a few days I haven't been able to start this bulk transfer. It would have been nice to just fill in a few parameters in the GUI or copy and paste a rysnc command and adjust a few parameters and let the process start days ago.

 

I honestly think it really should be easier to migrate from one unRAID server to another unRAID server, read via GUI.

_{I also think unRAID to unRAID sync/backup should be a built in feature in the GUI for many reasons but that's a topic for another thread.}

Anyone?

 

Maybe I've over thinking it but since I come from a windows world, running a large copy from command line I not sure the expected outcome during errors. Are they handled silently? Or will it just work perfectly (hard to believe)

I did this recently and used the "-avzh" options. My network isn't slow but figured it wasn't that big of a deal to compress the data.

 

Errors are handled quite well and I didn't have an issue except for a few times where I had to restart it. Picked up exactly where it was supposed to.

 

I also like using the "-h" to make the progress human readable along with "--progress" to get status as it's running.

 

 

If you're doing this with disks instead of user shares you shouldn't have any problem assuming the disks have enough capacity. When I did this with user shares I found that rsync created all the folders before transferring any files, with the result that it was going to try to write the files to specific disks that it had already created the folders on, and there wasn't enough capacity. I just stopped it when one disk would start to get full and then moved the empty folders that hadn't been written to yet to another drive then restarted the rsync so it could pick up where it left off.

Looks like I've run into the problem described by trurl.

 

Using rsync to copy to a user share that is set to allocation method of fill-up and a min free space of 200GB it just fills the drive entirely and fails.

 

Is there a reason rsync disregards the min free space??

Looks like I've run into the problem described by trurl.

 

Using rsync to copy to a user share that is set to allocation method of fill-up and a min free space of 200GB it just fills the drive entirely and fails.

 

Is there a reason rsync disregards the min free space??

I think the actual problem is split level. Split level take precedence over min free. If split level says don't split this folder, then the fact that rsync has already created the folders before copying the files determines in advance where the files should go even before any space has been used.

 

Also, consider how min free actually works. If a drive has more than min free, unRAID can write to it regardless of how large the file it is going to write is. When creating a file, it is not typically known how large the file will become, so that is not considered at all when beginning to write the file. This is why you set min free to be larger than the largest file you expect to write. Then if you write a file that makes the disk have less than min free, the write will complete, and subsequent writes will not use the drive because min free is no longer available.

You were right it was the split level.  I though I had it set right but I needed one more level deep.  The copies are filling up each drive to 200GB free then moving to the next disk.

 

Sent using Tapatalk