Syncing multiple unraid servers?

[hermy65]

Probably not the right place to post this but im not sure where else to put it.

 

I have 2 unraid servers, one is my main machine and the other is more of an archive machine located at another location that has irreplaceable data. Im looking for a way to keep the two machines in sync so that if something new gets added to a archive specific directory on my production machine it can sync over to the archive machine. Any suggestions on how to automate this whole thing?

[ashman70]

Although I have no experience with it or doing this, I believe Rsync is what you want.

[master.h]

I actually do this very thing myself. For a long time I had both my main and backup server in the same house (literally a foot away from each other) so a simple rsync worked just fine. However now that I've moved my backup server to an offsite location, I use a docker called Resilio. Resilio on the primary server indexes whatever folders I choose to pass through and copies them to my backup server's instance of Resilio, again to whatever folders I choose. It took a bit to set up, and the initial index takes a while (especially if you have tons of file, pictures for example) but now that it's set up it works really well.

[Hoopster]

You could also try the Syncthing docker.  I have used it between two servers and it works; however, for my needs, I prefer simple scripted rsync commands.  Since I only care about the data going one way (main server to backup) and I am only copying new files from six users shares on one server to the same user shares on another rather than keeping things in sync (not removing deleted files from backup), rsync works great.  The scripts turn on my backup server via IPMI, copy the files and then turn off the server.  Eventually, I may move the backup offsite, but,  can see the thread below about how one user manages copying data between servers over SSH via the Internet.  I modified his scripts for my purposes.

 

 

Edited November 3, 2017 by Hoopster

[AndrewT]

Since your secondary (backup) server is at a different location, that makes me wonder if it's usually powered off? If so, you might find it easiest to just create a cron job to execute a simple shell script that involves wakeonlan and rsync.

 

For now, I open a `screen -S share_name` and rsync one share, open another screen and do a second share. I've asked for guidance on the best rsync options to use but didn't seem to get much of an answer, perhaps because it might be more tailored to each person's needs. For example, how you handle symbolic links.

 

This is what I use once in screen on the primary (newer server) to backup `rsync --verbose --human-readable --recursive --update --copy-links --copy-dirlinks --times --perms --xattrs --owner --group --devices --specials --prune-empty-dirs --stats --log-file=/mnt/cache/rsync-logfile-ArrayTV.txt /mnt/user/TV/* root@<Backup_Server_IP>:/mnt/user/TV`