I thought I share my rsync config. I still feel like Unraid as a Storage Server should provide a build in method to backup or sync its data to an other location, but configuring rsync gives you a lot of power, and once set up, it's just great.
I based my config and setup on the tutorial found here: https://lime-technology.com/forums/topic/52830-syncronize-servers-using-rsync-over-ssh-next-door-or-across-the-world/
And I made some adjustments, specially the option to watch the sync going on, and the notification that works with the build in unraid notification. So on "errors" you will get a E-Mail notification if your unraid is setup for that.
The above tutorial covers the SSH setup and connect via key instead of login, and the basic rsync setup.
You can continue after that guide. What you need is the NerdPack Plugin, and there install "screen". It is used to run rsync in it own screen, giving you the option to pull the progress up in a console at any time.
The second one is optional but recommended, its the "User Scripts" plugin. It allows you to add own scripts and execute it via cron in your unraid GUI.
I created 2 scripts for my rsync:
The first one I called "rsync" and does the rsync job, and depending on its exit code, it gives the notification to the Unraid Notification Service.
#!/bin/bash
rsync -avu --bwlimit=4000 --numeric-ids --log-file=/mnt/user/Backup/rsync-logs/log.`date '+%Y_%m_%d__%H_%M_%S'`.log --progress --exclude 'C Sys' --timeout=600 -e "ssh -p 4222 -i /root/.ssh/DSStoDMS-key -T -o Compression=no -x " /mnt/user/Backup/Devices/
[email protected]:/volume2/Backup/Unraid_rsync
if [ $? -eq 0 ]
then
/usr/local/emhttp/webGui/scripts/notify -s "`hostname` to DMS Rsync Backup complete" -d "Sync compled. `date` `tail /mnt/user/Backup/rsync-logs/last.log`"
else
/usr/local/emhttp/webGui/scripts/notify -s "`hostname` to DMS Rsync Backup FAILED" -i "alert" -d "Sync compled. `date` `tail /mnt/user/Backup/rsync-logs/last.log`"
fi
Of course, the rsync command would need some adjustments.
If your Unraid server is setup to send you a E-Mail on errors, it will do that too if rsync fails!
The second one is for calling that script in a "screen" session.
#!/bin/bash
export SCREENDIR=/root/.screen
screen -dmS rsyncdms /boot/config/plugins/user.scripts/scripts/rsync/script
You can pull it up as long as it is running with
screen -r
from a console. You can just use the build in one from the WebGUI.
Attention, if you name the first script different than "rsync", you have to adjust this location.
Now just set a schedule for the second script. Watch its progress with "rsync -r"
I hope this can help somebody. Im not a rsync expert, so if anyone is around with more experience has some input, its very welcome.
And I have a questions myself:
I have large files to transfer, sometimes a lot of new data comes up at once. So if I have to transfer >400GB, it needs more than 24h, and the rsync backup script (which is scheduled to run daily) is started again. It starts with the files that is currently transfering in the first instance of rsync. It works, but it will transfer that big file twice. Should I check when running the script if rsync is already running? Or does running rsync with --partial do any good or bad here?