hennymugge

Hi Mgutt, Really loving your script and think it might become really useful for my usecase, being: backing up via rsync through ssh on port 8888 to an external server. I'm getting close but I do have some issues. The first being that the log file isn't uploaded to the server, but placed in the folder of the backup-file (in my case /var/backups/backup_script.sh) Using this script, rsync is starting as supposed but hasn't been able to finish yet due to a random broken pipe error... Any help would be greatly appreciated! #!/bin/bash # ##################################### # Script: rsync Incremental Backup v0.3 # Description: Creates incremental backups and deletes outdated versions # Author: Marc Gutt # # Changelog: # 0.3 # - rsync returns summary # - typo in notification corrected # - skip some rsync errors (defaults are "0" = skip on success and "24" = skip if some files vanish from the source while transfer) # - add timeout for backup renaming https://forums.unraid.net/topic/97958-rsync-incremental-backup/?tab=comments#comment-910188 # 0.2 # - use full path for source and destination # - backup multiple paths # - unraid notification on success is now optional # 0.1 # - initial release # # ######### Settings ################## source_paths=( "/media/data" ) backup_path="username@hostname:~/backup" days=14 # preserve backups of the last X days months=12 # preserve backups of the first day of the last X month years=3 # preserve backups of the first january of the last X years fails=3 # preserve the recent X failed backups notification=0 skip_errors=(0 24) # https://linux.die.net/man/1/rsync#:~:text=Exit%20Values rename_timeout=100 # ##################################### # # ######### Script #################### # make script race condition safe if [[ -d "/tmp/${0///}" ]] || ! mkdir "/tmp/${0///}"; then exit 1; fi; trap 'rmdir "/tmp/${0///}"' EXIT; # check user settings backup_root_path=$([[ "${backup_path: -1}" == "/" ]] && echo "${backup_path%?}" || echo "$backup_path") # loop through all source paths for source_path in "${source_paths[@]}"; do echo "Create backup of $source_path" backup_path="$backup_root_path" # check user settings source_path=$([[ "${source_path: -1}" == "/" ]] && echo "${source_path%?}" || echo "$source_path") # shorten the backup path if [[ $source_path == "/mnt/user"* ]]; then backup_path+="/Shares${source_path#'/mnt/user'}" echo "Backup path has been set to $backup_path" elif [[ $source_path == "/mnt"* ]]; then backup_path+="${source_path#'/mnt'}" echo "Backup path has been set to $backup_path" fi # new backup timestamp new_backup="$(date +%Y%m%d_%H%M%S)" # create directory tree as rsync is not able to do that (https://askubuntu.com/a/561239/227119) mkdir -p "${backup_path}/.${new_backup}" # create log file exec &> >(tee "${backup_path}/.${new_backup}/backup.log") # obtain most recent backup last_backup=$(ls -t "${backup_path}" | head -n1) # create incremental backup if [[ -n "${last_backup}" ]]; then echo "Create incremental backup ${new_backup} by using last backup ${last_backup}" rsync -av --stats --delete -e "ssh -p 8888 -i /home/username/.ssh/id_rsa" --link-dest="${backup_path}/${last_backup}" "${source_path}" "${backup_path}/.${new_backup}" else echo "Create full backup ${new_backup}" # create very first backup rsync -av --stats -e "ssh -p 8888 -i /home/username/.ssh/id_rsa" "${source_path}" "${backup_path}/.${new_backup}" fi rsync_status=$? job_name="$(dirname "$0")" job_name="$(basename "$job_name")" if [[ "${skip_errors[@]}" =~ "${rsync_status}" ]]; then if [ "$notification" = "1" ]; then /usr/local/emhttp/webGui/scripts/notify -i normal -s "Backup done." -d "Job $job_name:${backup_path}/${new_backup} successfully finished." fi # make backup visible rename_try=1 while true; do sleep 1 echo "Try #${rename_try} to make backup visible" mv "${backup_path}/.${new_backup}" "${backup_path}/${new_backup}" mv_status=$? if [[ $mv_status -eq 0 ]]; then break fi rename_try=$(($rename_try+1)) if [[ $rename_try -ge rename_timeout ]]; then /usr/local/emhttp/webGui/scripts/notify -i alert -s "Backup failed!" -d "Job $job_name:${backup_path}/${new_backup} failed because rename timeout has been reached!" break; fi done else /usr/local/emhttp/webGui/scripts/notify -i alert -s "Backup failed!" -d "Job $job_name:${backup_path}/${new_backup} failed (error ${rsync_status})!" fi # clean up ls -tA "${backup_path}/" | while read backup; do if [ "${backup:0:1}" = "." ]; then if [ "$fails" -gt "0" ]; then echo "Preserve failed backup: $backup" fails=$(($fails-1)) continue fi echo "Delete failed backup: $backup" rm -r "${backup_path}/${backup}" continue fi last_year=$year last_month=$month last_day=$day year=${backup:0:4} month=${backup:4:2} day=${backup:6:2} if [ "$last_day" = "$day" ] && [ "$last_month" = "$month" ] && [ "$last_year" = "$year" ]; then echo "Keep multiple backups per day: $backup" continue fi # preserve yearly backups if [ "$month" = "01" ] && [ "$day" = "01" ] && [ "$years" -gt "0" ]; then echo "Preserve yearly backup: $backup" years=$(($years-1)) continue fi # preserve monthly backups if [ "$day" = "01" ] && [ "$months" -gt "0" ]; then echo "Preserve monthly backup: $backup" months=$(($months-1)) continue fi # preserve daily backups if [ "$days" -gt "0" ]; then echo "Preserve daily backup: $backup" days=$(($days-1)) continue fi echo "Delete $backup" rm -r "${backup_path}/${backup}" done done