December 4, 201411 yr The title says it all, I'm looking for a way to schedule an automatic nightly backup of my appdata folder on my cache drive to a backup share in order to have it saved to the array in the event of or drive failure. I was looking at this thread as a starting point http://lime-technology.com/forum/index.php?topic=31246.0 it's in the 5.x thread and looks like its the right direction, I guess it would just need to be modified to stop and start the docker service to work for me. I have zero scripting skill so if anyone can let us know how you're backing up your appdata directory that would be great!
January 19, 201511 yr I am also looking to do this. Any help from the scripting gurus is greatly appreciated!
January 20, 201511 yr Nevermind, I think I figured it out. For anyone who is interested I used this script and set it to run daily in my Go file. You will need to modify your directories accordingly. I have my appdata on a disk mounted with SNAP outside of the array at /mnt/disk/appdisk/appdata/. My docker img file is under /mnt/disk/appdisk/docker/ Also, rsync does not like Plex and will give you a ton of "failed to set times" errors if you backup to a user share. However, it works fine if you backup directly to a disk share, which is why the script below references disk1 instead of my Backup user share. Script: #!/bin/bash #Stop Docker Apps docker stop CouchPotato docker stop CrashPlan docker stop Deluge docker stop MariaDB docker stop NzbDrone docker stop NZBGet docker stop PlexMediaServer docker stop Syncthing #Backup Docker App data via rsync date >/var/log/docker_app_backup.log date >/var/log/docker_img_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/appdata/ /mnt/disk1/Backup/appdata >>/var/log/docker_app_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/docker/ /mnt/disk1/Backup/docker >>/var/log/docker_img_backup.log #Start Docker Apps docker start CouchPotato docker start CrashPlan docker start Deluge docker start MariaDB docker start NzbDrone docker start NZBGet docker start PlexMediaServer docker start Syncthing Go file: # Set up daily Docker App backup cp /boot/custom/backup.sh /etc/cron.daily This was pretty simple, but I hope someone else finds this useful.
January 20, 201511 yr Nevermind, I think I figured it out. For anyone who is interested I used this script and set it to run daily in my Go file. You will need to modify your directories accordingly. I have my appdata on a disk mounted with SNAP outside of the array at /mnt/disk/appdisk/appdata/. My docker img file is under /mnt/disk/appdisk/docker/ Also, rsync does not like Plex and will give you a ton of "failed to set times" errors if you backup to a user share. However, it works fine if you backup directly to a disk share, which is why the script below references disk1 instead of my Backup user share. Script: #!/bin/bash #Stop Docker Apps docker stop CouchPotato docker stop CrashPlan docker stop Deluge docker stop MariaDB docker stop NzbDrone docker stop NZBGet docker stop PlexMediaServer docker stop Syncthing #Backup Docker App data via rsync date >/var/log/docker_app_backup.log date >/var/log/docker_img_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/appdata/ /mnt/disk1/Backup/appdata >>/var/log/docker_app_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/docker/ /mnt/disk1/Backup/docker >>/var/log/docker_img_backup.log #Start Docker Apps docker start CouchPotato docker start CrashPlan docker start Deluge docker start MariaDB docker start NzbDrone docker start NZBGet docker start PlexMediaServer docker start Syncthing Go file: # Set up daily Docker App backup cp /boot/custom/backup.sh /etc/cron.daily This was pretty simple, but I hope someone else finds this useful. Couple questions about rsync your code. rsync -avrtH --delete First: I am confused as to why you've specified -a (archive mode) and also -r (recursive) and -t (preserve modification times) when archive mode already includes -r and -t, -a is exactly the same as individually to specifying the following options -rlptgoD (recursive, copy symlinks as symlinks, preserve permissions, preserve mod time, preserve group, preserve owner, preserve device files, and preserve special files). Second: Does it really need to be verbose (-v) since you are running it daily and in the background? Is this for logging purposes? Wouldn't it be better have rsync make a log file (--log-file=FILENAME) in that case? Third: Are Hard-links (-H) really important? Does it break things if you don't preserve Hard-links? I legitimately want to know more about this and what should or should not be specified to make a good Rsync backup. Question about using Rsync this way.... won't running Rsync Daily with --delete prevent you from being able to roll back more than 24 hours? In the case of a disk dying that should be ok, but in the case of file system corruption or non disk failure events that cause errors this means you have to catch it within 24 hours or your backup will not be able to roll back... right? Can Rsync be used to create a “Time-Machine” like backup where you can roll back to more than one earlier state? If yes what should / should not be specified.
January 20, 201511 yr Good points. Frankly I am by no means an rsync expert so I hope someone else can chime in. I just modified a script from someone else on the 5.x support forum so it works with docker. I didn't tinker with the arguments.
February 8, 201511 yr Nevermind, I think I figured it out. For anyone who is interested I used this script and set it to run daily in my Go file. You will need to modify your directories accordingly. I have my appdata on a disk mounted with SNAP outside of the array at /mnt/disk/appdisk/appdata/. My docker img file is under /mnt/disk/appdisk/docker/ Also, rsync does not like Plex and will give you a ton of "failed to set times" errors if you backup to a user share. However, it works fine if you backup directly to a disk share, which is why the script below references disk1 instead of my Backup user share. Script: #!/bin/bash #Stop Docker Apps docker stop CouchPotato docker stop CrashPlan docker stop Deluge docker stop MariaDB docker stop NzbDrone docker stop NZBGet docker stop PlexMediaServer docker stop Syncthing #Backup Docker App data via rsync date >/var/log/docker_app_backup.log date >/var/log/docker_img_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/appdata/ /mnt/disk1/Backup/appdata >>/var/log/docker_app_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/docker/ /mnt/disk1/Backup/docker >>/var/log/docker_img_backup.log #Start Docker Apps docker start CouchPotato docker start CrashPlan docker start Deluge docker start MariaDB docker start NzbDrone docker start NZBGet docker start PlexMediaServer docker start Syncthing Go file: # Set up daily Docker App backup cp /boot/custom/backup.sh /etc/cron.daily This was pretty simple, but I hope someone else finds this useful. Brilliant, will be using this, one quick question, is there anyway to specify when the job runs. I'd like it to run at about 7.00am for example.
February 9, 201511 yr Author OK, I'm having some trouble with this, I created the following script: #!/bin/bash #Stop Docker Apps docker stop CouchPotato docker stop HTPC-Manager docker stop NzbDrone docker stop PlexMediaServer docker stop plexWatch docker stop SABnzbd #Backup Docker App data via rsync date >/var/log/docker_app_backup.log #date >/var/log/docker_img_backup.log /usr/bin/rsync -avrtH --delete /mnt/user/appdata/ /mnt/user/backup/appdatabackup >>/var/log/docker_app_backup.log #/usr/bin/rsync -avrtH --delete /mnt/cache/docker.img /mnt/user/backup/docker >>/var/log/docker_img_backup.log #Start Docker Apps docker start CouchPotato docker start HTPC-Manager docker start NzbDrone docker start PlexMediaServer docker start plexWatch docker start SABnzbd Saved as backup.sh in my custom folder, I'm using notepad Windows 8.1 save as backup.sh, save as type all files, encoding ANSI I then run chmod 775 backup.sh running ./backup.sh to test or just backup.sh gives me the following "/bin/bash^M: bad interpreter: No such file or directory" I'm totally new to Linux and I'm not sure what I'm doing wrong
February 9, 201511 yr It sounds as if the file has windows style end-of-line sequences rather than Unix (Linux) style ones. You need to create the file in an editor that understands Linux end-of-line sequences. Alternatively you can put the file through the Linux dos2unix utility to convert the file to Linux style end-of-lines.
February 9, 201511 yr ...Saved as backup.sh in my custom folder, I'm using notepad Windows 8.1 save as backup.sh, save as type all files, encoding ANSI I then run chmod 775 backup.sh running ./backup.sh to test or just backup.sh gives me the following "/bin/bash^M: bad interpreter: No such file or directory" I'm totally new to Linux and I'm not sure what I'm doing wrong Never use notepad with files you want unRAID to understand. The ^M in your error message is typical of this problem. You must use a linux compatible text editor. Search for Notepad++
February 9, 201511 yr Author I have recreated the .sh file with notepad++ but I'm still getting the same error, anyone have any ideas?
February 9, 201511 yr I have recreated the .sh file with notepad++ but I'm still getting the same error, anyone have any ideas?
February 9, 201511 yr Author thanks, that worked and my script ran, one more question, the script is saving log file to /var, where is that in unraid? I'm not seeing in on the root of my flash, do I nee to create this directory?
February 9, 201511 yr thanks, that worked and my script ran, one more question, the script is saving log file to /var, where is that in unraid? I'm not seeing in on the root of my flash, do I nee to create this directory? The /var directory is in RAM. If you want the log file to survive a reboot, then you need to take action to have it copied to the flash drive when the system shuts down.
February 10, 201511 yr How do I browse to the var directory and view my logs? You can't get there over the network, at least not simply. You can "browse" from telnet or console with mc, or just use linux commands to get there.
February 14, 201511 yr This script can be simplified for just appdata: #!/bin/bash #Stop Docker Containers docker stop $(docker ps -a -q) #Backup Docker App data via rsync date >>/var/log/docker_appdate_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/appdata/ /mnt/disk1/Backup/appdata >>/var/log/docker_appdate_backup.log #Start Docker Containers /etc/rc.d/rc.docker start there is still quite a bit of room for improvement and the logging is kinda fudly but its a starter for 10
March 16, 201511 yr For my own curiosity and knowledge, is there a reason its preferable to use docker stop $(docker ps -a -q) over /etc/rc.d/rc.docker stop ?? Thanks!
March 16, 201511 yr For my own curiosity and knowledge, is there a reason its preferable to use docker stop $(docker ps -a -q) over /etc/rc.d/rc.docker stop ?? Thanks! The first only stops the docker containers, the second stops the containers and docker service.
May 1, 201511 yr When does cron.daily run on unRAID? I think it's set at 4:40 AM, but this can also be changed. This script can be simplified for just appdata: #!/bin/bash #Stop Docker Containers docker stop $(docker ps -a -q) #Backup Docker App data via rsync date >>/var/log/docker_appdate_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/appdata/ /mnt/disk1/Backup/appdata >>/var/log/docker_appdate_backup.log #Start Docker Containers /etc/rc.d/rc.docker start there is still quite a bit of room for improvement and the logging is kinda fudly but its a starter for 10 Question about this, I see docker stop $(docker ps -a -q) is used to stop docker containers without stopping the docker service, but then /etc/rc.d/rc.docker start is used to start the docker service (and I assume containers) again, is there a command to just start docker containers without having to issue the start command to the docker service?
May 1, 201511 yr When does cron.daily run on unRAID? I think it's set at 4:40 AM, but this can also be changed. This script can be simplified for just appdata: #!/bin/bash #Stop Docker Containers docker stop $(docker ps -a -q) #Backup Docker App data via rsync date >>/var/log/docker_appdate_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/appdata/ /mnt/disk1/Backup/appdata >>/var/log/docker_appdate_backup.log #Start Docker Containers /etc/rc.d/rc.docker start there is still quite a bit of room for improvement and the logging is kinda fudly but its a starter for 10 Question about this, I see docker stop $(docker ps -a -q) is used to stop docker containers without stopping the docker service, but then /etc/rc.d/rc.docker start is used to start the docker service (and I assume containers) again, is there a command to just start docker containers without having to issue the start command to the docker service? I think you can use the below code to stop/start the individual containers without starting the service. If auto-start is turned on the containers will restart with the rc.docker start command Nevermind, I think I figured it out. For anyone who is interested I used this script and set it to run daily in my Go file. You will need to modify your directories accordingly. I have my appdata on a disk mounted with SNAP outside of the array at /mnt/disk/appdisk/appdata/. My docker img file is under /mnt/disk/appdisk/docker/ Also, rsync does not like Plex and will give you a ton of "failed to set times" errors if you backup to a user share. However, it works fine if you backup directly to a disk share, which is why the script below references disk1 instead of my Backup user share. Script: #!/bin/bash #Stop Docker Apps docker stop CouchPotato docker stop CrashPlan docker stop Deluge docker stop MariaDB docker stop NzbDrone docker stop NZBGet docker stop PlexMediaServer docker stop Syncthing #Backup Docker App data via rsync date >/var/log/docker_app_backup.log date >/var/log/docker_img_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/appdata/ /mnt/disk1/Backup/appdata >>/var/log/docker_app_backup.log /usr/bin/rsync -avrtH --delete /mnt/disk/appdisk/docker/ /mnt/disk1/Backup/docker >>/var/log/docker_img_backup.log #Start Docker Apps docker start CouchPotato docker start CrashPlan docker start Deluge docker start MariaDB docker start NzbDrone docker start NZBGet docker start PlexMediaServer docker start Syncthing Go file: # Set up daily Docker App backup cp /boot/custom/backup.sh /etc/cron.daily This was pretty simple, but I hope someone else finds this useful.
May 1, 201511 yr I think you can use the below code to stop/start the individual containers without starting the service. If auto-start is turned on the containers will restart with the rc.docker start command Thanks that is good info. It shows how to stop / start individual containers which might be relevant if there is a docker you don't want to stop while the back up is created. My question is mostly this though, if our goal is to stop all the containers without stopping the docker service, why then restart the service isntead of just starting the containers. I did a bit more research myself and found this list of Docker Command lines and I think I've got a few more ideas now. 1) Should we be using docker start $(docker ps -a -q) to restart ALL containers instead of /etc/rc.d/rc.docker start which does start all the containers that are set to auto-start but also restarts the docker service as well. Is this done because docker start $(docker ps -a -q) would start all containers even those that aren't set to auto-start? I guess I might need to understand the auto-start system better to figure out what this really should be... 2) Should we really be using docker pause $(docker ps -a -q) and docker unpause $(docker ps -a -q) to suspend the containers processes but not actually stop / restart them. Is this better in anyway, or are there drawbacks that I am not thinking of here? Just trying to learn and therefor asking questions.
July 8, 201510 yr This has been extremely helpful on my end, and for anyone who didn't look up all the options rsync has one that is great, especially if you backup plex is --exclude-from. This lets you setup an exclude file, so for example, mine looks like: #!/bin/bash #Stop Docker Containers docker stop $(docker ps -a -q) #Backup Docker App data via rsync date >>/mnt/user/Backup/HADES/docker_app_backup.log /usr/bin/rsync -avrtH --exclude-from '/boot/custom/exclude.txt' --delete /mnt/cache/apps/ /mnt/user/Backup/HADES/cache/apps >>/mnt/user/Backup/HADES/docker_app_backup.log #Start Docker Containers /etc/rc.d/rc.docker start #Backup Flash Drive data via rsync /usr/bin/rsync -avrtH --delete /boot/ /mnt/user/Backup/HADES/flash >>/mnt/user/Backup/HADES/docker_app_backup.log in my exclude.txt I have this: Plex/Library/Application Support/Plex Media Server/Cache/Transcode Because I really don't care about backing up my synced content, and its also like 60GB of data, who cares if you have to re-sync, as long as your DB is backed up etc etc. *note, you don't need a file, you can just use --exclude but I figured I might have some more exclusions down the road.
Archived
This topic is now archived and is closed to further replies.