Backup dockers and their data from Cache drive

[ThatDude]

I now have a few Dockers (NZBget, Deluge, Owncloud, CouchPotato etc) setup and working perfectly from my SSD cache drive and I would like to take a backup.

 

My disk layout is like this:

 

/mnt/cache/appdata/[docker-app-name]  <- settings for each Docker

/mnt/cache/downloads/[docker-app-name]  <- files generated by dockers such as NZBget and Deluge

/mnt/cache/docker.img

 

Everything is running exclusively from my SSD cache drive, it's fast and allows my unRAID array to stay spun down most of the time.

 

However I'd like to take a manual backup of my docker related settings incase the cache drive should fail.

 

If I was to execute the mover script to clear the cached shares, stop all of the Dockers and SSH into my server would a command like this do the job?

 

tar -czvf /mnt/user0/backup/cache-drive.tar.gz /mnt/cache/

 

And if I need to replace the cache drive, would this put everything back as of the backup date?

 

tar -xzvf /mnt/user0/backup/cache-drive.tar.gz /mnt/cache/

 

[ninthwalker]

Here is the method I use to backup my cache drive once a week to an unraid share.

The unraid share is then backup to to crashplan daily.

 

1) added this to my go file:

# copy cron job for cache backups to cron.weekly
cp /boot/custom/cache_weekly_backup.sh /etc/cron.weekly
chmod +x /etc/cron.weekly/cache_weekly_backup.sh

 

2) add this shell script to my /boot/custom directory:

#!/bin/bash

#Stop plugin services located in /etc/rc.d/
# enter in plugins to stop here, if any

#stop dockers
docker stop $(docker ps -a -q)

#Backup cache via rsync
date >/var/log/cache_backup.log
/usr/bin/rsync -avrtH --delete /mnt/cache/.Programs/ /mnt/disk4/Backups/unRAIDweeklybackups/Cache/dotPrograms >>/var/log/cache_backup.log

#Start plugin services
# enter in plugins to start here, if any

#start dockers
/etc/rc.d/rc.docker start

 

 

 

 

 

[NAS]

There is another thread discussing this which is where this code is being discussed.

 

keep in mind backing up docker containers is far les important that the config and data files which this is not doing.

[ThatDude]

Thanks NAS. So the Docker.img file is not important because I can just download all of the Dockers again right? But if I do restore the Docker.img will unraid recognise and use it?

 

I wanted to restore my cache drive and have it just work without having to reload all of my dockers and their configs.

 

I also wanted to take nightly backups of the /mnt/cache/appdata directory, I figured I could somehow prefix the date to my tar.gz files in my backup script, the resulting files are very small and it'll give me several restore points.

 

Thanks for your script, it's an education for a beginner like me. One a related question, is it possible to stop/start plugins (e.g openvpn) from a script? I store plugin data in the same directory.

[ninthwalker]

The script i mentioned above, which is also talked about in another thread (forget the link, search) does back up all of my data like NAS said is needed.

 

.Programs for me is my 'appdata' where things like /plex and /sickbeard etc, live.

It is more important to get all of that than the docker.img as that can be rebuilt more easily.

 

I do however back up my .Docker/docker.img separately as well though.

 

Plugins can also be stopped via their rc.d command usually depending on the plugin and added to the same script if needed.

I have everything in dockers now, so I don't stop any plugins.

[ThatDude]

I only have openVPN as a plugin, it's a practically perfect implementation IMHO and I can't see a reason to Dockerise (is that a word?) it.

 

I amended the script slightly to meet my needs:

 

#Stop plugin services located in /etc/rc.d/
# enter in plugins to stop here, if any

#stop dockers
docker stop $(docker ps -a -q)

#Backup cache compressed archive
tar -czvf /mnt/user0/backup/cache-drive-appdata_$(date +%y%m%d).tar.gz /mnt/cache/appdata/

#Start plugin services
# enter in plugins to start here, if any

#start dockers
/etc/rc.d/rc.docker start

 

This seems to be working fine and the compressed archives also allow me to restore to a given date.

[mr-hexen]

This fails to re-start the dockers if they are not set to auto-start, and even still would you not need to stop the docker service in order to restart docker at the bottom and have it autostart the dockers?

[sparklyballs]

I'm no scripting guru, but i do know that it would be better to use

docker stop $(docker ps -q)

and not

docker stop $(docker ps -a -q)

 

the -a switch lists all containers while without the -a only lists running containers.

 

since you can list only containers that are running it should be possible to put the list in a variable array and stop them using the variable and more importantly use the same list to restart them when the backup is completed.

[roland]

Thanks for this script.

 

I used the V5 version of this and it worked fine.

Unfortunately I did not change it when I started to move to dockers and it has locked up my system last night.

 

Here my "enhanced" code to only stop running containers and only restarting the ones I stopped.

What options should I give the docker start command? What are the defaults unRAID uses when starting containers?

 

As I had to "foce" stop my tower this morning it is now doing a parity check.

Therefore I have only run the script with the actual rsync command commented out.

Any improvements, comments are very welcome. (Error handling still to come...)

 

#!/bin/bash

LogFile=/var/log/cache_backup.log
BackupDir=/mnt/disk1/Backup/unRAID_cache

echo `date` "Starting cache drive backup to " $BackupDir >> $LogFile

#Stop plugin services located in /etc/rc.d/
# enter in plugins to stop here, if any
# /etc/rc.d/rc.plexmediaserver stop >> $LogFile


#stop dockers

  # find running docker containers
  declare -a Containers=(`docker ps -q`)

  # stop running containers
  for Cont  in "${Containers[@]}"
  do
    echo `date` " Stopping Container: " $Cont >> $LogFile
    docker stop $Cont >> $LogFile
  done


#Backup cache via rsync

/usr/bin/rsync -avrtH --delete /mnt/cache/ $BackupDir  >> $LogFile

## RESTORE
## /usr/bin/rsync -avrtH --delete  $BackupDir  /mnt/cache/

#Start plugin services
# enter in plugins to start here, if any
# /etc/rc.d/rc.plexmediaserver start  >> $LogFile


#start dockers previousy stopped

  for Cont  in "${Containers[@]}"
  do
    echo `date` " Starting Container: " $Cont >> $LogFile
    docker start $Cont >> $LogFile
  done


echo `date` "backup Completed " $BackupDir >> $LogFile

# send notification
/usr/local/sbin/notify -i normal -s "Cach Drive Backup Completed" -d " Cache Drive Backup completed at `date`"

 

EDIT: The parity check finished so I gave this a whirl. All running dokers stopped before the backup and the same dockers got restarted afterwards. So all looks good.

 

EDIT2: added full path to notify

[MrLondon]

hi there,

 

I tried the script of Roland and created the folder on Disk1 before running the script, however after many hours the script seems to have gotten stuck and afterwards my Kodi database was missing. So now I have to import all the entries again. This is now the 2nd time in 24 hours that I lost my database, this script was meant to avoid losing the databases. Any idea how this could have happened? Was that because you have the delete part of the script?

 

Thanks

 

Ingo

syslog.zip

[roland]

Sorry to hear,

 

I am running this script weekly without any problems.

It does not actually remove anything,but it does shut down the dockers.

Does the database usually "survive" a docker restart?

Is it possible your database got moved by the mover from cache to disk?

 

Regards

  Roland

[krakent3ch]

hi there,

 

I tried the script of Roland and created the folder on Disk1 before running the script, however after many hours the script seems to have gotten stuck and afterwards my Kodi database was missing. So now I have to import all the entries again. This is now the 2nd time in 24 hours that I lost my database, this script was meant to avoid losing the databases. Any idea how this could have happened? Was that because you have the delete part of the script?

 

Thanks

 

Ingo

 

I have been using this script without running into any issues.

[bmdegraaf]

Used the script and activated a weekly cron to run it. All worked except I received the following email error message:

cron for user root /usr/bin/run-parts /etc/cron.weekly 1> /dev/null

/etc/cron.weekly/backup-appdata.sh: line 50: notify: command not found

 

This corresponds with the following entry in the script:

# send notification

notify -i normal -s "Appdata Backup Completed" -d " Appdata Backup completed at `date`"

[roland]

Thanks,

I actually had the same issue.

Somehow cron does not seem to have the $PATH when it is executed.

 

Change the line to

/usr/local/sbin/notify -i normal -s "Appdata Backup Completed" -d " Appdata Backup completed at `date`"

and it should work. (updated the script above too)

 

 

[bmdegraaf]

Thanks for the swift reply. Awaiting next week to see if the cron works correctly...

[JimPhreak]

Here is the method I use to backup my cache drive once a week to an unraid share.

The unraid share is then backup to to crashplan daily.

 

1) added this to my go file:

# copy cron job for cache backups to cron.weekly
cp /boot/custom/cache_weekly_backup.sh /etc/cron.weekly
chmod +x /etc/cron.weekly/cache_weekly_backup.sh

 

2) add this shell script to my /boot/custom directory:

#!/bin/bash

#Stop plugin services located in /etc/rc.d/
# enter in plugins to stop here, if any

#stop dockers
docker stop $(docker ps -a -q)

#Backup cache via rsync
date >/var/log/cache_backup.log
/usr/bin/rsync -avrtH --delete /mnt/cache/.Programs/ /mnt/disk4/Backups/unRAIDweeklybackups/Cache/dotPrograms >>/var/log/cache_backup.log

#Start plugin services
# enter in plugins to start here, if any

#start dockers
/etc/rc.d/rc.docker start

 

With regard to this script, is there a way to schedule this to run say weekly at 5am Monday mornings for example?

[MyKroFt]

would you want to also stop the docker service?  Otherwise you can not backup the docker.img file while it is running?

 

Myk

 

[JimPhreak]

would you want to also stop the docker service?  Otherwise you can not backup the docker.img file while it is running?

 

Myk

 

Yes I would stop the docker service before running the backup.

[trurl]

Here is the method I use to backup my cache drive once a week to an unraid share.

The unraid share is then backup to to crashplan daily.

 

1) added this to my go file:

# copy cron job for cache backups to cron.weekly
cp /boot/custom/cache_weekly_backup.sh /etc/cron.weekly
chmod +x /etc/cron.weekly/cache_weekly_backup.sh

 

2) add this shell script to my /boot/custom directory:

#!/bin/bash

#Stop plugin services located in /etc/rc.d/
# enter in plugins to stop here, if any

#stop dockers
docker stop $(docker ps -a -q)

#Backup cache via rsync
date >/var/log/cache_backup.log
/usr/bin/rsync -avrtH --delete /mnt/cache/.Programs/ /mnt/disk4/Backups/unRAIDweeklybackups/Cache/dotPrograms >>/var/log/cache_backup.log

#Start plugin services
# enter in plugins to start here, if any

#start dockers
/etc/rc.d/rc.docker start

 

With regard to this script, is there a way to schedule this to run say weekly at 5am Monday mornings for example?

Skip part 1 of the instructions about editing your go file and instead create your own .cron file and put it in /boot/config/plugins. I have /boot/config/plugins/cache_backup/cache_backup.cron

# Weekly cache backup
0 0 5,12,19,26 * * /boot/scripts/cache_backup.sh &> /dev/null

This runs the script /boot/scripts/cache_backup.sh at

0 minutes

0 hours

5,12,19,26 day of month

*(all) months

*(all) day of week

 

For your example, you could use

0 5 * * 1

 

See here for more on crontab syntax

 

would you want to also stop the docker service?  Otherwise you can not backup the docker.img file while it is running?

 

Myk

 

Yes I would stop the docker service before running the backup.

The quoted script already stops docker before backup and restarts it after.

[JimPhreak]

Thanks trurl, VERY helpful!

[MyKroFt]

Here is the method I use to backup my cache drive once a week to an unraid share.

The unraid share is then backup to to crashplan daily.

 

1) added this to my go file:

# copy cron job for cache backups to cron.weekly
cp /boot/custom/cache_weekly_backup.sh /etc/cron.weekly
chmod +x /etc/cron.weekly/cache_weekly_backup.sh

 

2) add this shell script to my /boot/custom directory:

#!/bin/bash

#Stop plugin services located in /etc/rc.d/
# enter in plugins to stop here, if any

#stop dockers
docker stop $(docker ps -a -q)

#Backup cache via rsync
date >/var/log/cache_backup.log
/usr/bin/rsync -avrtH --delete /mnt/cache/.Programs/ /mnt/disk4/Backups/unRAIDweeklybackups/Cache/dotPrograms >>/var/log/cache_backup.log

#Start plugin services
# enter in plugins to start here, if any

#start dockers
/etc/rc.d/rc.docker start

 

With regard to this script, is there a way to schedule this to run say weekly at 5am Monday mornings for example?

Skip part 1 of the instructions about editing your go file and instead create your own .cron file and put it in /boot/config/plugins. I have /boot/config/plugins/cache_backup/cache_backup.cron

# Weekly cache backup
0 0 5,12,19,26 * * /boot/scripts/cache_backup.sh &> /dev/null

This runs the script /boot/scripts/cache_backup.sh at

0 minutes

0 hours

5,12,19,26 day of month

*(all) months

*(all) day of week

 

For your example, you could use

0 5 * * 1

 

See here for more on crontab syntax

 

would you want to also stop the docker service?  Otherwise you can not backup the docker.img file while it is running?

 

Myk

 

Yes I would stop the docker service before running the backup.

The quoted script already stops docker before backup and restarts it after.

 

Any script that is using the docker ps method is not stopping the docker service that will unmount the .img file as well.  Only /etc/rc.d/docker stop will do that.

 

Myk

[trurl]

would you want to also stop the docker service?  Otherwise you can not backup the docker.img file while it is running?

 

Myk

 

Yes I would stop the docker service before running the backup.

The quoted script already stops docker before backup and restarts it after.

 

Any script that is using the docker ps method is not stopping the docker service that will unmount the .img file as well.  Only /etc/rc.d/docker stop will do that.

 

Myk

You are correct. Just had a look at the script I am using. Apparently got it from some other thread. It doesn't do it either, but I do have a docker.img in my backup so it seems to have copied anyway.

 

Not sure I would ever want to restore that docker.img or even if you could. Might be better to just start over with it and pull them all again.

[sparklyballs]

would you want to also stop the docker service?  Otherwise you can not backup the docker.img file while it is running?

 

Myk

 

Yes I would stop the docker service before running the backup.

The quoted script already stops docker before backup and restarts it after.

 

Any script that is using the docker ps method is not stopping the docker service that will unmount the .img file as well.  Only /etc/rc.d/docker stop will do that.

 

Myk

You are correct. Just had a look at the script I am using. Apparently got it from some other thread. It doesn't do it either, but I do have a docker.img in my backup so it seems to have copied anyway.

 

Not sure I would ever want to restore that docker.img or even if you could. Might be better to just start over with it and pull them all again.

 

i'm not sure you'd need to back up the docker.img file, any dockers that aren't saving information outside the container are badly designed.

[shooga]

So based on what MyKroFt said, wouldn't it be better to use this?

 

#!/bin/bash

#stop dockers
/etc/rc.d/rc.docker stop

#Backup cache via rsync
date >/var/log/cache_backup.log
/usr/bin/rsync -avrtH --delete /mnt/cache/.Programs/ /mnt/disk4/Backups/unRAIDweeklybackups/Cache/dotPrograms >>/var/log/cache_backup.log

#start dockers
/etc/rc.d/rc.docker start

[interwebtech]

FYI this script fails to run due to changes in allowed paths for commands. Hopefully one of our resident experts can update it to work again.