Jump to content
Squid

[Plugin] CA Appdata Backup / Restore v2

218 posts in this topic Last Reply

Recommended Posts

 

@unRaide  I use this plugin and there is the ability to include running a "Custom Stop Script" which I use to do a few backups of additional things. It works out really well for me.

Share this post


Link to post
9 hours ago, unRaide said:

This might be a dumb question but I couldn’t find out how to search for it correctly in this thread 😁

 

Is anyone using this plugin to backup random non-appdata folders on their cache? Seems like it could be as easy as changing the source to mnt/cache and then using the "Excluded Folders" to deselect those I don't want to backup? Basically trying to backup the appdata folder and a few others that are only on the cache.

 

Everywhere in the plugin it refers specifically to the appdata folder so I wanted to make sure there isn’t anything specific to just that use case before trying to set this up.

Nothing says that you can't do that

Share this post


Link to post
1 hour ago, GHunter said:

 

@unRaide  I use this plugin and there is the ability to include running a "Custom Stop Script" which I use to do a few backups of additional things. It works out really well for me.

That sounds interesting, any chance you could share the script you're using for that.  

Share this post


Link to post
15 hours ago, Squid said:

Nothing says that you can't do that

Hi Squid, I’m trying to set this up but I’m getting the error msg below. Am I doing something wrong or is it not possible to backup all folders on my cache drive minus a few excluded ones?

 

 

 

68E35A61-E82B-46ED-8627-6C05DEA91483.jpeg

Share this post


Link to post
2 hours ago, unRaide said:

Hi Squid, I’m trying to set this up but I’m getting the error msg below. Am I doing something wrong or is it not possible to backup all folders on my cache drive minus a few excluded ones?

 

 

 

68E35A61-E82B-46ED-8627-6C05DEA91483.jpeg

Ok.  Looks like I lied.  Must've had a reason that I set the plug-in like that, but can't think of why right now. 🤔

Share this post


Link to post

Don't you need "appdata" in the source path?

Share this post


Link to post
2 hours ago, bonienl said:

Don't you need "appdata" in the source path?

He was trying to adapt it to backup the whole cache

3 hours ago, Squid said:

Must've had a reason that I set the plug-in like that, but can't think of why right now.

Maybe to prevent overwriting the source

Share this post


Link to post
31 minutes ago, trurl said:

Maybe to prevent overwriting the source

I *think* I have that test in there due to what happens on a restore.  A fail-safe because when restoring, any excess files from the source are deleted.  (and also to prevent a source and destination overlap)

Edited by Squid

Share this post


Link to post

@unRaide My script is highly customized for my use case (it does file deletes, renaming, ignores some folders, backup rotation and, well, poorly written for general use LOL) so I'd rather not share it, however I can edit it and add some additional commenting so it would be easier to digest and you can run with it from there. I'll let you make the call though.

 

Do you want me to post it as is or edit it a bit to just do some example backups?

 

Gary

Share this post


Link to post
2 hours ago, GHunter said:

@unRaide My script is highly customized for my use case (it does file deletes, renaming, ignores some folders, backup rotation and, well, poorly written for general use LOL) so I'd rather not share it, however I can edit it and add some additional commenting so it would be easier to digest and you can run with it from there. I'll let you make the call though.

 

Do you want me to post it as is or edit it a bit to just do some example backups?

 

Gary

Hey @GHunter, I'm quite the noob so any editing you could do would much appreciated by me and thothethat follow I'm sure.

 

That said im grateful for any assistance 😁 

Share this post


Link to post
20 hours ago, Squid said:

Ok.  Looks like I lied.  Must've had a reason that I set the plug-in like that, but can't think of why right now. 🤔

Ah, ok. Would be cool to include such functionality into the plugin as a one stop onsite backup shop. 

Share this post


Link to post

Posting my custom backup script for @unRaide and anyone else that wants to use it. I've been using it for months now and it works great.

 

Note: I only use the Appdata backup function of this plugin and have set backup rotation to off as it is handled by this script.

 

What it does: This script will backup your unRaid USB flash drive, your VM XML files, and OVMF files. Backup rotation is set to 3 days in the script and can be changed. Any dated backup folder will not be deleted if text is appended to the folder name. Example: this folder will not be deleted. 2019-01-22@02.30 Last backup of v6.6.6 before upgrade

 

CA_Backup_Custom_Stop.sh file is attached at the bottom of this post.

 

READ THE COMMENTS AND USE AT YOUR OWN RISK!! Comments are welcome. I'm still learning so be gentle! LOL There's always room for improvement.

#!/bin/bash
# CA_Backup_Custom_Stop.sh

# Change the locations below to match your backup locations
flashbackuplocation="/mnt/user/Backups-Gary/unRAID/Flash/"
nvrambackuplocation="/mnt/user/Backups-Gary/unRAID/NVRAM/"
vmxmlbackuplocation="/mnt/user/Backups-Gary/unRAID/VM_XML/"
appdatabackuplocation="/mnt/user/Backups-Gary/unRAID/Appdata/"

daysofbackupstokeep=3
# 1440 minutes in 1 day 7 days is 10080
(( daystominutes = daysofbackupstokeep * 1440 - 1 ))

# dont change anything below here
echo "..."
echo "Running CA Backup Custom Stop script"
echo "..."
datestamp=""`date '+%F'`"@"`date '+%H'`"."`date '+%M'`
dir="$flashbackuplocation"/"$datestamp"

# make a subfolder to store backup files
if [ ! -d $dir ] ; then
 
echo "Making a subfolder for todays date $datestamp"

# make the directory as it doesnt exist
mkdir -vp $dir
else
echo "As $dir exists continuing."
fi

# save the files using rsync and mod permissions
echo "Saving USB Flash drive files"
rsync -a --no-o /boot/* $dir/
chmod -R 777 $dir
echo "Finished backup of USB Flash drive files"

# deleting backup folders older than X number of days as declared at the top
echo "Deleting USB Flash Drive backups older than $daysofbackupstokeep days except those the folders that have"
echo "text appended to it"

# don't delete folders that have text appened to it. Useful to save older backups. Example: this folder will not be deleted 2019-01-22@02.30 Last backup of v6.6.6 before upgrade
find $flashbackuplocation -type d -maxdepth 1 -mindepth 1 -mmin +$daystominutes -not -name '?????????????????*' | xargs rm -rf
echo "..."
sleep 2

dir="$nvrambackuplocation"/"$datestamp"

# make a subfolder to store backup files
if [ ! -d $dir ] ; then
 
echo "Making a subfolder for todays date $datestamp"

# make the directory as it doesnt exist
mkdir -vp $dir
else
echo "As $dir exists continuing."
fi

# save the files using rsync and mod permissions
echo "Saving OVMF NVRAM files"
rsync -a --no-o /etc/libvirt/qemu/nvram/* $dir/
chmod -R 777 $dir
echo "Finished backup of OVMF NVRAM files"

# deleting backup folders older than X number of days as declared at the top
echo "Deleting NVRAM backups older than $daysofbackupstokeep days except those the folders that have text"
echo "appended to it"

# don't delete folders that have text appened to it. Useful to save older backups. Example: this folder will not be deleted 2019-01-22@02.30 Last backup of v6.6.6 before upgrade
find $nvrambackuplocation -type d -maxdepth 1 -mindepth 1 -mmin +$daystominutes -not -name '?????????????????*' | xargs rm -rf
echo "..."

dir="$vmxmlbackuplocation"/"$datestamp"

# make a subfolder to store backup files
if [ ! -d $dir ] ; then
 
echo "Making a subfolder for todays date $datestamp"

# make the directory as it doesnt exist
mkdir -vp $dir
else
echo "As $dir exists continuing."
fi

# save the files using rsync and mod permissions
echo "Saving VM XML files"
rsync -a --no-o /etc/libvirt/qemu/*xml $dir/
chmod -R 777 $dir
echo "Finished backup of VM XML files"

# rename xml files to *.txt as it's easier to open with a text editor
# this fuction is commented out but if you want to use it then uncomment the following 4 lines
# echo "Renaming all the XML files to TXT"
# cd $dir/
# rename .xml .txt *.xml
# echo "Finished renaming the XML files"

# deleting backup folders older than X number of days as declared at the top
echo "Deleting VM XML backups older than $daysofbackupstokeep days except those the folders that have"
echo "text appended to it"

# don't delete folders that have text appened to it. Useful to save older backups. Example: this folder will not be deleted 2019-01-22@02.30 Last backup of v6.6.6 before upgrade
find $vmxmlbackuplocation -type d -maxdepth 1 -mindepth 1 -mmin +$daystominutes -not -name '?????????????????*' | xargs rm -rf
echo "..."

# lets check and delete older backups from Appdata too for consistency that was performed by the plugin. Note: I disabled backup rotation of appdata folder in the plugin as it is handled by this script
echo "Deleting AppData backups older than $daysofbackupstokeep days except those the folders that have"
echo "text appended to it"

# don't delete folders that have text appened to it. Useful to save older backups. Example: this folder will not be deleted 2019-01-22@02.30 Last backup of v6.6.6 before upgrade
find $appdatabackuplocation -type d -maxdepth 1 -mindepth 1 -mmin +$daystominutes -not -name '?????????????????*' | xargs rm -rf
echo "..."

echo "Finished running CA Backup Custom Stop script"
echo "..."
exit

 

Enjoy, Gary

CA_Backup_Custom_Stop.sh

Edited by GHunter
attached file

Share this post


Link to post
7 hours ago, unRaide said:

Ah, ok. Would be cool to include such functionality into the plugin as a one stop onsite backup shop. 

Thought about a generic backup plugin, and ultimately decided that something like Duplicati is better for that.

Share this post


Link to post
21 hours ago, GHunter said:

Posting my custom backup script for @unRaide and anyone else that wants to use it. I've been using it for months now and it works great.

 

Note: I only use the Appdata backup function of this plugin and have set backup rotation to off as it is handled by this script.

 

What it does: This script will backup your unRaid USB flash drive, your VM XML files, and OVMF files. Backup rotation is set to 3 days in the script and can be changed. Any dated backup folder will not be deleted if text is appended to the folder name. Example: this folder will not be deleted. 2019-01-22@02.30 Last backup of v6.6.6 before upgrade

 

CA_Backup_Custom_Stop.sh file is attached at the bottom of this post.

 

READ THE COMMENTS AND USE AT YOUR OWN RISK!! Comments are welcome. I'm still learning so be gentle! LOL There's always room for improvement.


#!/bin/bash
# CA_Backup_Custom_Stop.sh

# Change the locations below to match your backup locations
flashbackuplocation="/mnt/user/Backups-Gary/unRAID/Flash/"
nvrambackuplocation="/mnt/user/Backups-Gary/unRAID/NVRAM/"
vmxmlbackuplocation="/mnt/user/Backups-Gary/unRAID/VM_XML/"
appdatabackuplocation="/mnt/user/Backups-Gary/unRAID/Appdata/"

daysofbackupstokeep=3
# 1440 minutes in 1 day 7 days is 10080
(( daystominutes = daysofbackupstokeep * 1440 - 1 ))

# dont change anything below here
echo "..."
echo "Running CA Backup Custom Stop script"
echo "..."
datestamp=""`date '+%F'`"@"`date '+%H'`"."`date '+%M'`
dir="$flashbackuplocation"/"$datestamp"

# make a subfolder to store backup files
if [ ! -d $dir ] ; then
 
echo "Making a subfolder for todays date $datestamp"

# make the directory as it doesnt exist
mkdir -vp $dir
else
echo "As $dir exists continuing."
fi

# save the files using rsync and mod permissions
echo "Saving USB Flash drive files"
rsync -a --no-o /boot/* $dir/
chmod -R 777 $dir
echo "Finished backup of USB Flash drive files"

# deleting backup folders older than X number of days as declared at the top
echo "Deleting USB Flash Drive backups older than $daysofbackupstokeep days except those the folders that have"
echo "text appended to it"

# don't delete folders that have text appened to it. Useful to save older backups. Example: this folder will not be deleted 2019-01-22@02.30 Last backup of v6.6.6 before upgrade
find $flashbackuplocation -type d -maxdepth 1 -mindepth 1 -mmin +$daystominutes -not -name '?????????????????*' | xargs rm -rf
echo "..."
sleep 2

dir="$nvrambackuplocation"/"$datestamp"

# make a subfolder to store backup files
if [ ! -d $dir ] ; then
 
echo "Making a subfolder for todays date $datestamp"

# make the directory as it doesnt exist
mkdir -vp $dir
else
echo "As $dir exists continuing."
fi

# save the files using rsync and mod permissions
echo "Saving OVMF NVRAM files"
rsync -a --no-o /etc/libvirt/qemu/nvram/* $dir/
chmod -R 777 $dir
echo "Finished backup of OVMF NVRAM files"

# deleting backup folders older than X number of days as declared at the top
echo "Deleting NVRAM backups older than $daysofbackupstokeep days except those the folders that have text"
echo "appended to it"

# don't delete folders that have text appened to it. Useful to save older backups. Example: this folder will not be deleted 2019-01-22@02.30 Last backup of v6.6.6 before upgrade
find $nvrambackuplocation -type d -maxdepth 1 -mindepth 1 -mmin +$daystominutes -not -name '?????????????????*' | xargs rm -rf
echo "..."

dir="$vmxmlbackuplocation"/"$datestamp"

# make a subfolder to store backup files
if [ ! -d $dir ] ; then
 
echo "Making a subfolder for todays date $datestamp"

# make the directory as it doesnt exist
mkdir -vp $dir
else
echo "As $dir exists continuing."
fi

# save the files using rsync and mod permissions
echo "Saving VM XML files"
rsync -a --no-o /etc/libvirt/qemu/*xml $dir/
chmod -R 777 $dir
echo "Finished backup of VM XML files"

# rename xml files to *.txt as it's easier to open with a text editor
# this fuction is commented out but if you want to use it then uncomment the following 4 lines
# echo "Renaming all the XML files to TXT"
# cd $dir/
# rename .xml .txt *.xml
# echo "Finished renaming the XML files"

# deleting backup folders older than X number of days as declared at the top
echo "Deleting VM XML backups older than $daysofbackupstokeep days except those the folders that have"
echo "text appended to it"

# don't delete folders that have text appened to it. Useful to save older backups. Example: this folder will not be deleted 2019-01-22@02.30 Last backup of v6.6.6 before upgrade
find $vmxmlbackuplocation -type d -maxdepth 1 -mindepth 1 -mmin +$daystominutes -not -name '?????????????????*' | xargs rm -rf
echo "..."

# lets check and delete older backups from Appdata too for consistency that was performed by the plugin. Note: I disabled backup rotation of appdata folder in the plugin as it is handled by this script
echo "Deleting AppData backups older than $daysofbackupstokeep days except those the folders that have"
echo "text appended to it"

# don't delete folders that have text appened to it. Useful to save older backups. Example: this folder will not be deleted 2019-01-22@02.30 Last backup of v6.6.6 before upgrade
find $appdatabackuplocation -type d -maxdepth 1 -mindepth 1 -mmin +$daystominutes -not -name '?????????????????*' | xargs rm -rf
echo "..."

echo "Finished running CA Backup Custom Stop script"
echo "..."
exit

 

Enjoy, Gary

CA_Backup_Custom_Stop.sh

This is great Gary, thanks!!,

 

Really well commented and laid out. I’ll start playing around with it this weekend. 

Share this post


Link to post
15 hours ago, Squid said:

Thought about a generic backup plugin, and ultimately decided that something like Duplicati is better for that.

Fair enough. I looked at duplicati briefly but seemed like a mixed bag from what I read.

 

I really liked the idea of using a more straight fwd plugin integrated into Unraid with just enough of a UI. 😁

Share this post


Link to post

Posting to promote/upvote a feature-request:

Would be nice to have the Appdata Backup separate the tar into individual tars for each app or primary folder instead of one gigantic tar.

  • Like 1

Share this post


Link to post
1 hour ago, Stupifier said:

Posting to promote/upvote a feature-request:

Would be nice to have the Appdata Backup separate the tar into individual tars for each app or primary folder instead of one gigantic tar.

I would like to upvote that too. A feature to backup the folders sequentially would also be nice. In that way small services can be started again and not all of them have to be down just to wait for the large Plex database do finish backing up.

  • Like 1
  • Upvote 1

Share this post


Link to post

Yeah, i would like that as a option too.

 

But more important i also would like to have a set of (e.g.) 10 last backups from dockers and flash. Not only from plugins.

 

Like

plugins 14.02.19.tar

plugins 15.02.19.tar

plugins 16.02.19.tar

 

dockers 14.02.19.tar

dockers 15.02.19.tar

dockers 16.02.19.tar

 

flash 14.02.19.tar

flash 15.02.19.tar

flash 16.02.19.tar

Edited by nuhll
  • Like 1

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now