Jump to content
danioj

unraid-autovmbackup: automate backup of virtual machines in unRAID - v0.4

171 posts in this topic Last Reply

Recommended Posts

Looks like using user.scripts, you'd have to change the line

official_script_name="unraid-autovmbackup-0-4.sh"

to be

official_script_name="script"

  Not quite sure why danioj would be a test on the name of the script though  :o

 

To execute it manually, just type in at a console

/boot/unraid-autovmbackup-0-4.sh

 

Thanks for the help.  Got it working via user scripts and also via your excellent CA plugin.  The problem was I had the location setup as /mnt/user/backup/vm_backup but my share was called /mnt/Backup so it was failing.  The progress info in the user scripts plugin allowed me to see what was going wrong, whereas the CA plugin just moves on and doesn't say anything in the progress window.

 

Feedback for script:

 

- a way to set xx backups stored automatically would be great (see it's already on the wish list)

- VMs should only be restarted IF they were already running, or at least an option to choose.  I have VMs that aren't always on that I don't want starting after the backup

Share this post


Link to post

Check for updates on CA. Already modifed it

 

Sent from my LG-D852 using Tapatalk

 

 

Share this post


Link to post

Hi Danioj!

 

I have finally found time to play around with this script for my VMs, I used it on a test VM first just to be safe. It all worked perfectly however some of the configuration options confused me slightly (not difficult!)

 

For example the startvm setting, in the config file

 

1) default is 0 but set this to 1 if you would like to start a vm after it has successfully been backed up

 

in the log file while running this set to 1 it shows

 

2) information: start_vm_after_backup is 1 vms will not be started following a successfull backup

 

and the log after the backup shows

 

3) action: start_vm_after_backup is 1. starting Ubuntu.

 

so it looks like the first log note is incorrect noting that if set to 1 the vm will not be restarted. This also applies to killvm and startvm_afterfailure

 

other than that it was entirely successful, thanks!

 

Captured this. This was a typo error in the logging.

Share this post


Link to post

 

Feedback for script:

 

- a way to set xx backups stored automatically would be great (see it's already on the wish list)

- VMs should only be restarted IF they were already running, or at least an option to choose.  I have VMs that aren't always on that I don't want starting after the backup

 

Point 1: I am implementing this now.

 

Point 2: I give an option (albeit with a logging issue until now) to restart a vm after a backup. You can specify this in the options. But so I understand you want this:

 

- if vm is started and option to restart is set then start

- if vm is NOT started and option to restart is set then DONT start (because it wasn't started to begin with)?

Share this post


Link to post

 

Feedback for script:

 

- a way to set xx backups stored automatically would be great (see it's already on the wish list)

- VMs should only be restarted IF they were already running, or at least an option to choose.  I have VMs that aren't always on that I don't want starting after the backup

 

Point 1: I am implementing this now.

 

Point 2: I give an option (albeit with a logging issue until now) to restart a vm after a backup. You can specify this in the options. But so I understand you want this:

 

- if vm is started and option to restart is set then start

- if vm is NOT started and option to restart is set then DONT start (because it wasn't started to begin with)?

 

Yes, exactly:

1a - start VM after backup - yes/no

1b - if 1a is yes, only start VM if VM previously on - yes/no

 

Thanks for looking into this

Share this post


Link to post

 

Feedback for script:

 

- a way to set xx backups stored automatically would be great (see it's already on the wish list)

- VMs should only be restarted IF they were already running, or at least an option to choose.  I have VMs that aren't always on that I don't want starting after the backup

 

Point 1: I am implementing this now.

 

Point 2: I give an option (albeit with a logging issue until now) to restart a vm after a backup. You can specify this in the options. But so I understand you want this:

 

- if vm is started and option to restart is set then start

- if vm is NOT started and option to restart is set then DONT start (because it wasn't started to begin with)?

 

Yes, exactly:

1a - start VM after backup - yes/no

1b - if 1a is yes, only start VM if VM previously on - yes/no

 

Thanks for looking into this

 

NP. Easy to implement. Tested and seems to be working ok. Just working on getting the function to only keep backups of a certain age. Want to make sure I get this right as I am deleting files.

 

For those who are keeping watch, am implementing system whereby backups of # days old are kept not # of backups.

 

E.g.:

 

If backup_days_to_keep = 3 then backups of 3 days and younger will be kept. Anything older will be deleted.

 

Some restrictions:

 

- Only works when timestamp switch is also enabled.

- Only works with files which match script time stamping format.

- Only executes on a successful backup.

- Will ensure there is always at least 1 backup remaining.

 

Meaning (as per example above) create as many #'s of backups and if they are younger than 3 days they will all be left alone. As soon as they are older they will all be deleted.

 

I like this method better than just defining number of backups to keep.

Share this post


Link to post

Hi Danioj

I am loving your excellent script and have made a youtube video on it here http://lime-technology.com/forum/index.php?topic=52784.0

 

I was thinking it would be really cool to add some pushbullet and pushover notifications after vms have been backed up with your script.

 

maybe something like this

#set whether to use push notication [0- none] [1-pushover] [2-pushbullet] 
pushnotifications="0"
#pushover api (only fill in if set above to pushover above)
apitoken="token=xxxxxxxxxxxxxxxxxxxx"
userkey="user=xxxxxxxxxxxxxxxxxxxx"
#pushbullet api (only fill in if set above to pushbullet above)
API="xxxxxxxxxxxxxxxxxxx"


#set function "pushnotice" to push type

if [[ "$pushnotifications" =~ ^(1|2)$ ]]; then

	if [ "$pushnotifications" -eq 1 ]; then

		function pushnotice {
				 curl -s \
			--form-string $apitoken \
			--form-string $userkey \
			--form-string "message=$1" \
			https://api.pushover.net/1/messages.json
			}
			echo "set for pushover"


		elif [ "$pushnotifications" -eq 2 ]; then


			function pushnotice {
			curl -u $API: https://api.pushbullet.com/v2/pushes -d type=note -d title="unRAID status" -d body="$1"
			}

			echo "set for pushbullet"			

		fi

	else

		function pushnotice {
			echo "$1"
		}


	fi

 

Also would be great for ovmf machines to backup the ovmf nvram aswell.

 

I am really liking the way this script is developing and thankyou for all the hard work you put in in making this.

Share this post


Link to post

Hi Danioj

I am loving your excellent script and have made a youtube video on it here http://lime-technology.com/forum/index.php?topic=52784.0

 

I was thinking it would be really cool to add some pushbullet and pushover notifications after vms have been backed up with your script.

 

maybe something like this

 

Also would be great for ovmf machines to backup the ovmf nvram aswell.

 

I am really liking the way this script is developing and thankyou for all the hard work you put in in making this.

 

I spoke to danioj earlier, unfortunately he's not well at the moment, so don't get upset if he doesn't reply for a while.  ;)

Share this post


Link to post

Hi Danioj

I am loving your excellent script and have made a youtube video on it here http://lime-technology.com/forum/index.php?topic=52784.0

 

I was thinking it would be really cool to add some pushbullet and pushover notifications after vms have been backed up with your script.

 

maybe something like this

 

Also would be great for ovmf machines to backup the ovmf nvram aswell.

 

I am really liking the way this script is developing and thankyou for all the hard work you put in in making this.

 

I spoke to danioj earlier, unfortunately he's not well at the moment, so don't get upset if he doesn't reply for a while.  ;)

 

Oh i am very sorry to hear that news and wish danioj a speedy recovery.

Share this post


Link to post

 

I was thinking it would be really cool to add some pushbullet and pushover notifications after vms have been backed up with your script.

 

Probably easier just to use the built-in notification system since it already supports those, and coming soon Notify My Android and more

 

/usr/local/emhttp/webGui/scripts/notify

 

notify [-e "event"] [-s "subject"] [-d "description"] [-i "normal|warning|alert"] [-m "message"] [-x] [-t] [add]
  create a notification
  use -e to specify the event
  use -s to specify a subject
  use -d to specify a short description
  use -i to specify the severity
  use -m to specify a message (long description)
  use -x to create a single notification ticket
  use -t to force send email only (for testing)
  all options are optional

Share this post


Link to post

Possible addition to this script: Enabling Hibernate / suspend-to-disk inside respective VMs, then using virsh dompmsuspend <domain> disk to save for the backup. This does, of course, assume your VMs are in a position to be suspended, and if they are not configured to suspend, they will likely shut down anyway.

Share this post


Link to post

Love the script! Thanks =)

 

I've made a couple of crude changes in it for my setup. It now allows qcow2-files to be backed up as well as img-files. And it only starts the VM if it was running before the backup. Maybe something for ver. 0.5 =)

 

Mattias

Share this post


Link to post

Great job - lots of good work done here. Thanks to all contributors but particularly Dan!

 

 

Share this post


Link to post

Can this script be use to backup a Running Windows 10 VM (without shutting it down) ?  My VM can't go down at all. 

Share this post


Link to post

Can this script be use to backup a Running Windows 10 VM (without shutting it down) ?  My VM can't go down at all.

Not reliably. I suggest you look into something like acronis or similar software that will run inside the VM, so that it can be aware of pending file system changes and use shadow copies to make a successful backup.

 

As a side note, a windows 10 VM with no scheduled downtime? Really? You are aware that MS is going to be constantly fighting you for control of updates, and attempting to restart the machine at what will end up being the most inconvenient times. There have already been several automatic update bypasses that MS has disabled, it's going to be an ongoing battle to maintain control.

Share this post


Link to post

As a side note, a windows 10 VM with no scheduled downtime? Really? You are aware that MS is going to be constantly fighting you for control of updates, and attempting to restart the machine at what will end up being the most inconvenient times. There have already been several automatic update bypasses that MS has disabled, it's going to be an ongoing battle to maintain control.

 

Disabling the windows update service has always been reliable. They have never circumvented that approach to stopping autoupdates.

Share this post


Link to post

As a side note, a windows 10 VM with no scheduled downtime? Really? You are aware that MS is going to be constantly fighting you for control of updates, and attempting to restart the machine at what will end up being the most inconvenient times. There have already been several automatic update bypasses that MS has disabled, it's going to be an ongoing battle to maintain control.

 

Disabling the windows update service has always been reliable. They have never circumvented that approach to stopping autoupdates.

Yet.

Share this post


Link to post

relaying a question i had from a post i submitted earlier today. Maybe itll be better asked here.

 

if i modify the code below

rsync -av$rsync_dry_run_option "$disk" "$backup_location/$vm/$timestamp$new_disk"

to

rsync --sparse -av$rsync_dry_run_option "$disk" "$backup_location/$vm/$timestamp$new_disk"

 

will i still have to run this on the restore to get sparse working properly?

cp --sparse=always /path/to/backup.img /path/to/vdisk.img

Share this post


Link to post

 

 

will i still have to run this on the restore to get sparse working properly?

cp --sparse=always /path/to/backup.img /path/to/vdisk.img

 

That flag is only needed to turn a non sparse file into one, if the backup is sparse you can use just cp without any flag.

Share this post


Link to post

 

That flag is only needed to turn a non sparse file into one, if the backup is sparse you can use just cp without any flag.

 

Thank you so much. this worked perfectly

Share this post


Link to post
On 10/2/2016 at 8:48 AM, danioj said:

 

NP. Easy to implement. Tested and seems to be working ok. Just working on getting the function to only keep backups of a certain age. Want to make sure I get this right as I am deleting files.

 

For those who are keeping watch, am implementing system whereby backups of # days old are kept not # of backups.

 

E.g.:

 

If backup_days_to_keep = 3 then backups of 3 days and younger will be kept. Anything older will be deleted.

 

Hi

 

are you still working on adding max backups to keep?  I have 3 50GB VMs now so I have to remember every couple of days to delete the old backups, otherwise they start taking up a lot of space

Share this post


Link to post

Awesome work danioj. Like,Mattias I have some VMs with qcow2 formatted disks. 

 

I just changed line 956 (depending on how many vms you add :D) to this:


 

 if [ ! "${disk##*.}" == "img" ] && [ ! "${disk##*.}" == "qcow2" ]; then

Seems to be working. Might be a good thing to add.

 

Thanks for sharing your hard work!

Edited by daemian

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.