Jump to content
danioj

unraid-autovmbackup: automate backup of virtual machines in unRAID - v0.4

171 posts in this topic Last Reply

Recommended Posts

run in conjunction with appdata backup with CA.

 

Is there any chance of adding VM backup natively into CA's backup function?

I think this would be the most streamlined way while we await (at some point) getting snapshot's added into the VM manager.

Waiting to hear from danioj on that one.  I don't run VM's so need to get together with him to sort that all out.

Share this post


Link to post

run in conjunction with appdata backup with CA.

 

Is there any chance of adding VM backup natively into CA's backup function?

I think this would be the most streamlined way while we await (at some point) getting snapshot's added into the VM manager.

Waiting to hear from danioj on that one.  I don't run VM's so need to get together with him to sort that all out.

 

Good deal, thanks for the update and continued development.

 

It's 5 o'clock somewhere, I tossed a "beer" your way!  ;)

Share this post


Link to post

Thanks for the beer, man...  They don't go to waste!

 

 

Share this post


Link to post

Hi Danioj!

 

I have finally found time to play around with this script for my VMs, I used it on a test VM first just to be safe. It all worked perfectly however some of the configuration options confused me slightly (not difficult!)

 

For example the startvm setting, in the config file

 

1) default is 0 but set this to 1 if you would like to start a vm after it has successfully been backed up

 

in the log file while running this set to 1 it shows

 

2) information: start_vm_after_backup is 1 vms will not be started following a successfull backup

 

and the log after the backup shows

 

3) action: start_vm_after_backup is 1. starting Ubuntu.

 

so it looks like the first log note is incorrect noting that if set to 1 the vm will not be restarted. This also applies to killvm and startvm_afterfailure

 

other than that it was entirely successful, thanks!

 

Share this post


Link to post

Hi Danioj!

 

I have finally found time to play around with this script for my VMs, I used it on a test VM first just to be safe. It all worked perfectly however some of the configuration options confused me slightly (not difficult!)

 

For example the startvm setting, in the config file

 

1) default is 0 but set this to 1 if you would like to start a vm after it has successfully been backed up

 

in the log file while running this set to 1 it shows

 

2) information: start_vm_after_backup is 1 vms will not be started following a successfull backup

 

and the log after the backup shows

 

3) action: start_vm_after_backup is 1. starting Ubuntu.

 

so it looks like the first log note is incorrect noting that if set to 1 the vm will not be restarted. This also applies to killvm and startvm_afterfailure

 

other than that it was entirely successful, thanks!

 

Hi Ashe. I am not surprised there are some kinks. Your outcome though is similar to mine, it works - just needs to be cleaned up.

 

I think I will collude somewhat with Andrew and see if I can get it finalised.

 

Ill spend some time on it this week.

Share this post


Link to post

The main part is it works as intended. The rest is cosmetic.

 

I was going to look into adding mount / umount for my external backup hdd. Is it something you have considered? I was going to look at adding it to the script or adding a wrapper script to call you script in between mounting and unmounting my drive

Share this post


Link to post

This script did not seem to shutdown the VM for me. I had no problem shutting down the VM manually, after which, the script ran fine. Also, I expected the log window to echo the comments as it stepped through the script but, it waited until after my 100GB VM img file was copied and the rest of the script fully executed before showing the echoed comments.

Share this post


Link to post

This script did not seem to shutdown the VM for me. I had no problem shutting down the VM manually, after which, the script ran fine. Also, I expected the log window to echo the comments as it stepped through the script but, it waited until after my 100GB VM img file was copied and the rest of the script fully executed before showing the echoed comments.

 

The main part is it works as intended. The rest is cosmetic.

 

I was going to look into adding mount / umount for my external backup hdd. Is it something you have considered? I was going to look at adding it to the script or adding a wrapper script to call you script in between mounting and unmounting my drive

 

I did some work on this yesterday. I found a number of bugs based on the comments that have been made to date.

 

I am putting together an update.

Share this post


Link to post

2nd cron backup worked fine. The next issue I see is the multiple dated backups taking up considerable space. Will look at deleting after a set number or time period

Share this post


Link to post

2nd cron backup worked fine. The next issue I see is the multiple dated backups taking up considerable space. Will look at deleting after a set number or time period

 

Would you like to see only one maintained or a simple switch which allows you to choose how many you can keep?

Share this post


Link to post

2nd cron backup worked fine. The next issue I see is the multiple dated backups taking up considerable space. Will look at deleting after a set number or time period

I was able to reduce my filesize by gzip.

 

Dan, would be nice to add an automatic gzip to the backup file. Since my VM img is 100GB, the backup is also. After gzip, it reduced to 17GB......quite a difference.

Share this post


Link to post

2nd cron backup worked fine. The next issue I see is the multiple dated backups taking up considerable space. Will look at deleting after a set number or time period

 

Would you like to see only one maintained or a simple switch which allows you to choose how many you can keep?

 

I think the switch would be the ideal version.

 

Probably one for the roadmap (if at all) is selecting which vdisks are backed up if multiple are used for one VM. For example, I use a second vdisk on a Windows VM as a one drive disk as there is no one drive docker yet but I don't need that backed up as I sync it to my array and of course in the cloud

Share this post


Link to post

2nd cron backup worked fine. The next issue I see is the multiple dated backups taking up considerable space. Will look at deleting after a set number or time period

I was able to reduce my filesize by gzip.

 

Dan, would be nice to add an automatic gzip to the backup file. Since my VM img is 100GB, the backup is also. After gzip, it reduced to 17GB......quite a difference.

 

For simplicity I would prefer any compression to be optional but I could see it being a useful option for some

Share this post


Link to post

Thanks for the script, very useful.

 

One suggestion, since vdisks by default are sparse use --sparse on rsync (or cp instead) to make the backups, say you have a 120GB vdisk but only 20GB are allocated, the way it is now the backup will use 120GB, with --sparse or cp it would only use 20GB.

 

 

Share this post


Link to post

Thanks for the script, very useful.

 

One suggestion, since vdisks by default are sparse use --sparse on rsync (or cp instead) to make the backups, say you have a 120GB vdisk but only 20GB are allocated, the way it is now the backup will use 120GB, with --sparse or cp it would only use 20GB.

 

Thanks. Excellent suggestion.

 

I will probably use the rsync switch.

Share this post


Link to post

Thanks for the script, very useful.

 

One suggestion, since vdisks by default are sparse use --sparse on rsync (or cp instead) to make the backups, say you have a 120GB vdisk but only 20GB are allocated, the way it is now the backup will use 120GB, with --sparse or cp it would only use 20GB.

 

Thanks. Excellent suggestion.

 

I will probably use the rsync switch.

 

Great work on this  :)

 

Or maybe you could export the disks using

 

qemu-img convert -O output_format source destination

Share this post


Link to post

I know I'm probably doing something daft (again), but can someone tell me why the script isn't running please.

 

In CA I've got '/boot/unraid-autovmbackup-0-4.sh' in 'Path to custom start script' and I've got the /mnt/user/backup/vm_backup/ folder all setup ready to receive backups.

 

Thanks in advance

unraid-autovmbackup-0-4.zip

Share this post


Link to post

Without looking at the script, when you set the path does it throw up any errors?

(i'll also check CA to make sure that part of it is still working because AFAIK you're the first to use it)

 

Sent from my LG-D852 using Tapatalk

 

Share this post


Link to post

Without looking at the script, when you set the path does it throw up any errors?

(i'll also check CA to make sure that part of it is still working because AFAIK you're the first to use it)

 

Sent from my LG-D852 using Tapatalk

 

Where would I check for script errors?  Sorry, not very technical and new to unraid/linux

Share this post


Link to post

It would be In red at the top of the backup settings.  Only makes sure it exists is executable etc.  Doesn't check anything else

 

Sent from my LG-D852 using Tapatalk

 

 

Share this post


Link to post

It would be In red at the top of the backup settings.  Only makes sure it exists is executable etc.  Doesn't check anything else

 

Sent from my LG-D852 using Tapatalk

 

Don't think I'm getting any errors.  Just ran backup again and in the status window it said:

 

Backup / Restore Status: Running Your docker containers will be automatically restarted at the conclusion of the backup/restore 
Executing custom stop scriptStopping binhex-couchpotato
Stopping binhex-delugevpn
Stopping binhex-plexpass
Stopping binhex-sabnzbd

 

(cut and paste exactly i.e. no space between 'script' and 'Stopping binhex...')

Share this post


Link to post

Since I don't run VM's, I can't really help with the actual script.

 

But, I can tell you that the start and stop scripts do execute at the appropriate times (before docker stops and after docker starts)

 

When I originally added in the hooks for the scripts, I was expecting people to use (in bash) either the logger command to output to the syslog, or manually specify redirection on echo's.  After seeing your output sample (minor issue with no newline after the executing note), I can see that I should force any echo's from the script to my log file so that it'll display in that backup status by default.  Will update CA in about 5 minutes.

 

Beyond that, does the script that you've got actually work calling it manually from the command line?

Share this post


Link to post

Since I don't run VM's, I can't really help with the actual script.

 

But, I can tell you that the start and stop scripts do execute at the appropriate times (before docker stops and after docker starts)

 

When I originally added in the hooks for the scripts, I was expecting people to use (in bash) either the logger command to output to the syslog, or manually specify redirection on echo's.  After seeing your output sample (minor issue with no newline after the executing note), I can see that I should force any echo's from the script to my log file so that it'll display in that backup status by default.  Will update CA in about 5 minutes.

 

Beyond that, does the script that you've got actually work calling it manually from the command line?

 

I don't know how to run scripts manually, so I worked out how to add it to the user scripts plugin.  it fails:

 

failure: official_script_name is unraid-autovmbackup-0-4.sh. script name is invalid. exiting.

 

But it could be because I've done something wrong.

Share this post


Link to post

Looks like using user.scripts, you'd have to change the line

official_script_name="unraid-autovmbackup-0-4.sh"

to be

official_script_name="script"

  Not quite sure why danioj would be a test on the name of the script though  :o

 

To execute it manually, just type in at a console

/boot/unraid-autovmbackup-0-4.sh

Share this post


Link to post

Sorry for the delays guys. I've been working interstate past couple of weeks and am in Sydney now.

 

I have a new version running on my servers at home. If they have run without error when I get back I shall upload. I think I've ironed out much of the remaining bugs.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.