VM Backup Plugin


Recommended Posts

Turns out my agentless backups complete fine...or when the VM is powered off. 
The problem is when trying to do one with qemu-guest-agent so I can do a snapshot backup of a running VM. 
 
It then kills libvirtd. So the workaround for me since this is just a home setup, is to allow the VM's to be shutdown to complete the backup. 
Not a fan of that, but better than killing my whole server. 
 
2020-09-08 14:07:15 information: Elements can be found on the system. attempting backup.2020-09-08 14:07:15 information: creating local Elements.xml to work with during backup.2020-09-08 14:07:15 information: /mnt/user/share/BACKUP/VMBackups/Elements exists. continuing.2020-09-08 14:07:15 information: skip_vm_shutdown is false and use_snapshots is 1. skipping vm shutdown procedure. Elements is running. can_backup_vm set to y.2020-09-08 14:07:15 information: actually_copy_files is 1.2020-09-08 14:07:15 information: can_backup_vm flag is y. starting backup of Elements configuration, nvram, and vdisk(s).sending incremental file listElements.xmlsent 6,545 bytes received 35 bytes 13,160.00 bytes/sectotal size is 6,455 speedup is 0.982020-09-08 14:07:15 information: copy of Elements.xml to /mnt/user/share/BACKUP/VMBackups/Elements/20200908_1406_Elements.xml complete.2020-09-08 14:07:15 information: Elements does not appear to have an nvram file. skipping.2020-09-08 14:07:15 information: able to perform snapshot for disk /mnt/nvme_mirror/VM/Elements/elements.img on Elements. use_snapshots is 1. vm_state is running. vdisk_type is raw2020-09-08 14:07:15 information: qemu agent found. enabling quiesce on snapshot.Domain snapshot Elements-elements.snap created2020-09-08 14:07:15 information: snapshot command succeeded on elements.snap for Elements./mnt/nvme_mirror/VM/Elements/elements.img : 40.77% (21474836480 => 8754302341 bytes, /mnt/user/share/BACKUP/VMBackups/Elements/20200908_1406_elements.img.zst)2020-09-08 14:07:54 information: copy of /mnt/nvme_mirror/VM/Elements/elements.img to /mnt/user/share/BACKUP/VMBackups/Elements/20200908_1406_elements.img.zst complete.2020-09-08 14:07:54 information: backup of /mnt/nvme_mirror/VM/Elements/elements.img vdisk to /mnt/user/share/BACKUP/VMBackups/Elements/20200908_1406_elements.img.zst complete.error: Disconnected from qemu:///system due to end of fileerror: internal error: client socket is closed

 

I never trusted any of the snapshot stuff. When we're talking about backups....I want 110% reliability/stability. That snapshot stuff was added into the script (not plugin) way later and sorta as an extra/experiment.
Link to comment
1 hour ago, Stupifier said:

I never trusted any of the snapshot stuff. When we're talking about backups....I want 110% reliability/stability. That snapshot stuff was added into the script (not plugin) way later and sorta as an extra/experiment.

For extra credit, use backup software inside the client, you can get a live image backup with most decent backups.

 

Backing up the entire vdisk image is handy for some things, for others real backup software works much better.

  • Like 1
Link to comment
  • 2 weeks later...

I'm having an issue where old backups aren't being deleted. When it gets to that point, it just says "did not find any vdisk image files to remove". Any ideas?

 

sent 131,240 bytes received 35 bytes 262,550.00 bytes/sec
total size is 131,072 speedup is 1.00
2020-09-21 04:00:55 information: copy of /etc/libvirt/qemu/nvram/6c4208e6-3174-12af-66a1-1bbc0b7c992a_VARS-pure-efi.fd to /mnt/user/backups/Win10Dev01/20200921_0400_6c4208e6-3174-12af-66a1-1bbc0b7c992a_VARS-pure-efi.fd complete.
2020-09-21 04:00:55 information: able to perform snapshot for disk /mnt/user/domains/Win10Dev01/vdisk1.img on Win10Dev01. use_snapshots is 1. vm_state is running. vdisk_type is raw
2020-09-21 04:00:55 information: qemu agent found. enabling quiesce on snapshot.
Domain snapshot Win10Dev01-vdisk1.snap created
2020-09-21 04:01:04 information: snapshot command succeeded on vdisk1.snap for Win10Dev01.
'/mnt/user/domains/Win10Dev01/vdisk1.img' -> '/mnt/user/backups/Win10Dev01/20200921_0400_vdisk1.img'
2020-09-21 04:23:16 information: copy of /mnt/user/domains/Win10Dev01/vdisk1.img to /mnt/user/backups/Win10Dev01/20200921_0400_vdisk1.img complete.
2020-09-21 04:23:19 information: backup of /mnt/user/domains/Win10Dev01/vdisk1.img vdisk to /mnt/user/backups/Win10Dev01/20200921_0400_vdisk1.img complete.
Block commit: [100 %]
Successfully pivoted
2020-09-21 04:23:33 information: commited changes from snapshot for /mnt/user/domains/Win10Dev01/vdisk1.img on Win10Dev01.
removed '/mnt/user/domains/Win10Dev01/vdisk1.snap'
2020-09-21 04:23:33 information: forcibly removed snapshot /mnt/user/domains/Win10Dev01/vdisk1.snap for Win10Dev01.
2020-09-21 04:23:33 information: extension for /mnt/user/isos/virtio-win-0.1.185.iso on Win10Dev01 was found in vdisks_extensions_to_skip. skipping disk.
2020-09-21 04:23:33 information: the extensions of the vdisks that were backed up are img.
2020-09-21 04:23:33 information: vm_state is running. vm_original_state is running. not starting Win10Dev01.
2020-09-21 04:23:33 information: backup of Win10Dev01 to /mnt/user/backups/Win10Dev01 completed.
2020-09-21 04:23:33 information: number of days to keep backups set to indefinitely.
2020-09-21 04:23:33 information: cleaning out backups over 2 in location /mnt/user/backups/Win10Dev01/
2020-09-21 04:23:34 information: removed '/mnt/user/backups/Win10Dev01/20200919_0400_Win10Dev01.xml' config file.
2020-09-21 04:23:34 information: removed '/mnt/user/backups/Win10Dev01/20200919_0400_6c4208e6-3174-12af-66a1-1bbc0b7c992a_VARS-pure-efi.fd' nvram file.
2020-09-21 04:23:34 information: did not find any vdisk image files to remove.
2020-09-21 04:23:34 information: did not find any vm log files to remove.
2020-09-21 04:23:34 information: removing local Win10Dev01.xml.
removed 'Win10Dev01.xml'

 

Link to comment
  • 1 month later...

I have many of the same errors others are reporting after installing this plugin.  While I appreciate the efforts of the original dev, and I recognize it is marked as alpha beta, when such potentially risky issues are identified it seems like it should also be flagged with a warning in some way.  I'd have been fine using the userscript version, but I wasn't noticing these errors on my headless rig.  Anyway, I've uninstalled it for now and I hope maybe it will get fixed someday.

 

EDIT - Feeling more strongly about this after I see it marked beta now....

Edited by BurntOC
Link to comment
  • 3 months later...

Hello, long time no see.

I am truly sorry to see so many of you have had an issue with this plugin, and it was not my intention to abandon it for as long as I have. Sadly, life had other plans (as it often does). I've recently found myself with time to tinker again, and as such I've released an update that does a few things to try and address some of the issues I am aware of. Unfortunately, I haven't been able to replicate many of the issues others are having, so my ability to test has been limited.

 

I also can see that the operation of some of the advanced functions isn't immediately clear either. A few of the issues people have had could have been resolved with a settings change in one of the more advanced settings tabs (usually Danger Zone). I'll try to find a way to make some of those things clearer as I progress on addressing the larger issues.

 

The big issue I'm attempting to address in the new release is the issue some people have had with the array not wanting to stop. I have adjusted how the backup scripts are checked and it will hopefully be able kill stuck ones more readily.

 

The other thing I have done is set a max version of 6.8.3 until I have time to test on 6.9. I currently do not have a test server, so it could be a bit before I have a chance to spin one up for troubleshooting.

 

Again, I do sincerely apologize for those that feel I left them in the lurch. Thankfully, as others have rightly pointed out, the script that the plugin uses on the back-end is much more stable and a solid way to go.

I would also like to add that I am more that willing to add contributors to the project if others would like to help me maintain it.

 

Best,

JTok

Edited by JTok
clarity
  • Like 7
  • Thanks 3
Link to comment
  • 3 weeks later...
On 2/3/2021 at 2:48 PM, JTok said:

The other thing I have done is set a max version of 6.8.3 until I have time to test on 6.9. I currently do not have a test server, so it could be a bit before I have a chance to spin one up for troubleshooting.

DM me if you plan to/get a test server up and running ;-)

On 2/3/2021 at 2:48 PM, JTok said:

I would also like to add that I am more that willing to add contributors to the project if others would like to help me maintain it.

I will spread the word to try and help you out!

Link to comment
2 hours ago, Roman Melnik said:

The same
I have created an issue in bug tracker: https://github.com/JTok/unraid.vmbackup/issues/25

If like me you upgraded to 6.9 before installing the latest version I think you'd be fine - v0.2.1 works for me. I'm pretty sure there's a way to install specific versions manually with the webGUI but I can't remember how.

 

Edit: I remembered - go to plugins -> Install Plugin then paste the following in the URL field:

https://raw.githubusercontent.com/jtok/unraid.vmbackup/v0.2.1/vmbackup.plg

Edited by CS01-HS
Link to comment
14 hours ago, CS01-HS said:

If like me you upgraded to 6.9 before installing the latest version I think you'd be fine - v0.2.1 works for me. I'm pretty sure there's a way to install specific versions manually with the webGUI but I can't remember how.

 

Edit: I remembered - go to plugins -> Install Plugin then paste the following in the URL field:

https://raw.githubusercontent.com/jtok/unraid.vmbackup/v0.2.1/vmbackup.plg

 

I probably screwed up the process somehow, but using the link wouldn't complete because of a URL error somewhere in the script.  I foolishly uninstalled my existing version and I tried the v21 and v22 file URLs and I couldn't get that either.  I will probably try downloading them to install locally b/c I want to get it back.   If that fails, can the dev post the steps needed to bypass that check for those of us that are willing to take the risk until a tested version is available?

Link to comment

@Roman Melnik @CS01-HS Okay, I got the new version installed on 6.9 RC2 and I expanded my comfort zone a bit to boot.  Here's how I did it - anyone following these steps is doing so at their own risk.

 

Step 1 - went to OP's beta Github repo for this here.

Step 2 - forked a copy for me to edit

Step 3 - edited the vmbackup.plg file to change the max to 6.13 (in my case, arbitrary just to get me through the next several revs hopefully official support will resume)

Step 4 - downloaded the zip of the repository files, extracted the zip, and copied the files to /boot/temp

Step 5 - Used Install Plugins to browse to /boot/temp and selected the vmbackup.plg file.

Step 6 - Done.  VM Backup is back in my Settings area.

 

  • Like 1
Link to comment
38 minutes ago, BurntOC said:

@Roman Melnik @CS01-HS Okay, I got the new version installed on 6.9 RC2 and I expanded my comfort zone a bit to boot.  Here's how I did it - anyone following these steps is doing so at their own risk.

 

Step 1 - went to OP's beta Github repo for this here.

Step 2 - forked a copy for me to edit

Step 3 - edited the vmbackup.plg file to change the max to 6.13 (in my case, arbitrary just to get me through the next several revs hopefully official support will resume)

Step 4 - downloaded the zip of the repository files, extracted the zip, and copied the files to /boot/temp

Step 5 - Used Install Plugins to browse to /boot/temp and selected the vmbackup.plg file.

Step 6 - Done.  VM Backup is back in my Settings area.

 


I just had a chance to config and attempt a backup. I'd had this working in the past, though I'm pretty sure I'd tweaked a few things because of open issues mentioned in the OP and throughout the thread.  FWIW, my initial backup attempt with the latest version and basically stock mostly works (at least there are few errors), but this single error seems to be bypassing the actual backup:

 

2021-03-01 10:12:49 information: /mnt/user/_backups_unraid1/VMs/GamingPC exists. continuing.
/tmp/vmbackup/scripts/default/user-script.sh: line 424: vdisk_types["$vdisk_path"]: bad array subscript

 

Link to comment

Last update of the day.  I found others with this issue in the thread, and it lead me to believe it was dying on an ISO I still had mounted and/or the SSD I pass through directly by-id.  I wanted to keep the latter if possible, so I removed the VirtIO Drivers ISO and ran it again and the backup appears to have completed just fine.  Haven't tried a restore, but it looks really promising.

 

EDIT - adding that normal backup of this VM was about 107 GB in size, but with zstandard it is 39 GB.  That sounds a little too small, but there are no errors thrown.

 

EDIT2 - welp, now the VM BSODs a lot.  Re-adding the Virtio Drivers ISO in the VM settings seems to have stabilized it, but the VM Backup script is failing again with the error above.

Edited by BurntOC
Link to comment

In light of the official release of 6.9.0 I have removed the max version restriction. I am not able to confirm how well it works this week because I am moving across several time-zones.

 

 

I am also still working on a new version.

As a quick heads up, the new version will be dropping support for pigz, but gzip will still work for anyone using legacy compression (this change should not be breaking or require any user input).

I also fixed a few bugs, and am adding a few new features.

Edited by JTok
  • Like 2
  • Thanks 1
Link to comment
10 hours ago, JTok said:

In light of the official release of 6.9.0 I have removed the max version restriction. I am not able to confirm how well it works this week because I am moving across several time-zones.

 

 

I am also still working on a new version.

As a quick heads up, the new version will be dropping support for pigz, but gzip will still work for anyone using legacy compression (this change should not be breaking or require any user input).

I also fixed a few bugs, and am adding a few new features.

 

Thanks for this.  I know this is a weird issue but if I remove VirtIO Drivers ISO from my VM then it starts crashing within a minute - two at the most.  Over and over.  If I put that back, it works fine.  I know it shouldn't do that, but it does, and it doesn't matter whether I store the ISO on the array or on a UA drive, the VM Backup script gives this error and fails if that ISO is in the settings:

 

/tmp/vmbackup/scripts/default/user-script.sh: line 424: vdisk_types["$vdisk_path"]: bad array subscript

 

Would it be difficult to add a toggle to ignore those, or make it do that by default as most people wouldn't want those included anyway?

Link to comment
16 hours ago, JTok said:

In light of the official release of 6.9.0 I have removed the max version restriction. I am not able to confirm how well it works this week because I am moving across several time-zones.

 

 

I am also still working on a new version.

As a quick heads up, the new version will be dropping support for pigz, but gzip will still work for anyone using legacy compression (this change should not be breaking or require any user input).

I also fixed a few bugs, and am adding a few new features.

I'll the following error

Package pigz-2.3-x86_64-2_slonly.txz installed.
plugin: downloading: https://raw.githubusercontent.com/jtok/unraid.vmbackup/master/vmbackup-v0.2.2-2021.02.03.txz ... failed (Invalid URL / Server error response)
plugin: https://raw.githubusercontent.com/jtok/unraid.vmbackup/master/vmbackup-v0.2.2-2021.02.03.txz download failure (Invalid URL / Server error response)
 

Link to comment
23 hours ago, malac said:

I'll the following error

Package pigz-2.3-x86_64-2_slonly.txz installed.
plugin: downloading: https://raw.githubusercontent.com/jtok/unraid.vmbackup/master/vmbackup-v0.2.2-2021.02.03.txz ... failed (Invalid URL / Server error response)
plugin: https://raw.githubusercontent.com/jtok/unraid.vmbackup/master/vmbackup-v0.2.2-2021.02.03.txz download failure (Invalid URL / Server error response)
 

Whoops! typed the wrong filename. I think it should work now.

Link to comment
9 hours ago, JTok said:

Whoops! typed the wrong filename. I think it should work now.

I can't seem to update the plugin even though it tells me there's an update.

I tried removing the plugin to reinstall it, and now it still fails so it never installs.
 

plugin: updating: vmbackup.plg
Removing package: xmlstarlet-1.6.1-x86_64-1_slonly
Removing files:
Removing package: pigz-2.3-x86_64-2_slonly
Removing files:

+==============================================================================
| Installing new package /boot/config/plugins/vmbackup/packages/xmlstarlet-1.6.1-x86_64-1_slonly.txz
+==============================================================================

Verifying package xmlstarlet-1.6.1-x86_64-1_slonly.txz.
Installing package xmlstarlet-1.6.1-x86_64-1_slonly.txz:
PACKAGE DESCRIPTION:
# xmlstarlet (command line xml tool)
#
# XMLStarlet is a command line XML toolkit that can be used to
# transform, query, validate, and edit XML documents and files using
# a simple set of shell commands, which work similarly to 'grep',
# 'sed', 'awk', 'tr', 'diff', or 'patch' on plain text files.
#
# Homepage https://sourceforge.net/projects/xmlstar/
#
Executing install script for xmlstarlet-1.6.1-x86_64-1_slonly.txz.
Package xmlstarlet-1.6.1-x86_64-1_slonly.txz installed.

+==============================================================================
| Installing new package /boot/config/plugins/vmbackup/packages/pigz-2.3-x86_64-2_slonly.txz
+==============================================================================

Verifying package pigz-2.3-x86_64-2_slonly.txz.
Installing package pigz-2.3-x86_64-2_slonly.txz:
PACKAGE DESCRIPTION:
# pigz (Parallel gzip)
#
# pigz, which stands for parallel implementation of gzip, is a fully
# functional replacement for gzip that exploits multiple processors and
# multiple cores to the hilt when compressing data. pigz was written by
# Mark Adler, and uses the zlib and pthread libraries.
#
# Home page: http://www.zlib.net/pigz/
#
Package pigz-2.3-x86_64-2_slonly.txz installed.

+==============================================================================
| Skipping package vmbackup-v0.2.2-2021.02.03 (already installed)
+==============================================================================

plugin: run failed: /bin/bash retval: 1

 

 

After upgrade to stable 6.9.0 from RC1, I tried to install the plugin again. Still Fails:


 

plugin: installing: https://raw.githubusercontent.com/jtok/unraid.vmbackup/master/vmbackup.plg
plugin: downloading https://raw.githubusercontent.com/jtok/unraid.vmbackup/master/vmbackup.plg
plugin: downloading: https://raw.githubusercontent.com/jtok/unraid.vmbackup/master/vmbackup.plg ... done
No such package: xmlstarlet*. Can't remove.
No such package: pigz*. Can't remove.

+==============================================================================
| Installing new package /boot/config/plugins/vmbackup/packages/xmlstarlet-1.6.1-x86_64-1_slonly.txz
+==============================================================================

Verifying package xmlstarlet-1.6.1-x86_64-1_slonly.txz.
Installing package xmlstarlet-1.6.1-x86_64-1_slonly.txz:
PACKAGE DESCRIPTION:
# xmlstarlet (command line xml tool)
#
# XMLStarlet is a command line XML toolkit that can be used to
# transform, query, validate, and edit XML documents and files using
# a simple set of shell commands, which work similarly to 'grep',
# 'sed', 'awk', 'tr', 'diff', or 'patch' on plain text files.
#
# Homepage https://sourceforge.net/projects/xmlstar/
#
Executing install script for xmlstarlet-1.6.1-x86_64-1_slonly.txz.
Package xmlstarlet-1.6.1-x86_64-1_slonly.txz installed.

+==============================================================================
| Installing new package /boot/config/plugins/vmbackup/packages/pigz-2.3-x86_64-2_slonly.txz
+==============================================================================

Verifying package pigz-2.3-x86_64-2_slonly.txz.
Installing package pigz-2.3-x86_64-2_slonly.txz:
PACKAGE DESCRIPTION:
# pigz (Parallel gzip)
#
# pigz, which stands for parallel implementation of gzip, is a fully
# functional replacement for gzip that exploits multiple processors and
# multiple cores to the hilt when compressing data. pigz was written by
# Mark Adler, and uses the zlib and pthread libraries.
#
# Home page: http://www.zlib.net/pigz/
#
Package pigz-2.3-x86_64-2_slonly.txz installed.

+==============================================================================
| Installing new package /boot/config/plugins/vmbackup/vmbackup-v0.2.2-2021.02.03.txz
+==============================================================================

Verifying package vmbackup-v0.2.2-2021.02.03.txz.
Installing package vmbackup-v0.2.2-2021.02.03.txz:
PACKAGE DESCRIPTION:
Package vmbackup-v0.2.2-2021.02.03.txz installed.
plugin: run failed: /bin/bash retval: 126

 

Edited by Darksurf
New info
Link to comment

I have an odd experience:

I downloaded the plugin and after the installation finished successfully, the WebUI of UnRaid froze. I can click on the main menu (dashboard, main, shares, etc) and it opens the site, but I can't click/open any docker/VM/share/user/settings/etc.

 

When I open the UnRaid WebUI on my mobile phone I get the error message (and with the same behaviour like on the desktop):

Warning: syntax error, unexpected '=' in Unknown on line 13 in /usr/local/emttp/plugins/dynamix/include/PageBuilder.php on line 34

Warning: syntax error, unexpected '$' in Unknown on line 17 in /usr/local/emttp/plugins/dynamix/include/PageBuilder.php on line 34

 

I don't know if it's just a coincidence or the plugin messed up with the WebUI (honestly, I can't imagine how). Right now I can't reboot the server to check, if the error persists or not.

 

Did someone had a similar problem? Maybe with another plugin?

Thank you.

Link to comment
2 hours ago, riddler0815 said:

I have an odd experience:

I downloaded the plugin and after the installation finished successfully, the WebUI of UnRaid froze. I can click on the main menu (dashboard, main, shares, etc) and it opens the site, but I can't click/open any docker/VM/share/user/settings/etc.

 

When I open the UnRaid WebUI on my mobile phone I get the error message (and with the same behaviour like on the desktop):

Warning: syntax error, unexpected '=' in Unknown on line 13 in /usr/local/emttp/plugins/dynamix/include/PageBuilder.php on line 34

Warning: syntax error, unexpected '$' in Unknown on line 17 in /usr/local/emttp/plugins/dynamix/include/PageBuilder.php on line 34

 

I don't know if it's just a coincidence or the plugin messed up with the WebUI (honestly, I can't imagine how). Right now I can't reboot the server to check, if the error persists or not.

 

Did someone had a similar problem? Maybe with another plugin?

Thank you.

 

I couldn't leave well enough alone and I tried to install VM Backup from CA over the version I'd installed per my posts a few days ago.  It failed, as the poster above indicated, with various issues from the one he/she described to a md5 check on a file it pulled.  AND, I started seeing the problem you're desribing above.  Unfortunately, if I delete the plugin out of /config/plugins it still won't let me go back to my previous version either.  I'm going to try to delete the vmbackup folder out of plugins and just leave it be until @JTok has some time to put that new version out.  Too many hiccups at the moment.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.