VM Backup Plugin


Recommended Posts

1 minute ago, dja said:

That sounds right if the path exist. Can you navigate to the target in terminal?
...  Do you have Virtio drivers installed for the VM? 

I wouldn't have a clue how to get to that share in terminal. I can do it through my regular windows explorer no problem, and I can get to it via MC by browsing to the disk it physically resides on, but beyond that I'm a bit lost, sorry.

I looked at the logs, and it's weird. The actual log files show that there was an error, but not what it was specifically. I've attached them for your perusal.  If I open the [show log] in the GUI, then it shows mostly the same info, except that I see the actual error I asked about. No idea why they would be different.

Here's a clip of the relevant part of the on-screen log, showing the error and surrounding info...

2021-12-06 02:00:02 information: can_backup_vm flag is y. starting backup of BlueIris Server (W10) configuration, nvram, and vdisk(s).
sending incremental file list
BlueIris Server (W10).xml

sent 6,671 bytes received 35 bytes 13,412.00 bytes/sec
total size is 6,550 speedup is 0.98
2021-12-06 02:00:02 information: copy of BlueIris Server (W10).xml to /mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20211206_0200_BlueIris Server (W10).xml complete.
sending incremental file list
1f335446-855e-7049-961e-043582ac2630_VARS-pure-efi.fd

sent 131,247 bytes received 35 bytes 262,564.00 bytes/sec
total size is 131,072 speedup is 1.00
2021-12-06 02:00:02 information: copy of /etc/libvirt/qemu/nvram/1f335446-855e-7049-961e-043582ac2630_VARS-pure-efi.fd to /mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20211206_0200_1f335446-855e-7049-961e-043582ac2630_VARS-pure-efi.fd complete.
2021-12-06 02:00:02 information: able to perform snapshot for disk /mnt/cache/domains/Windows 10/vdisk1.img on BlueIris Server (W10). use_snapshots is 1. vm_state is running. vdisk_type is raw
2021-12-06 02:00:02 information: qemu agent found. enabling quiesce on snapshot.
error: internal error: missing storage backend for 'file' storage

2021-12-06 02:00:02 failure: snapshot command failed on vdisk1.snap for BlueIris Server (W10).
2021-12-06 02:00:02 failure: snapshot_fallback is 0. skipping backup for BlueIris Server (W10) to prevent data loss. no cleanup will be performed for this vm.

 

20211206_0200_unraid-vmbackup_error.log 20211206_0200_unraid-vmbackup.log

Link to comment
1 hour ago, Elmojo said:

I wouldn't have a clue how to get to that share in terminal. I can do it through my regular windows explorer no problem, and I can get to it via MC by browsing to the disk it physically resides on, but beyond that I'm a bit lost, sorry.

I looked at the logs, and it's weird. The actual log files show that there was an error, but not what it was specifically. I've attached them for your perusal.  If I open the [show log] in the GUI, then it shows mostly the same info, except that I see the actual error I asked about. No idea why they would be different.

Here's a clip of the relevant part of the on-screen log, showing the error and surrounding info...

2021-12-06 02:00:02 information: can_backup_vm flag is y. starting backup of BlueIris Server (W10) configuration, nvram, and vdisk(s).
sending incremental file list
BlueIris Server (W10).xml

sent 6,671 bytes received 35 bytes 13,412.00 bytes/sec
total size is 6,550 speedup is 0.98
2021-12-06 02:00:02 information: copy of BlueIris Server (W10).xml to /mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20211206_0200_BlueIris Server (W10).xml complete.
sending incremental file list
1f335446-855e-7049-961e-043582ac2630_VARS-pure-efi.fd

sent 131,247 bytes received 35 bytes 262,564.00 bytes/sec
total size is 131,072 speedup is 1.00
2021-12-06 02:00:02 information: copy of /etc/libvirt/qemu/nvram/1f335446-855e-7049-961e-043582ac2630_VARS-pure-efi.fd to /mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20211206_0200_1f335446-855e-7049-961e-043582ac2630_VARS-pure-efi.fd complete.
2021-12-06 02:00:02 information: able to perform snapshot for disk /mnt/cache/domains/Windows 10/vdisk1.img on BlueIris Server (W10). use_snapshots is 1. vm_state is running. vdisk_type is raw
2021-12-06 02:00:02 information: qemu agent found. enabling quiesce on snapshot.
error: internal error: missing storage backend for 'file' storage

2021-12-06 02:00:02 failure: snapshot command failed on vdisk1.snap for BlueIris Server (W10).
2021-12-06 02:00:02 failure: snapshot_fallback is 0. skipping backup for BlueIris Server (W10) to prevent data loss. no cleanup will be performed for this vm.

 

20211206_0200_unraid-vmbackup_error.log 8.97 kB · 0 downloads 20211206_0200_unraid-vmbackup.log 8.97 kB · 0 downloads

I think the issue may be that you have it shutting down your VM and it is NOT shutting down. Notice the 2nd line below in logging. I may be reading the log message incorrectly though.

Did you install the Virtio drivers on the VM? That is the easiest thing to check/do first and may resolve it. I would install that and allow the backup to run without shutting down the VM.  

Also- where is your VM image (disk) file stored? See message below-
image.thumb.png.9f4b2809c2b9fa4add63dd1c04309f99.png
 

2021-12-06 02:00:02 failure: snapshot command failed on vdisk1.snap for BlueIris Server (W10).
2021-12-06 02:00:02 failure: snapshot_fallback is 0. skipping backup for BlueIris Server (W10) to prevent data loss. no cleanup will be performed for this vm.

 

Link to comment

That's the thing, I don't have it set to shut down. I selected the snapshot option, which should not be trying to shut down.  Before I enabled the snapshot, I tested it the default way (shut down) and it worked fine. I just want it to use snapshots so that it won't take down my camera server while it's doing the backups.

Yes, the VirtIO drives are installed.

Yes, I read the help info, and my vdisk paths are specified as on cache, as noted in the help tip.

The clip you included is from the actual log file. That's what I said is odd, since it doesn't show what the error actually is. Only the on-screen log shows the "internal error" that references the file system thing. I can't figure how or why the logs wouldn't reflect all the info, but here we are. lol

Link to comment

Installed from Community Applications but ended up needing to change the backup locations. Drop down would not let me start over, and I figured I would reset the settings to default and see if that allowed me. Nope. Then I tried uninstalling it, and reinstalling it. The installer thinks it is still installed. Just Giving feed back as would if I were the developer. Look forward to trying it again when I am able to get it installed. 

Link to comment
  • 2 weeks later...

I am new here and thank you for the great plugin. 

I searched here for a way to restore imaged. I have one machine I am backing up (to test it out) and it is a Windows VM.
My back up has 3 files:

1. VARS-pure-efi.df

2. vdisk.img

3. .xml file

 

What's the best and safest way to restore the image to the original VM? I saw in a post that I have to create a new vm. I don't want to do that because I do have a passthrough GPU and I don't want to redo it every time I do a restore.

 

Or maybe there is a built in feature to restore that I am not seeing?  I have the latest version of the tool. 

Link to comment
4 hours ago, MorphiousGX said:

I am new here and thank you for the great plugin. 

I searched here for a way to restore imaged. I have one machine I am backing up (to test it out) and it is a Windows VM.
My back up has 3 files:

1. VARS-pure-efi.df

2. vdisk.img

3. .xml file

 

What's the best and safest way to restore the image to the original VM? I saw in a post that I have to create a new vm. I don't want to do that because I do have a passthrough GPU and I don't want to redo it every time I do a restore.

 

Or maybe there is a built in feature to restore that I am not seeing?  I have the latest version of the tool. 

I posted in the thread how to restore, maybe a couple pages back.

 

Link to comment
On 11/18/2021 at 7:48 PM, dja said:

PS, you'll need to create a custom VM once your .img file is restored. Paste the contents of the associated backed up XML file in. LOOK AT YOUR XML FILE  closely!! Ensure the img file is in the same folder as was/is spec'd in the XML and also ensure the following are present: 

(File names may vary, look at backed up files) 

Thanks @dja

But as you can see, this is the step I am not clear on. I saw this, and that's what I was referring to about not wanting to recreate a new VM for every VM I want to restore because I do have GPU passthrough and I don't want to risk having issues with it. 

 

Can you elaborate more about this temporary VM? Why can't I use the original VM?

Link to comment
2 hours ago, dja said:

See below. I added some context in. No need to re-do anything other than configuration items and copying a few files. 

HOW TO RESTORE!!!!

You will need:
1.  Your backup .img file (after extraction)
2.  Backed up XML file
3.  Your backed up .fd file

Step 1- In terminal, extract img file from .zst backup. Example below. Replace with your .zst file name. 

zstd -d -C --no-check 20211114_0300_vdisk1.img.zst


You will likely get the error below IF you run the command above without --no-check

 Decoding error (36) : Restored data doesn't match checksum


*Note, -no-check option MAY NOT be supported by Unraid.  If this doesn't work in terminal, try Cygwin (Windows) and run it. https://www.cygwin.com. Place your .zst file in the c:\Cygwin folder.  With Windows you can now also install a Linux environment as well. (Beyond scope of this guide!) Copy the backup to the local machine or it will take forever! 

 

Step 2- Place .img file back into directory where it was backed up from. 
Step 3- Place backed up .fd file to: (file name may vary)

/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd
/etc/libvirt/qemu/nvram/4a2b120f-0ea9-846a-6e11-f097002e442d_VARS-pure-efi.fd

Step 5- verify ALL files are where they were BEFORE the backup.  Look at XML file for file locations. 
Step 6- Open  your .xml file. Copy the contents. Don't mess with it! 

Step 7- Create a new VM in Unraid.  When asked, it does NOT matter what type, (Win11, 10 etcetera..) just pick one. Once you do that and BEFORE hitting "Create", you will have the VM options page. Select the "Form View" button at top right of screen. Change to  XML view. Select all contents and delete. Paste contents of backed up XML in. Select create and away you go! 

 

@dja Thank you so much. OK couple of question, do I delete the other VM (Original)? And why can't I replace the files in the original one? 

Link to comment
1 hour ago, MorphiousGX said:

@dja Thank you so much. OK couple of question, do I delete the other VM (Original)? And why can't I replace the files in the original one? 

I'm not sure I understand what you are trying to do. The point of the back up is to restore your VM should the working version stop or otherwise be made unavailable. 

Link to comment
15 hours ago, dja said:

I'm not sure I understand what you are trying to do. The point of the back up is to restore your VM should the working version stop or otherwise be made unavailable. 

I am trying to restore a VM using one the backups.  What I don't understand is why do I need to create a new VM?

Per your instructions, I need to create new VM. Why is that necessary?

Can I restore it to the ORIGNAL VM, instead of creating  new one? 

Link to comment
23 hours ago, MorphiousGX said:

I am trying to restore a VM using one the backups.  What I don't understand is why do I need to create a new VM?

Per your instructions, I need to create new VM. Why is that necessary?

Can I restore it to the ORIGNAL VM, instead of creating  new one? 

I think you are looking at this the wrong way. You are not adding a new VM. You are adding it back. The backed up XML will have all your settings and everything else in the VM is in the backed up .img file.  There is no other way to restore VMs from this plugin. (That I know of anyway..)  Again, clicking 'new' is just giving you a spot to re-add the XML data for restoring. 

Edited by dja
Clarity
Link to comment
  • 4 weeks later...

I've got the plugin installed with a backup schedule set for my Home Assistant VM, and it works perfectly.  I even tested doing a restore (in case the day comes where I'll actually need it) and it worked as expected.  My only nagging problem is every time my scheduled backup runs, I get the following notification from Unraid:

 

Event: Unraid Status
Subject: VM Backup plugin
Description: cannot run /boot/config/plugins/vmbackup/configs/home-assistant/
Importance: warning

2022-01-14 03:00 User script file does not exist. Exiting.

 

I never put anything in the pre or post-script locations in the config for this backup because I don't have any scripts I need to run.  Is it normal for the plugin to report a warning if you do not have a script set to run as part of your backup?  Is there a way to stop this notification?

 

Note: When I run the backup manually, it does not trigger this notification.

 

Thanks

 

Link to comment
  • 2 weeks later...

So something is suddenly broken, and I don't know how to go about figuring out what.

VM Backup has been running fine for a while, but in the past couple weeks, I've started getting a pop-up notification on the dashboard:

"Unraid Status: 25-01-2022 02:00

VM Backup plugin
cannot run <myprofilename>"

If I go to the plugin page and try to open the logs, I just get a blank window. It never populates.

If I run the job manually, it works just fine, except that it fails to backup one of my Windows VMs that is currently paused. Is that a known bug?

What gives?

Edited by Elmojo
Link to comment

Hey fellows,

 

I'm trying to backup a Windows 10 VM, but it seems that the VM Backup is not able to shutdown it to perform the backup.

...
2022-01-31 14:36:38 information: skip_vm_shutdown is false. beginning vm shutdown procedure.
2022-01-31 14:36:38 infomration: Windows 10 is running. vm desired state is shut off.
Domain Windows 10 is being shutdown

2022-01-31 14:36:38 information: performing 20 30 second cycles waiting for Windows 10 to shutdown cleanly.
2022-01-31 14:36:38 information: cycle 1 of 20: waiting 30 seconds before checking if the vm has entered the desired state.
2022-01-31 14:37:08 information: Windows 10 is running. vm desired state is shut off.
...
2022-01-31 14:46:38 information: Windows 10 is running. vm desired state is shut off.
2022-01-31 14:46:38 failure: Windows 10 is running. vm desired state is shut off. can_backup_vm set to n.
2022-01-31 14:46:38 information: actually_copy_files is 1.
2022-01-31 14:46:38 information: vm_state is running. vm_original_state is running. starting Windows 10.
2022-01-31 14:46:38 warning: vm_state is running. vm_original_state is running. unable to start Windows 10.
2022-01-31 14:46:38 failure: backup of Windows 10 to /mnt/user/backup/domains/Windows 10 failed.
2022-01-31 14:46:38 information: backup of Windows 10 to /mnt/user/backup/domains/Windows 10 completed.
2022-01-31 14:46:38 information: cleaning out backups older than 15 days in location ONLY if newer files exist in /mnt/user/backup/domains/Windows 10/
2022-01-31 14:46:38 information: did not find any config files to remove.
2022-01-31 14:46:38 information: did not find any nvram files to remove.
/tmp/vmbackup/scripts/default/user-script.sh: line 873: ): command not found
2022-01-31 14:46:38 information: did not find any vdisk image files to remove.
2022-01-31 14:46:38 information: did not find any vm log files to remove.
2022-01-31 14:46:38 information: cleaning out backups over 3 in location /mnt/user/backup/domains/Windows 10/
2022-01-31 14:46:38 information: did not find any config files to remove.
2022-01-31 14:46:38 information: did not find any nvram files to remove.
/tmp/vmbackup/scripts/default/user-script.sh: line 895: ): command not found
2022-01-31 14:46:38 information: did not find any vdisk image files to remove.
2022-01-31 14:46:38 information: did not find any vm log files to remove.
2022-01-31 14:46:38 information: removing local Windows 10.xml.
removed 'Windows 10.xml'
2022-01-31 14:46:39 information: finished attempt to backup Windows 10 to /mnt/user/backup/domains.
2022-01-31 14:46:39 information: cleaning out logs over 1.
2022-01-31 14:46:39 information: removed '/mnt/user/backup/domains/logs/20220131_0500_unraid-vmbackup.log'.
2022-01-31 14:46:39 information: cleaning out error logs over 10.
2022-01-31 14:46:39 information: did not find any error log files to remove.
2022-01-31 14:46:39 warning: errors found. creating error log file.
sending incremental file list
20220131_1436_unraid-vmbackup.log

 

Is there anyway to setup to perform a backup without shutdown the windows? Or a way to properly shutdown it?

 

Thanks!

Link to comment
1 hour ago, eltonk said:

Hey fellows,

 

I'm trying to backup a Windows 10 VM, but it seems that the VM Backup is not able to shutdown it to perform the backup.

2022-01-31 14:36:38 information: performing 20 30 second cycles waiting for Windows 10 to shutdown cleanly.
2022-01-31 14:36:38 information: cycle 1 of 20: waiting 30 seconds before checking if the vm has entered the desired state.
2022-01-31 14:37:08 information: Windows 10 is running. vm desired state is shut off.

 

Is there anyway to setup to perform a backup without shutdown the windows? Or a way to properly shutdown it?

 

Thanks!

I have never used this to shut down my Win10 VM- but I can tell you that snapshots DO in-fact work. I would adjust that setting to 'yes' and see where it gets you.  Maybe try it without compression first so you don't have to deal with that headache. 

Edit- also, did you install VirtIO drivers on the VM itself? 

Edited by dja
Link to comment
38 minutes ago, dja said:

I have never used this to shut down my Win10 VM

How do you use the plugin, if it doesn't shut down and restart running VMs? That's kinda one of the basic functions of it. If you have to manually shut down a VM in order for it to backup, then you would only be able to run manual backups, and never have them run on a schedule. I'm confused.

Mine was working perfectly until recently. Was there an update? I can't recall. Now it fails on one of my VMs, but doesn't really say why, and I can't access the logs.

Link to comment
45 minutes ago, Elmojo said:

How do you use the plugin, if it doesn't shut down and restart running VMs?

If you set 'enable snapshots' to yes it will backup your VM after powering down and restore it back to the original state once completed. See the 'danger zone' tab of the plugin, you can see additional options including the 'restore to original state' and other shutdown/restart selections.  The problems I have had with it doing the shutdown during backup were resolved by updating the VirtIO drivers. Also, you should find a logging file included in the destination directory even if it fails. Maybe post that so people can review. 

Edit- my logs file for VM backup is here-->
/mnt/user/Backups/VM/Windows10/logs

Backup files are (for me) /mnt/user/Backups/VM/Windows10/Windows 10

Edited by dja
Link to comment

Ah, I was wondering where those logs were going, thanks!

I have never been able to get snapshots to work correctly. When I first set this up, I had trouble with it, so you suggested that I disable that feature and just let it run full backups, which has been working fine up until recently.

Looking at the logs, I think I see maybe the issue. It appears that if a machine is suspended, rather than running or shut down, it gets confused and doesn't know what to do, so it errors out. This isn't a problem, since I just have to not suspend my VMs, no biggie. :) 

My next backup is scheduled to run at 2am tomorrow morning, so we'll see how it goes with all machines running.

I expect it'll work fine, since I ran a manual backup the other day and it didn't have any problem shutting down and restarting the VMs as needed.

Great work on this plugin, BTW. It's giving me a little peace of mind that I won't mess around an break something too badly. lol

Now, if we just had in-GUI snapshot management, like in VMWare, now THAT would be killer.... ;)

  • Like 1
Link to comment
17 minutes ago, Rocka374 said:

Can anyone share a noobie guide how to restore from a successful backup?

 

I have 3 files and I have no idea what to do with them.

I did a short guide a few posts up...

@mods...can this be pinned / sticky'd to the top of the thread? 

 

Edited by dja
Link to comment

Good news/odd news? lol

Good news: my backups completed successfully last night, as expected.

Odd news: it reported the error "cannot run default"

Umm...okay? Default is disabled, it's not supposed to run. Why am I getting this message now? I don't recall ever seeing it previously.

Otherwise, the plugin seems to be running fine. No errors in the log. :)

 

VMplugin.jpg

Edited by Elmojo
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.