VM Backup Plugin


Recommended Posts

7 hours ago, Elmojo said:

Odd news: it reported the error "cannot run default"

Umm...okay? Default is disabled, it's not supposed to run. Why am I getting this message now? I don't recall ever seeing it previously.

 

 

 

I believe you may be seeing that because you have "Enable Backups" marked as "NO". It looks like you manually ran it however, which is why you see the notification telling you it completed. Think of the warning more as a "Heads up, auto backups won't run". 

Edited by dja
Link to comment

Hello everyone. 

I created a small bash script to automate the restoration of a virtual machine backup. 
It's the first time I participate in the creation of a tool for the community, so please be lenient 😅

I created a repository with some explanations and to consider your feedback or improvement. 
https://github.com/Emilien-Lambert/Unraid-VM-Backup-Plugin-Restoration-Script 


I hope you find it useful 😃

  • Like 4
Link to comment
On 1/14/2022 at 6:00 AM, PaulieORF said:

I've got the plugin installed with a backup schedule set for my Home Assistant VM, and it works perfectly.  I even tested doing a restore (in case the day comes where I'll actually need it) and it worked as expected.  My only nagging problem is every time my scheduled backup runs, I get the following notification from Unraid:

 

Event: Unraid Status
Subject: VM Backup plugin
Description: cannot run /boot/config/plugins/vmbackup/configs/home-assistant/
Importance: warning

2022-01-14 03:00 User script file does not exist. Exiting.

 

I never put anything in the pre or post-script locations in the config for this backup because I don't have any scripts I need to run.  Is it normal for the plugin to report a warning if you do not have a script set to run as part of your backup?  Is there a way to stop this notification?

 

Note: When I run the backup manually, it does not trigger this notification.

 

Thanks

 

@PaulieORFI just set up the same backup for my HA. Curious on how you executed the restore option.  I am curious how you handled the .fd file. Looking at the directions posted above (pinned), I don't see the etc/libvert.... folder (at least on my server).  But the libvert entry is in my .xml file from the backup.  Appreciate your insights.

Link to comment
8 hours ago, Twinkie0101 said:

@PaulieORFI just set up the same backup for my HA. Curious on how you executed the restore option.  I am curious how you handled the .fd file. Looking at the directions posted above (pinned), I don't see the etc/libvert.... folder (at least on my server).  But the libvert entry is in my .xml file from the backup.  Appreciate your insights.

You should definitely have that folder, it is in root folder of UNRAID.  If you edit your VM config, change to XML view and look for this line-
 

<nvram>/etc/libvirt/qemu/nvram/*Your_FileName_Here_*VARS-pure-efi.fd></nvram>

You need to grab the VARS-pure-efi.fd file from your backup and place it where the XML has it listed.  Don't forget to rename/remove the date/time stamp as noted in directions.  There is a script listed above which may help you, but I would suggest understanding how to restore manually before using that just in case. 

Link to comment
12 hours ago, dja said:

You should definitely have that folder, it is in root folder of UNRAID.  If you edit your VM config, change to XML view and look for this line-
 

<nvram>/etc/libvirt/qemu/nvram/*Your_FileName_Here_*VARS-pure-efi.fd></nvram>

You need to grab the VARS-pure-efi.fd file from your backup and place it where the XML has it listed.  Don't forget to rename/remove the date/time stamp as noted in directions.  There is a script listed above which may help you, but I would suggest understanding how to restore manually before using that just in case. 

@dja I definitely have the folder, but my inexperience in accessing this location is the challenge.  In my xml file I have the following entry: <nvram>/etc/libvirt/qemu/nvram/....

 

I can see the root directory using Krusader, but cannot access the folder:

image.png.c57db5b9655df1ff374dc946e9beac12.png

 

There are no rwx permissions which I am definite is preventing access (even though signed in with my root user).  Do I need to do this through another tool?  Thanks for your help as I really want to understand how to handle this manually as you suggested.

Edited by Twinkie0101
Link to comment
13 hours ago, Twinkie0101 said:

Do I need to do this through another tool?  Thanks for your help as I really want to understand how to handle this manually as you suggested.

I would highly suggest getting familiar with the command line, Unraid will be a much better experience! :) You can access this from the top left tool-bar. Be careful! You are root when you do this. 

 image.png.8e7bf56bc21655781add860af5edd15a.png


Once you have that open, you can navigate around OR use something called Midnight Commander. You can launch it from the terminal by typing 'mc'. It is standard with Unraid and has a nice simple panel style GUI. Here is a brief guide on its use. 

 


The best option in my opinion, IF you use Windows...is to use WinSCP. That can integrate with Putty, which will give you the same terminal access I explained above, PLUS Midnight Commander! I wouldn't use WinSCP for moving large amounts of data/files (Use MC!!!), but for a quick edit or moving a couple files it works perfectly and gives you the Windows explorer experience. You can assign a master password to WinSCP and save your credentials for any configured site, pretty handy! Doing that lets you save a single password for everything and it can even pass the credentials to your Putty session so you don't have to login (again) when launching that. 

On top of all that- WinSCP (SSH really..)  can save you if the Unraid gui should be become unavailable, you can still SSH in most times. This would allow you to troubleshoot and do a proper shutdown- possibly saving data loss or a parity re-sync.  

Edited by dja
  • Thanks 1
Link to comment

So I'm still getting these odd "cannot run" error messages, even though everything seems to be working correctly. Any update on how to make it stop throwing false errors? It's really disconcerting when I open the dashboard, and see an orange warning message. Makes me thing something has gone wrong. lol

Here's my complete log file for the job that ran last night...

2022-02-15 02:00:01 Start logging to log file.
2022-02-15 02:00:01 information: send_notifications is 1. notifications will be sent.
2022-02-15 02:00:01 information: only_send_error_notifications is 0. normal notifications will be sent if send_notifications is enabled.
2022-02-15 02:00:01 information: keep_log_file is 1. log files will be kept.
2022-02-15 02:00:01 information: number_of_log_files_to_keep is 1. this is probably a sufficient number of log files to keep.
2022-02-15 02:00:01 information: enable_vm_log_file is 0. vm specific logs will not be created.
2022-02-15 02:00:01 information: backup_all_vms is 1. vms_to_backup will be ignored. all vms will be backed up.
2022-02-15 02:00:01 information: use_snapshots is 0. vms will not be backed up using snapshots.
2022-02-15 02:00:01 information: kill_vm_if_cant_shutdown is 0. vms will not be forced to shutdown if a clean shutdown can not be detected.
2022-02-15 02:00:01 information: set_vm_to_original_state is 1. vms will be set to their original state after backup.
2022-02-15 02:00:01 information: number_of_days_to_keep_backups is 0. backups will be kept indefinitely. be sure to set number_of_backups_to_keep to keep backups storage usage down.
2022-02-15 02:00:01 information: number_of_backups_to_keep is 2. this is probably a sufficient number of backups to keep.
2022-02-15 02:00:01 information: inline_zstd_compress is 1. vdisk images will be inline compressed but will not be compared afterwards or post compressed.
2022-02-15 02:00:01 information: zstd_level is 3.
2022-02-15 02:00:01 information: zstd_threads is 2.
2022-02-15 02:00:01 information: use_snapshots disabled, not adding snapshot_extension to vdisk_extensions_to_skip.
2022-02-15 02:00:01 information: snapshot_fallback is 0. snapshots will fallback to standard backups.
2022-02-15 02:00:01 information: pause_vms is 0. vms will be shutdown for standard backups.
2022-02-15 02:00:01 information: enable_reconstruct_write is 0. reconstruct write will not be enabled by this script.
2022-02-15 02:00:01 information: backup_xml is 1. vms will have their xml configurations backed up.
2022-02-15 02:00:01 information: backup_nvram is 1. vms will have their nvram backed up.
2022-02-15 02:00:01 information: backup_vdisks is 1. vms will have their vdisks backed up.
2022-02-15 02:00:01 information: start_vm_after_backup is 0. vms will not be started following successful backup.
2022-02-15 02:00:01 information: start_vm_after_failure is 0. vms will not be started following an unsuccessful backup.
2022-02-15 02:00:01 information: disable_delta_sync is 0. rsync will be used to perform delta sync backups.
2022-02-15 02:00:01 information: rsync_only is 0. cp will be used when applicable.
2022-02-15 02:00:01 information: actually_copy_files is 1. files will be copied.
2022-02-15 02:00:01 information: clean_shutdown_checks is 20. this is probably a sufficient number of shutdown checks.
2022-02-15 02:00:01 information: seconds_to_wait is 30. this is probably a sufficient number of seconds to wait between shutdown checks.
2022-02-15 02:00:01 information: keep_error_log_file is 1. error log files will be kept.
2022-02-15 02:00:01 information: number_of_error_log_files_to_keep is 10. this is probably a sufficient error number of log files to keep.
2022-02-15 02:00:01 information: started attempt to backup BlueIris Server (W10), Sandbox (W10 ), Windows 11 to /mnt/user/Backup/xVM Backups
2022-02-15 02:00:02 information: BlueIris Server (W10) can be found on the system. attempting backup.
2022-02-15 02:00:02 information: creating local BlueIris Server (W10).xml to work with during backup.
2022-02-15 02:00:02 information: /mnt/user/Backup/xVM Backups/BlueIris Server (W10) exists. continuing.
2022-02-15 02:00:02 information: skip_vm_shutdown is false. beginning vm shutdown procedure.
2022-02-15 02:00:02 infomration: BlueIris Server (W10) is running. vm desired state is shut off.
2022-02-15 02:00:03 information: performing 20 30 second cycles waiting for BlueIris Server (W10) to shutdown cleanly.
2022-02-15 02:00:03 information: cycle 1 of 20: waiting 30 seconds before checking if the vm has entered the desired state.
2022-02-15 02:00:33 information: BlueIris Server (W10) is shut off. vm desired state is shut off. can_backup_vm set to y.
2022-02-15 02:00:33 information: actually_copy_files is 1.
2022-02-15 02:00:33 information: can_backup_vm flag is y. starting backup of BlueIris Server (W10) configuration, nvram, and vdisk(s).
2022-02-15 02:00:33 information: copy of BlueIris Server (W10).xml to /mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20220215_0200_BlueIris Server (W10).xml complete.
2022-02-15 02:00:33 information: copy of /etc/libvirt/qemu/nvram/1f335446-855e-7049-961e-043582ac2630_VARS-pure-efi.fd to /mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20220215_0200_1f335446-855e-7049-961e-043582ac2630_VARS-pure-efi.fd complete.
2022-02-15 02:04:54 information: copy of /mnt/cache/domains/Windows 10/vdisk1.img to /mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20220215_0200_vdisk1.img.zst complete.
2022-02-15 02:04:54 information: backup of /mnt/cache/domains/Windows 10/vdisk1.img vdisk to /mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20220215_0200_vdisk1.img.zst complete.
2022-02-15 02:04:54 information: the extensions of the vdisks that were backed up are img.
2022-02-15 02:04:54 information: vm_state is shut off. vm_original_state is running. starting BlueIris Server (W10).
2022-02-15 02:04:55 information: backup of BlueIris Server (W10) to /mnt/user/Backup/xVM Backups/BlueIris Server (W10) completed.
2022-02-15 02:04:55 information: number of days to keep backups set to indefinitely.
2022-02-15 02:04:55 information: cleaning out backups over 2 in location /mnt/user/Backup/xVM Backups/BlueIris Server (W10)/
2022-02-15 02:04:55 information: removed '/mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20220125_1020_BlueIris Server (W10).xml' config file.
2022-02-15 02:04:55 information: removed '/mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20220125_1020_1f335446-855e-7049-961e-043582ac2630_VARS-pure-efi.fd' nvram file.
2022-02-15 02:04:55 information: removed '/mnt/user/Backup/xVM Backups/BlueIris Server (W10)/20220125_1020_vdisk1.img.zst' vdisk image file.
2022-02-15 02:04:55 information: did not find any vm log files to remove.
2022-02-15 02:04:55 information: removing local BlueIris Server (W10).xml.
2022-02-15 02:04:55 information: Sandbox (W10 ) can be found on the system. attempting backup.
2022-02-15 02:04:55 information: creating local Sandbox (W10 ).xml to work with during backup.
2022-02-15 02:04:55 information: /mnt/user/Backup/xVM Backups/Sandbox (W10 ) exists. continuing.
2022-02-15 02:04:55 information: extension for /mnt/user/isos/ru-en_windows_10_20H2_x86-x64_28in1_HWID-act.iso on Sandbox (W10 ) was found in vdisks_extensions_to_skip. skipping disk.
2022-02-15 02:04:55 information: extension for /mnt/user/isos/virtio-win-0.1.208.iso on Sandbox (W10 ) was found in vdisks_extensions_to_skip. skipping disk.
2022-02-15 02:04:55 information: skip_vm_shutdown is false. beginning vm shutdown procedure.
2022-02-15 02:04:55 infomration: Sandbox (W10 ) is running. vm desired state is shut off.
2022-02-15 02:05:55 information: performing 20 30 second cycles waiting for Sandbox (W10 ) to shutdown cleanly.
2022-02-15 02:05:55 information: cycle 1 of 20: waiting 30 seconds before checking if the vm has entered the desired state.
2022-02-15 02:06:25 information: Sandbox (W10 ) is running. vm desired state is shut off.
2022-02-15 02:06:25 information: cycle 2 of 20: waiting 30 seconds before checking if the vm has entered the desired state.
2022-02-15 02:06:55 information: Sandbox (W10 ) is shut off. vm desired state is shut off. can_backup_vm set to y.
2022-02-15 02:06:55 information: actually_copy_files is 1.
2022-02-15 02:06:55 information: can_backup_vm flag is y. starting backup of Sandbox (W10 ) configuration, nvram, and vdisk(s).
2022-02-15 02:06:55 information: copy of Sandbox (W10 ).xml to /mnt/user/Backup/xVM Backups/Sandbox (W10 )/20220215_0200_Sandbox (W10 ).xml complete.
2022-02-15 02:06:55 information: copy of /etc/libvirt/qemu/nvram/6eee4b38-2964-616c-20e2-b7ce678ba076_VARS-pure-efi.fd to /mnt/user/Backup/xVM Backups/Sandbox (W10 )/20220215_0200_6eee4b38-2964-616c-20e2-b7ce678ba076_VARS-pure-efi.fd complete.
2022-02-15 02:10:15 information: copy of /mnt/cache/domains/Win10 Sandbox/vdisk1.img to /mnt/user/Backup/xVM Backups/Sandbox (W10 )/20220215_0200_vdisk1.img.zst complete.
2022-02-15 02:10:15 information: backup of /mnt/cache/domains/Win10 Sandbox/vdisk1.img vdisk to /mnt/user/Backup/xVM Backups/Sandbox (W10 )/20220215_0200_vdisk1.img.zst complete.
2022-02-15 02:10:15 information: extension for /mnt/user/isos/ru-en_windows_10_20H2_x86-x64_28in1_HWID-act.iso on Sandbox (W10 ) was found in vdisks_extensions_to_skip. skipping disk.
2022-02-15 02:10:15 information: extension for /mnt/user/isos/virtio-win-0.1.208.iso on Sandbox (W10 ) was found in vdisks_extensions_to_skip. skipping disk.
2022-02-15 02:10:15 information: the extensions of the vdisks that were backed up are img.
2022-02-15 02:10:15 information: vm_state is shut off. vm_original_state is running. starting Sandbox (W10 ).
2022-02-15 02:10:16 information: backup of Sandbox (W10 ) to /mnt/user/Backup/xVM Backups/Sandbox (W10 ) completed.
2022-02-15 02:10:16 information: number of days to keep backups set to indefinitely.
2022-02-15 02:10:16 information: cleaning out backups over 2 in location /mnt/user/Backup/xVM Backups/Sandbox (W10 )/
2022-02-15 02:10:16 information: removed '/mnt/user/Backup/xVM Backups/Sandbox (W10 )/20220125_1020_Sandbox (W10 ).xml' config file.
2022-02-15 02:10:16 information: removed '/mnt/user/Backup/xVM Backups/Sandbox (W10 )/20220125_1020_6eee4b38-2964-616c-20e2-b7ce678ba076_VARS-pure-efi.fd' nvram file.
2022-02-15 02:10:16 information: removed '/mnt/user/Backup/xVM Backups/Sandbox (W10 )/20220125_1020_vdisk1.img.zst' vdisk image file.
2022-02-15 02:10:16 information: did not find any vm log files to remove.
2022-02-15 02:10:16 information: removing local Sandbox (W10 ).xml.
2022-02-15 02:10:16 information: Windows 11 can be found on the system. attempting backup.
2022-02-15 02:10:16 information: creating local Windows 11.xml to work with during backup.
2022-02-15 02:10:16 information: /mnt/user/Backup/xVM Backups/Windows 11 exists. continuing.
2022-02-15 02:10:16 information: skip_vm_shutdown is false. beginning vm shutdown procedure.
2022-02-15 02:10:16 information: Windows 11 is shut off. vm desired state is shut off. can_backup_vm set to y.
2022-02-15 02:10:16 information: actually_copy_files is 1.
2022-02-15 02:10:16 information: can_backup_vm flag is y. starting backup of Windows 11 configuration, nvram, and vdisk(s).
2022-02-15 02:10:16 information: copy of Windows 11.xml to /mnt/user/Backup/xVM Backups/Windows 11/20220215_0200_Windows 11.xml complete.
2022-02-15 02:10:16 information: copy of /etc/libvirt/qemu/nvram/985ce70c-f343-81f3-65e3-94e77b37d0e7_VARS-pure-efi.fd to /mnt/user/Backup/xVM Backups/Windows 11/20220215_0200_985ce70c-f343-81f3-65e3-94e77b37d0e7_VARS-pure-efi.fd complete.
2022-02-15 02:12:43 information: copy of /mnt/user/domains/Windows 11/vdisk1.img to /mnt/user/Backup/xVM Backups/Windows 11/20220215_0200_vdisk1.img.zst complete.
2022-02-15 02:12:43 information: backup of /mnt/user/domains/Windows 11/vdisk1.img vdisk to /mnt/user/Backup/xVM Backups/Windows 11/20220215_0200_vdisk1.img.zst complete.
2022-02-15 02:12:43 information: the extensions of the vdisks that were backed up are img.
2022-02-15 02:12:43 information: vm_state is shut off. vm_original_state is shut off. not starting Windows 11.
2022-02-15 02:12:43 information: backup of Windows 11 to /mnt/user/Backup/xVM Backups/Windows 11 completed.
2022-02-15 02:12:43 information: number of days to keep backups set to indefinitely.
2022-02-15 02:12:43 information: cleaning out backups over 2 in location /mnt/user/Backup/xVM Backups/Windows 11/
2022-02-15 02:12:44 information: did not find any config files to remove.
2022-02-15 02:12:44 information: did not find any nvram files to remove.
2022-02-15 02:12:44 information: did not find any vdisk image files to remove.
2022-02-15 02:12:44 information: did not find any vm log files to remove.
2022-02-15 02:12:44 information: removing local Windows 11.xml.
2022-02-15 02:12:44 information: finished attempt to backup BlueIris Server (W10), Sandbox (W10 ), Windows 11 to /mnt/user/Backup/xVM Backups.
2022-02-15 02:12:44 information: cleaning out logs over 1.
2022-02-15 02:12:44 information: removed '/mnt/user/Backup/xVM Backups/logs/20220201_0200_unraid-vmbackup.log'.
2022-02-15 02:12:44 information: cleaning out error logs over 10.
2022-02-15 02:12:44 information: did not find any error log files to remove.
2022-02-15 02:12:44 Stop logging to log file.

And here's what the message is saying on screen....

 

backup_error.jpg

For reference, the "default" job is disabled completely, both by selecting Enable Backups? to 'No' and by setting Backup Frequency to 'Disabled'.

Edited by Elmojo
added info
Link to comment
35 minutes ago, Elmojo said:

backup_error.jpg

For reference, the "default" job is disabled completely, both by selecting Enable Backups? to 'No' and by setting Backup Frequency to 'Disabled'.

 

Can you post your custom job file from /boot/config/plugins/vmbackup/Elmojo

Edited by dja
Link to comment
On 2/6/2022 at 3:01 PM, petchav said:

Hello everyone. 

I created a small bash script to automate the restoration of a virtual machine backup. 
It's the first time I participate in the creation of a tool for the community, so please be lenient 😅

I created a repository with some explanations and to consider your feedback or improvement. 
https://github.com/Emilien-Lambert/Unraid-VM-Backup-Plugin-Restoration-Script 


I hope you find it useful 😃

Thank you petchav for this awesome script, the restore of a vm couldn't be much easier. 

 

 

I have two comments for other people using the script:

- If you get the output "Invalid input" after the "Are you sure?" question in the script, type in yes and don't just press enter.
 

 

 

- If you have a vm which is called for example "DietPi Test", a direct restore seems not possible, no matter which name you give the vm in the script. I was able to restore this vm with the following steps:
 

- Stop all vms
- Go to your backup path and rename "DietPi Test" to "DietPi-Test"
- Go into this folder and change the date_time_DietPi Test.xml to date_time_DietPi-Test.xml
- execute the script and stop&start the array
- When you want to start the vm, an error appears, that the vdisk file is not found in the supposed directory.
- Go to the console and rename the folder for this vm:
    - cd /path/to/vms
    - mv DietPi-Test "DietPi Test"

- Start vm and give petchav a like for his script. ;-)

 

 

Maginos

  • Like 1
Link to comment
On 2/20/2022 at 9:23 AM, Maginos said:

Thank you petchav for this awesome script, the restore of a vm couldn't be much easier. 

 

 

I have two comments for other people using the script:

- If you get the output "Invalid input" after the "Are you sure?" question in the script, type in yes and don't just press enter.
 

 

 

- If you have a vm which is called for example "DietPi Test", a direct restore seems not possible, no matter which name you give the vm in the script. I was able to restore this vm with the following steps:
 

- Stop all vms
- Go to your backup path and rename "DietPi Test" to "DietPi-Test"
- Go into this folder and change the date_time_DietPi Test.xml to date_time_DietPi-Test.xml
- execute the script and stop&start the array
- When you want to start the vm, an error appears, that the vdisk file is not found in the supposed directory.
- Go to the console and rename the folder for this vm:
    - cd /path/to/vms
    - mv DietPi-Test "DietPi Test"

- Start vm and give petchav a like for his script. ;-)

 

 

Maginos


Indeed, my script should probably be improved in terms of user input management. I didn't think about the fact that a VM could have a name with a space. I'll see if a solution is possible. 

  • Like 1
Link to comment
  • 2 weeks later...

Hello every1,

 

First of all Giant thanks for the plug, its been serving me for almost a year on 3 different machines and does work.

However i do have a strange issue on one of my heavyVM machines.

 

There are 7 VMs there , all with their VMdisks in .img format, and all are working.

for some reason tho SOME of them do not get their overhead copies of .img files deleted.

for example i have 2 machines that are called "Askedfor1"  and "Askedfor2" and they are completely similar in how they were created etc. the issue tho affects only one of them , where backup script DOESNT see the Vdisk backup file to delete.

 

They are backed up at the same time and those are SSDs/NVME drives and i do have that issues with like 2 more of my VMs but those had spaces in the names , so i thought thats the reason so i changed the names , backup created new folder but still does the same with "did not find any Vdisk image files to remove"

 

What would be the cause of that issue?

 

2022-03-05 00:38:23 information: AskedFor2 can be found on the system. attempting backup.
2022-03-05 00:38:23 information: creating local AskedFor2.xml to work with during backup.
2022-03-05 00:38:23 information: /mnt/user/VMsBackupShare/AskedFor2 exists. continuing.
2022-03-05 00:38:23 information: skip_vm_shutdown is false and use_snapshots is 1. skipping vm shutdown procedure. AskedFor2 is running. can_backup_vm set to y.
2022-03-05 00:38:23 information: actually_copy_files is 1.
2022-03-05 00:38:23 information: can_backup_vm flag is y. starting backup of AskedFor2 configuration, nvram, and vdisk(s).
2022-03-05 00:38:23 information: copy of AskedFor2.xml to /mnt/user/VMsBackupShare/AskedFor2/20220305_0000_AskedFor2.xml complete.
2022-03-05 00:38:23 information: copy of /etc/libvirt/qemu/nvram/232c1702-aff4-5622-15ff-9f3557867781_VARS-pure-efi.fd to /mnt/user/VMsBackupShare/AskedFor2/20220305_0000_232c1702-aff4-5622-15ff-9f3557867781_VARS-pure-efi.fd complete.
2022-03-05 00:38:23 information: able to perform snapshot for disk /mnt/disk4/Disk4SSD1TB/askedfor2.img on AskedFor2. use_snapshots is 1. vm_state is running. vdisk_type is raw
2022-03-05 00:38:23 information: qemu agent found. enabling quiesce on snapshot.
2022-03-05 00:38:25 information: snapshot command succeeded on askedfor2.snap for AskedFor2.
2022-03-05 00:44:51 information: copy of /mnt/disk4/Disk4SSD1TB/askedfor2.img to /mnt/user/VMsBackupShare/AskedFor2/20220305_0000_askedfor2.img complete.
2022-03-05 00:44:51 information: backup of /mnt/disk4/Disk4SSD1TB/askedfor2.img vdisk to /mnt/user/VMsBackupShare/AskedFor2/20220305_0000_askedfor2.img complete.
2022-03-05 00:44:57 information: commited changes from snapshot for /mnt/disk4/Disk4SSD1TB/askedfor2.img on AskedFor2.
2022-03-05 00:44:57 information: forcibly removed snapshot /mnt/disk4/Disk4SSD1TB/askedfor2.snap for AskedFor2.
2022-03-05 00:44:57 information: extension for /mnt/user/isos/virtio-win-0.1.190-1.iso on AskedFor2 was found in vdisks_extensions_to_skip. skipping disk.
2022-03-05 00:44:57 information: the extensions of the vdisks that were backed up are img.
2022-03-05 00:44:57 information: vm_state is running. vm_original_state is running. not starting AskedFor2.
2022-03-05 00:44:57 information: backup of AskedFor2 to /mnt/user/VMsBackupShare/AskedFor2 completed.
2022-03-05 00:44:57 information: number of days to keep backups set to indefinitely.
2022-03-05 00:44:57 information: cleaning out backups over 2 in location /mnt/user/VMsBackupShare/AskedFor2/
2022-03-05 00:44:57 information: removed '/mnt/user/VMsBackupShare/AskedFor2/20220226_0000_AskedFor2.xml' config file.
2022-03-05 00:44:57 information: removed '/mnt/user/VMsBackupShare/AskedFor2/20220226_0000_232c1702-aff4-5622-15ff-9f3557867781_VARS-pure-efi.fd' nvram file.
2022-03-05 00:44:57 information: did not find any vdisk image files to remove.
2022-03-05 00:44:57 information: did not find any vm log files to remove.
2022-03-05 00:44:57 information: removing local AskedFor2.xml.
2022-03-05 00:44:57 information: AskedFor1 can be found on the system. attempting backup.
2022-03-05 00:44:57 information: creating local AskedFor1.xml to work with during backup.
2022-03-05 00:44:57 information: /mnt/user/VMsBackupShare/AskedFor1 exists. continuing.
2022-03-05 00:44:57 information: skip_vm_shutdown is false and use_snapshots is 1. skipping vm shutdown procedure. AskedFor1 is running. can_backup_vm set to y.
2022-03-05 00:44:57 information: actually_copy_files is 1.
2022-03-05 00:44:57 information: can_backup_vm flag is y. starting backup of AskedFor1 configuration, nvram, and vdisk(s).
2022-03-05 00:44:57 information: copy of AskedFor1.xml to /mnt/user/VMsBackupShare/AskedFor1/20220305_0000_AskedFor1.xml complete.
2022-03-05 00:44:57 information: copy of /etc/libvirt/qemu/nvram/1654e5f3-480a-e5c8-0a8e-f9922c315dc0_VARS-pure-efi.fd to /mnt/user/VMsBackupShare/AskedFor1/20220305_0000_1654e5f3-480a-e5c8-0a8e-f9922c315dc0_VARS-pure-efi.fd complete.
2022-03-05 00:44:57 information: able to perform snapshot for disk /mnt/disk4/Disk4SSD1TB/askedfor1.img on AskedFor1. use_snapshots is 1. vm_state is running. vdisk_type is raw
2022-03-05 00:44:57 information: qemu agent found. enabling quiesce on snapshot.
2022-03-05 00:44:58 information: snapshot command succeeded on askedfor1.snap for AskedFor1.
2022-03-05 00:53:02 information: copy of /mnt/disk4/Disk4SSD1TB/askedfor1.img to /mnt/user/VMsBackupShare/AskedFor1/20220305_0000_askedfor1.img complete.
2022-03-05 00:53:03 information: backup of /mnt/disk4/Disk4SSD1TB/askedfor1.img vdisk to /mnt/user/VMsBackupShare/AskedFor1/20220305_0000_askedfor1.img complete.
2022-03-05 00:53:08 information: commited changes from snapshot for /mnt/disk4/Disk4SSD1TB/askedfor1.img on AskedFor1.
2022-03-05 00:53:08 information: forcibly removed snapshot /mnt/disk4/Disk4SSD1TB/askedfor1.snap for AskedFor1.
2022-03-05 00:53:08 information: extension for /mnt/user/isos/virtio-win-0.1.190-1.iso on AskedFor1 was found in vdisks_extensions_to_skip. skipping disk.
2022-03-05 00:53:08 information: the extensions of the vdisks that were backed up are img.
2022-03-05 00:53:08 information: vm_state is running. vm_original_state is running. not starting AskedFor1.
2022-03-05 00:53:08 information: backup of AskedFor1 to /mnt/user/VMsBackupShare/AskedFor1 completed.
2022-03-05 00:53:08 information: number of days to keep backups set to indefinitely.
2022-03-05 00:53:08 information: cleaning out backups over 2 in location /mnt/user/VMsBackupShare/AskedFor1/
2022-03-05 00:53:08 information: removed '/mnt/user/VMsBackupShare/AskedFor1/20220226_0000_AskedFor1.xml' config file.
2022-03-05 00:53:08 information: removed '/mnt/user/VMsBackupShare/AskedFor1/20220226_0000_1654e5f3-480a-e5c8-0a8e-f9922c315dc0_VARS-pure-efi.fd' nvram file.
2022-03-05 00:53:09 information: removed '/mnt/user/VMsBackupShare/AskedFor1/20220226_0000_askedfor1.img' vdisk image file.
2022-03-05 00:53:09 information: did not find any vm log files to remove.
2022-03-05 00:53:09 information: removing local AskedFor1.xml.

 

Link to comment
On 3/8/2022 at 7:02 PM, Xcage said:

Hello every1,

 

First of all Giant thanks for the plug, its been serving me for almost a year on 3 different machines and does work.

However i do have a strange issue on one of my heavyVM machines.

 

There are 7 VMs there , all with their VMdisks in .img format, and all are working.

for some reason tho SOME of them do not get their overhead copies of .img files deleted.

for example i have 2 machines that are called "Askedfor1"  and "Askedfor2" and they are completely similar in how they were created etc. the issue tho affects only one of them , where backup script DOESNT see the Vdisk backup file to delete.

 

They are backed up at the same time and those are SSDs/NVME drives and i do have that issues with like 2 more of my VMs but those had spaces in the names , so i thought thats the reason so i changed the names , backup created new folder but still does the same with "did not find any Vdisk image files to remove"

 

What would be the cause of that issue?

 

2022-03-05 00:38:23 information: AskedFor2 can be found on the system. attempting backup.
2022-03-05 00:38:23 information: creating local AskedFor2.xml to work with during backup.
2022-03-05 00:38:23 information: /mnt/user/VMsBackupShare/AskedFor2 exists. continuing.
2022-03-05 00:38:23 information: skip_vm_shutdown is false and use_snapshots is 1. skipping vm shutdown procedure. AskedFor2 is running. can_backup_vm set to y.
2022-03-05 00:38:23 information: actually_copy_files is 1.
2022-03-05 00:38:23 information: can_backup_vm flag is y. starting backup of AskedFor2 configuration, nvram, and vdisk(s).
2022-03-05 00:38:23 information: copy of AskedFor2.xml to /mnt/user/VMsBackupShare/AskedFor2/20220305_0000_AskedFor2.xml complete.
2022-03-05 00:38:23 information: copy of /etc/libvirt/qemu/nvram/232c1702-aff4-5622-15ff-9f3557867781_VARS-pure-efi.fd to /mnt/user/VMsBackupShare/AskedFor2/20220305_0000_232c1702-aff4-5622-15ff-9f3557867781_VARS-pure-efi.fd complete.
2022-03-05 00:38:23 information: able to perform snapshot for disk /mnt/disk4/Disk4SSD1TB/askedfor2.img on AskedFor2. use_snapshots is 1. vm_state is running. vdisk_type is raw
2022-03-05 00:38:23 information: qemu agent found. enabling quiesce on snapshot.
2022-03-05 00:38:25 information: snapshot command succeeded on askedfor2.snap for AskedFor2.
2022-03-05 00:44:51 information: copy of /mnt/disk4/Disk4SSD1TB/askedfor2.img to /mnt/user/VMsBackupShare/AskedFor2/20220305_0000_askedfor2.img complete.
2022-03-05 00:44:51 information: backup of /mnt/disk4/Disk4SSD1TB/askedfor2.img vdisk to /mnt/user/VMsBackupShare/AskedFor2/20220305_0000_askedfor2.img complete.
2022-03-05 00:44:57 information: commited changes from snapshot for /mnt/disk4/Disk4SSD1TB/askedfor2.img on AskedFor2.
2022-03-05 00:44:57 information: forcibly removed snapshot /mnt/disk4/Disk4SSD1TB/askedfor2.snap for AskedFor2.
2022-03-05 00:44:57 information: extension for /mnt/user/isos/virtio-win-0.1.190-1.iso on AskedFor2 was found in vdisks_extensions_to_skip. skipping disk.
2022-03-05 00:44:57 information: the extensions of the vdisks that were backed up are img.
2022-03-05 00:44:57 information: vm_state is running. vm_original_state is running. not starting AskedFor2.
2022-03-05 00:44:57 information: backup of AskedFor2 to /mnt/user/VMsBackupShare/AskedFor2 completed.
2022-03-05 00:44:57 information: number of days to keep backups set to indefinitely.
2022-03-05 00:44:57 information: cleaning out backups over 2 in location /mnt/user/VMsBackupShare/AskedFor2/
2022-03-05 00:44:57 information: removed '/mnt/user/VMsBackupShare/AskedFor2/20220226_0000_AskedFor2.xml' config file.
2022-03-05 00:44:57 information: removed '/mnt/user/VMsBackupShare/AskedFor2/20220226_0000_232c1702-aff4-5622-15ff-9f3557867781_VARS-pure-efi.fd' nvram file.
2022-03-05 00:44:57 information: did not find any vdisk image files to remove.
2022-03-05 00:44:57 information: did not find any vm log files to remove.
2022-03-05 00:44:57 information: removing local AskedFor2.xml.
2022-03-05 00:44:57 information: AskedFor1 can be found on the system. attempting backup.
2022-03-05 00:44:57 information: creating local AskedFor1.xml to work with during backup.
2022-03-05 00:44:57 information: /mnt/user/VMsBackupShare/AskedFor1 exists. continuing.
2022-03-05 00:44:57 information: skip_vm_shutdown is false and use_snapshots is 1. skipping vm shutdown procedure. AskedFor1 is running. can_backup_vm set to y.
2022-03-05 00:44:57 information: actually_copy_files is 1.
2022-03-05 00:44:57 information: can_backup_vm flag is y. starting backup of AskedFor1 configuration, nvram, and vdisk(s).
2022-03-05 00:44:57 information: copy of AskedFor1.xml to /mnt/user/VMsBackupShare/AskedFor1/20220305_0000_AskedFor1.xml complete.
2022-03-05 00:44:57 information: copy of /etc/libvirt/qemu/nvram/1654e5f3-480a-e5c8-0a8e-f9922c315dc0_VARS-pure-efi.fd to /mnt/user/VMsBackupShare/AskedFor1/20220305_0000_1654e5f3-480a-e5c8-0a8e-f9922c315dc0_VARS-pure-efi.fd complete.
2022-03-05 00:44:57 information: able to perform snapshot for disk /mnt/disk4/Disk4SSD1TB/askedfor1.img on AskedFor1. use_snapshots is 1. vm_state is running. vdisk_type is raw
2022-03-05 00:44:57 information: qemu agent found. enabling quiesce on snapshot.
2022-03-05 00:44:58 information: snapshot command succeeded on askedfor1.snap for AskedFor1.
2022-03-05 00:53:02 information: copy of /mnt/disk4/Disk4SSD1TB/askedfor1.img to /mnt/user/VMsBackupShare/AskedFor1/20220305_0000_askedfor1.img complete.
2022-03-05 00:53:03 information: backup of /mnt/disk4/Disk4SSD1TB/askedfor1.img vdisk to /mnt/user/VMsBackupShare/AskedFor1/20220305_0000_askedfor1.img complete.
2022-03-05 00:53:08 information: commited changes from snapshot for /mnt/disk4/Disk4SSD1TB/askedfor1.img on AskedFor1.
2022-03-05 00:53:08 information: forcibly removed snapshot /mnt/disk4/Disk4SSD1TB/askedfor1.snap for AskedFor1.
2022-03-05 00:53:08 information: extension for /mnt/user/isos/virtio-win-0.1.190-1.iso on AskedFor1 was found in vdisks_extensions_to_skip. skipping disk.
2022-03-05 00:53:08 information: the extensions of the vdisks that were backed up are img.
2022-03-05 00:53:08 information: vm_state is running. vm_original_state is running. not starting AskedFor1.
2022-03-05 00:53:08 information: backup of AskedFor1 to /mnt/user/VMsBackupShare/AskedFor1 completed.
2022-03-05 00:53:08 information: number of days to keep backups set to indefinitely.
2022-03-05 00:53:08 information: cleaning out backups over 2 in location /mnt/user/VMsBackupShare/AskedFor1/
2022-03-05 00:53:08 information: removed '/mnt/user/VMsBackupShare/AskedFor1/20220226_0000_AskedFor1.xml' config file.
2022-03-05 00:53:08 information: removed '/mnt/user/VMsBackupShare/AskedFor1/20220226_0000_1654e5f3-480a-e5c8-0a8e-f9922c315dc0_VARS-pure-efi.fd' nvram file.
2022-03-05 00:53:09 information: removed '/mnt/user/VMsBackupShare/AskedFor1/20220226_0000_askedfor1.img' vdisk image file.
2022-03-05 00:53:09 information: did not find any vm log files to remove.
2022-03-05 00:53:09 information: removing local AskedFor1.xml.

 

 

If I'm understanding your issue correctly, I had the same problem. The issue is...

 

2022-03-05 00:53:08 information: number of days to keep backups set to indefinitely.

 

If you set the number of backups to keep to N but leave the number of days to keep backups as "0" then the logic fails and you end up with old image files being left behind.  This should be considered a bug, if number of backups is set but number of days to keep isn't set or is set to 0 then the script should be smart enough to only use one of those values.

 

I have backups run daily and set number of backups to keep and number of days to keep backups both to "7" and that has fixed my issue with old images being left behind.

Link to comment
36 minutes ago, DieselDrax said:

If I'm understanding your issue correctly, I had the same problem. The issue is...

Well , i would agree if that would be true for all the backups of all VMs , but it only happens to some

therefore script does follow the correct logic , but not in all cases

Link to comment
  • 1 month later...

Hi, having issues with the plugin removing old backups. 

 

I have number of backups to keep set to 2 at the moment. 

 

I have a Windows 10 VM, and this works perfectly, the script runs, backs up, and then deletes any other than 2 versions, no problem - the file type is .img. 

 

I have 2 other VMs, LibreNMS which is also a .img file, and Home Assistant which is a qcow2 file. 

 

When the backup runs, it deletes all the config related files for any backup over 2, but doesnt actually remove the disk image, so im left with manually having to clear out just the disk images. 

 

Ive pasted the log from the last back-up job, and highlighted this...

 

2022-05-08 06:05:47 information: backup of /mnt/user/vms/librenms/old_librenms-ubuntu-20.04-amd64.img vdisk to /mnt/user/sysbackup/vms/autobackup/LibreNMS/20220508_0440_old_librenms-ubuntu-20.04-amd64.img complete.
2022-05-08 06:05:47 information: the extensions of the vdisks that were backed up are img.
2022-05-08 06:05:47 information: vm_state is shut off. vm_original_state is running. starting LibreNMS.
2022-05-08 06:05:48 information: backup of LibreNMS to /mnt/user/sysbackup/vms/autobackup/LibreNMS completed.
2022-05-08 06:05:48 information: number of days to keep backups set to indefinitely.
2022-05-08 06:05:48 information: cleaning out backups over 2 in location /mnt/user/sysbackup/vms/autobackup/LibreNMS/
2022-05-08 06:05:48 information: removed '/mnt/user/sysbackup/vms/autobackup/LibreNMS/20220424_0440_LibreNMS.xml' config file.
2022-05-08 06:05:48 information: did not find any nvram files to remove.
2022-05-08 06:05:48 information: did not find any vdisk image files to remove.

 

So the backup says it cant find a disk image to remove, but looking in the directory, its there...(24th April 2022)

 

-rw-r--r-- 1 root   root   20G Apr 24 05:47 20220424_0440_old_librenms-ubuntu-20.04-amd64.img
-rw-rw-rw- 1 root   root  6.4K May  1 05:39 20220501_0440_LibreNMS.xml
-rw-r--r-- 1 root   root   20G May  1 05:44 20220501_0440_old_librenms-ubuntu-20.04-amd64.img
-rw-rw-rw- 1 root   root  2.6K May  1 05:44 20220501_0440_unraid-vmbackup.log
-rw-rw-rw- 1 root   root  6.4K May  8 05:59 20220508_0440_LibreNMS.xml
-rw-r--r-- 1 root   root   20G May  8 06:05 20220508_0440_old_librenms-ubuntu-20.04-amd64.img
-rw-rw-rw- 1 root   root  2.8K May  8 06:05 20220508_0440_unraid-vmbackup.log

 

Any ideas why its happy with the Windows 10 backup, sees the img file, and deletes anything older than 2 files, but won't do this for LibreNMS (and the qcow2 for the other VM) but does delete all the other data older than 2 for the other vms? Thanks

20220508_0440_unraid-vmbackup.log

Link to comment

same happens with my 7 VMs , some of them get backed up , some dont , and they are all win10

so its something to do with unraid being able to identify the images.

 

I ended up opting for timing option tho , just set it to 14 days (in my case i do backups every week and want to keep 2 old copies)

 

also one thing to consider: VMs that are backed up properly , and those that don't changed when i changed the name of the VM

maybe that is something that could give ppl ideas on whats causing it.

Link to comment
54 minutes ago, rorton said:

 Do you have number of backups and number of days both set?

Yep, I actually set it to 15 days, because it was wonky with 14 or 13 I don’t remember exactly , and it actually saves 3 copies for those which images “number of days to keep” doesn’t recognize , but the issue was that it counted from the day it started at 23:00 and sometimes took more than an hour which made it think it’s more days old than it actually was, sadly it was too long ago for me to remember exactly but you get the gist 

Link to comment

Even if you delete the config and only use Default, you get the message.

 

With Default there is no message. However, the old config seems to be still somewhere in the planner because it still triggers the message.

 

Event: Unraid Status
Subject: VM Backup plugin
Description: cannot run /boot/config/plugins/vmbackup/configs/Test/
Importance: warning

2022-05-12 00:30 User script file does not exist. Exiting.

 

Which makes sense this time, because you deleted the config. But how do I get it out of the scheduler?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.