[VM Backup Plugin]

Alright, so I found one other issue to fix with the plugin, plus some errors in my last update notes (fixed above and in the code)

 

v0.2.7 - 2024/04/21a

Loud Sounds in the Night

• fix v0.2.6 update notes
• fix config directory not being created automatically

And to repeat my note from the post above:

While I genuinely do not have the time to spend much time maintaining this right now, I am happy add others to the repository or review PRs for fixes/updates/etc

[VM Backup Plugin]

For those (that like me) still find the plugin still works for them I have released a minor update today with two fixes:

 

v0.2.6 - 2024/04/21

Booger Nights

• workaround for issue with empty path with unassigned devices (thanks to Juugis)
• fix issue with plugin requiring internet at server boot (thanks to realizelol)

• respect existing installed packages that might be used by other plugins (thanks to realizelol)

• moved old cronjob removal to pre-install step (thanks to realizelol)

While I genuinely do not have the time to spend much time maintaining this right now, I am happy add others to the repository or review PRs for fixes/updates/etc

[VM Backup Plugin]

9 hours ago, ncohafmuta said:

I have this problem as well. I ended up editing the number_of_backups_to_keep var in the user-script.sh script directly.

 

On 8/23/2022 at 5:10 AM, IronBeardKnight said:

I have discovered what I think may be a bug with this plugin.

 

I'm sure this one could be a simple fix but its possible I might be missing something.
 

Using 2 for this field allows the plugin to work as expected.

 

for some reason I cannot select 1 backup for the amount of copies as I only want one backup and then I use other tools to ship a copy encrypted offsite.

 

Please see below how it turns read then will not allow you apply 1 even thought the vm only has 1 vdisk.

 

You cannot proceed and press apply when its red.

@ncohafmuta and @IronBeardKnight you can bypass pretty much every safety check that won't actually break the script in the Danger Zone tab.
Scroll all the way to the bottom and look for Disable restrictive validation and set it to Yes, after that you can set Number of backups to keep to 1 without the script complaining (screenshots attached).

 

I am intending to update the help to note which fields can be deactivated and where to do that in a future update, but for now hopefully this helps. 

 

fwiw, the plugin is very customizable and I didn't want to prevent anyone from doing something just because I considered it non-standard (this is one of my biggest pet-peeves with software!). Because of that, I made sure to include options to disable anything that wasn't explicitly breaking. The only reason it doesn't let you do it out of the box is because it is too easy for someone who just wants an easy-to-use backup plugin to make a mistake that doesn't protect them or causes data issues.

I highly recommend reading through the help for the more advanced options if you ever run into something like this again... or even just for good measure - there may be other useful things in there you want to use!

[VM Backup Plugin]

On 3/25/2021 at 3:03 PM, jlficken said:

I get this error:

 

+==============================================================================
| Skipping package vmbackup-v0.2.3-2021.03.11 (already installed)
+==============================================================================

This is something I found right after the last release. It will be fixed in the next version. Glad you figured it out though.

If anyone else runs into this, you can also fix it by running this from the terminal (which will force the vmbackup plugin to fully uninstall).
 

removepkg vmbackup*

 

[VM Backup Plugin]

1 minute ago, peter76 said:

THANX JTOK - now I can run ... */3 ...
PS: it's only "Disable custom cron validation?" which has to be set to YES

 

Yeah, that should be the only thing you need to change. If it isn't, then I have something to fix haha

[VM Backup Plugin]

@CS01-HS Your issue can be fixed by going to the Danger Zone tab and setting "Disable restrictive validation" to "Yes"

 

@peter76 your issue can be fixed by going to the Danger Zone tab and setting "Disable custom cron validation" to "Yes"

Additional Information:
 

@CS01-HS There are limits in place to prevent people from choosing values that could be dangerous without making sure they understand them. Also, if you only need 1 backup, why not just use "Number of Backups to Keep" and set it to 1 instead of using "number of days to keep backups"?

@peter76 With cron it is very complicated to validate all possible options, so I went with  something that would cover most scenarios while leaving the disable switch to allow for more complex ones like yours.



I've been trying to make the plugin's default state the bare minimum amount of restrictive necessary to help protect less experienced users from themselves, while still giving power users the ability to remove those restrictions and be more adventurous.

Nearly every setting can can be overridden somewhere, but I know that isn't very obvious right now. Making that more apparent is on my list of things to do in a future release.

 

[VM Backup Plugin]

2 hours ago, Squid said:

Why not bump the version of the plugin so that every version to avoid any issues?

 

Normally I would have, but I just didn't have a chance to yet. My life is still a bit of a mess after moving (I'm currently sitting on the floor while I type this because most of my furniture won't arrive for another week haha).

 

I'm planning on incrementing the version later today though, but I wanted to get the quick and dirty fix out sooner rather than later.

 

[VM Backup Plugin]

I finally found the issue. For some reason, several of the files had their EOL character changed when I pushed the last update (I'm blaming GitHub).

 

As long as you have removed the broken version this should work.

If you get an error saying that the plugin is already installed when you try to install this version, you will need to remove the the old version.

 

If you have already removed the old version and are still getting an error saying that the plugin is already installed, run the following terminal command and try again.

removepkg vmbackup*

 

[VM Backup Plugin]

I just wanted to check in and say I am still working on this. Unfortunately my new ISP couldn’t get my Internet going yesterday, and had to schedule another appointment for earlier today. Which meant I wasn’t able to start on this until now.

Hopefully it is a quick fix


Sent from my iPhone using Tapatalk

[VM Backup Plugin]

Dang. Sorry for the issues with the latest version. I wasn’t having them on my server (although mine was still on 6.8.3), but I must have messed something up when I uploaded the latest version to GitHub.

I am arriving at my new place this afternoon, so I’ll update my server and try to get a fix out tonight.


Sent from my iPhone using Tapatalk

[VM Backup Plugin]

23 hours ago, malac said:

I'll the following error

Package pigz-2.3-x86_64-2_slonly.txz installed.
plugin: downloading: https://raw.githubusercontent.com/jtok/unraid.vmbackup/master/vmbackup-v0.2.2-2021.02.03.txz ... failed (Invalid URL / Server error response)
plugin: https://raw.githubusercontent.com/jtok/unraid.vmbackup/master/vmbackup-v0.2.2-2021.02.03.txz download failure (Invalid URL / Server error response)
 

Whoops! typed the wrong filename. I think it should work now.

[VM Backup Plugin]

In light of the official release of 6.9.0 I have removed the max version restriction. I am not able to confirm how well it works this week because I am moving across several time-zones.

 

 

I am also still working on a new version.

As a quick heads up, the new version will be dropping support for pigz, but gzip will still work for anyone using legacy compression (this change should not be breaking or require any user input).

I also fixed a few bugs, and am adding a few new features.

[VM Backup Plugin]

Hello, long time no see.

I am truly sorry to see so many of you have had an issue with this plugin, and it was not my intention to abandon it for as long as I have. Sadly, life had other plans (as it often does). I've recently found myself with time to tinker again, and as such I've released an update that does a few things to try and address some of the issues I am aware of. Unfortunately, I haven't been able to replicate many of the issues others are having, so my ability to test has been limited.

 

I also can see that the operation of some of the advanced functions isn't immediately clear either. A few of the issues people have had could have been resolved with a settings change in one of the more advanced settings tabs (usually Danger Zone). I'll try to find a way to make some of those things clearer as I progress on addressing the larger issues.

 

The big issue I'm attempting to address in the new release is the issue some people have had with the array not wanting to stop. I have adjusted how the backup scripts are checked and it will hopefully be able kill stuck ones more readily.

 

The other thing I have done is set a max version of 6.8.3 until I have time to test on 6.9. I currently do not have a test server, so it could be a bit before I have a chance to spin one up for troubleshooting.

 

Again, I do sincerely apologize for those that feel I left them in the lurch. Thankfully, as others have rightly pointed out, the script that the plugin uses on the back-end is much more stable and a solid way to go.

I would also like to add that I am more that willing to add contributors to the project if others would like to help me maintain it.

 

Best,

JTok

[VM Backup Plugin]

On 2/17/2020 at 2:29 PM, DoeBoye said:

It was mentioned once before, but I didn't see the response (My apologies if it has already been brought up). I was wondering if support for passthrough disk backup is in the cards? I understand Snapshots wouldn't work (The vm will need to be shutdown), but it would be great if the disk could be backed up to an img using something like qemu-img convert...

That's definitely something I would be willing to implement at some point, but there is other functionality that I would like to ensure is working before I try to mess with it.

Unfortunately, I cannot promise more than that right now.

[VM Backup Plugin]

I am currently focusing on implementing a restore function next.

 

Based on feedback, I have decided that in order for restore functionality to work, you will need to allow the plugin to convert the existing backup structure. This will be optional, but without it, you will need to perform restores manually.

 

As development progresses, I will work out additional details and reach back out here for feedback.

 

[VM Backup Plugin]

Some behind the scenes changes thanks to @sjerisman that include zstandard compression and updates to the logging.

Make sure you read the help if you are going to be switching from legacy gzip compression to zstandard.

 

I also made some changes to how configs are shown to the users that should make it clearer how they work, and what is currently being edited.

 

Full change-log below.

 

 

v0.2.1 - 2020/02/20
Pika Pika

- merged changes from unraid-vmbackup script v1.3.1.
- added seconds to logs.
- added option to use zstandard compression.
- added option to create a vm specific log in each vm's subfolder.
- added config drop-down selection to the top of each tab.
- updated method used to determine cpu thread count.

 

https://github.com/JTok/unraid.vmbackup/tree/v0.2.1

 

-JTok

[VM Backup Plugin]

On 2/14/2020 at 10:03 AM, HammerNL said:

Did you find anything interesting, or do you know how I should troubleshoot this problem?

I think I might have an idea. How are you having the backups cleaned?
Are you using "Number of days to keep backups" or "Number of backups to keep"?

 

Because it looks like you are using "Number of backups to keep". In order for that to properly function, the vdisks must be named in a specific way.

If you expand the help, it should give the following info:
"If a VM has multiple vdisks, then they must end in sequential numbers in order to be correctly backed up (i.e. vdisk1.img, vdisk2.img, etc.)."

 

-JTok

[Unable to update docker images]

@torch2k

 

Hmmmm 🤔. Okay, just did some checking and it looks like docker looks for pigz by default.

That said, when the plugin installs pigz, it does add /usr/bin/unpigz. Possibly your installation was corrupted somehow, since I just checked my dockers and two of them needed updated. I then ran Update All without issue.

 

It seems like reinstalling the plugin could fix the issue for you... unless I'm missing something?

 

-JTok

 

[Unable to update docker images]

Will push a fix. Thx.


Sent from my iPhone using Tapatalk

[VM Backup Plugin]

I have set number of backups to keep to 2, but's it's not deleting the actual image, only the .xml and .fd files. Is it because of the qcow2 extension? 

It shouldn’t matter what the extension is, but I’ll create a qcow2 drive this afternoon and run some tests.


Sent from my iPhone using Tapatalk

[VM Backup Plugin]

8 hours ago, sjerisman said:

Just a thought... What if any old backup files were converted to the new format after a plugin upgrade or after opting into the new naming convention? (i.e. use the filenames read and then remove the timestamps and move the files to new subfolders)

 

That's a good idea, I was a little worried about messing with people's backups, so that's why I was leaning towards the more hands-off approach. Though I suppose it would be easy enough to make it optional 🤔, and even include a button in Danger Zone to do it manually at a later time if it's opted out of it during the initial transition.

 

[VM Backup Plugin]

I like the cleaner file structure, but would not want to lose the ability to clean up my older backups. Thinking out loud here. Could you take a listing of the folders within the backup directory and convert that to a date or variable to use? Say the backup location is /mnt/user/backups and then the VM Backup app creates dates as directories from there. I know within Linux you can get a listing of files or directories that are older than X days. Is there a way to generate this list on the fly and then purge out old directories that are older than X? Within the app the user can define the value for X.

 

Sorry, I was unclear. What I meant was that once you switch to the new structure, backups made under the old structure will need manually cleaned out until only new structure backups exist.

Backups made under the new structure would still be able to be cleaned out.

I will update the original post to clarify this.

 

 

Sent from my iPhone using Tapatalk

[VM Backup Plugin]

I am looking to get some feedback on the following:

 

I'd like to change how the plugin saves backups, but it would be a breaking change. I want to change the timestamp from being part of the filename to being a sub-folder.

 

Instead of this structure:

/backup_dir/vm_name/20200204_1555_vm_name.xml/backup_dir/vm_name/20200204_1555_asdf564asd12sd5dfsd.fd/backup_dir/vm_name/20200204_1555_vdisk1.img/backup_dir/vm_name/20200204_1555_vdisk2.img/backup_dir/vm_name/20200205_1555_vm_name.xml/backup_dir/vm_name/20200205_1555_asdf564asd12sd5dfsd.fd/backup_dir/vm_name/20200205_1555_vdisk1.img/backup_dir/vm_name/20200205_1555_vdisk2.img

 

 

Backups would have this structure:

/backup_dir/vm_name/20200204_1555/vm_name.xml/backup_dir/vm_name/20200204_1555/asdf564asd12sd5dfsd.fd/backup_dir/vm_name/20200204_1555/vdisk1.img/backup_dir/vm_name/20200204_1555/vdisk2.img/backup_dir/vm_name/20200205_1555/vm_name.xml/backup_dir/vm_name/20200205_1555/asdf564asd12sd5dfsd.fd/backup_dir/vm_name/20200205_1555/vdisk1.img/backup_dir/vm_name/20200205_1555/vdisk2.img

 

 

this should come with a few advantages:

 

1. implementing restore functionality should be much easier and therefore quicker to implement.
2. structure of backups will be less cluttered.
3. handling files not part of a normal vm structure (such as in the case of SpaceInvaderOne's Macinabox) will be easier to handle.

However, this will break the existing file cleaning functions. So you would need to manually clean out old backups until all your backups were part of the new structure. So once you switch to the new structure, only new structure backups would be cleaned by the plugin.  

 

 

So with that in mind, here is my proposal:

Upon installation of whatever version contains this new structure, the first time you open the plugin page it will prompt you to choose which structure you want to use with a warning message (and a link to further explanation).

Don't worry if the plugin auto-updates before you choose a structure, it will continue to use the old structure so your backups will still run even if you haven't picked one yet.

Moving forward, development will focus on the new structure. the features that are part of the old structure will stay and keep working, but will not be updated to support new functionality. This will almost certainly include restoring and scenarios like Macinabox.

 

This way, everything continues to work as it has in the past, and gives everyone time to transition at their own pace.

 

Then, somewhere down the line a discussion regarding removing the old structure entirely may take place if it becomes necessary.

 

 

Thoughts? Concerns?

 

Thanks,

JTok

[VM Backup Plugin]

19 hours ago, DBJordan said:

Is there a way to get it to back up the rest of the files? I can probably survive without the icons, but I'm not sure if the VM will work without the others.

I'll have to make some changes to the script to handle that scenario, but it should be reasonably easy. I think I've got a general way to handle this that should work for other similar situations down the line. I'll add that to my list of things to do.

[VM Backup Plugin]

On 2/3/2020 at 9:57 AM, Fiservedpi said:

how would i go about restoring? from said back-up 

Do I just replace the vdisk1.img with the backed up one of my choice?

Generally that should work, but if you run into issues, you may also need to replace the config and/or nvram.