JTok

I'll have to make some changes to the script to handle that scenario, but it should be reasonably easy. I think I've got a general way to handle this that should work for other similar situations down the line. I'll add that to my list of things to do.

Generally that should work, but if you run into issues, you may also need to replace the config and/or nvram.

Thanks, I'm having trouble tracking down what might be causing this issue, so I will need to find time to set up an NFS share in my test environment. I did notice there was a recent update to unassigned devices relating to NFS shares... it didn't seem like it would be related to your issue, but possibly it will correct it. I'm hoping to have an opportunity to set up a share for testing sometime in the next week, but I'm out of town the next two weekends, so it may take me longer than that if time doesn't permit during the week.

Could you let me know what you have the following set to? Settings: Enable Snapshots Compress Backups Other Settings: Compare files during backup Disable delta syncs for backups Only use rsync for backups Danger Zone: Fall back to standard backups Pause VMs instead of shutting down

@dodgypast it looks like there is an issue with whatever data the script is pulling from the config. are you able to DM/send me the config (xml file) for that VM? Thanks, JTok

I see what you mean, I’ll try to find a way to make this clearer and/or more intuitive. This is the design paradigm used by the rest of unRAID, so I don’t think it is a good idea for me make my plugin the exception as that could cause further confusion. The functions of those buttons are defined by the plugin system, and while I’ve bypassed those default functions to implement them in a specific way, I kept the basic functionality the same for consistency across all the other plugins/forms. Here is a breakdown of how the buttons work by default in unRAID: The Default button resets the form to defaults. The Apply button submits the form. The Done button takes you out of the current plugin/screen. The Done button also changes to Reset if something on the form has been altered but the Apply button hasn’t been hit yet. When it is a Reset button, it will reset the changes in the form back to the last saved settings. Also, you should be getting a warning pop up when you hit the Default button asking you if you are sure you want to reset to Defaults. Sent from my iPad using Tapatalk

I get what you're saying, but I don't want to make creating a profile necessary to get started... or going to any other tabs for that matter. I want it to be as close to working out of the box as possible, with as few fields as possible to fill in to get started, and all of them in the same place. To give you an idea of where my head is at, my design philosophy is that the main tab (i.e. Settings tab) is the only tab you actually need to have a basic functioning backup with reasonable settings. The other tabs are for more advanced configurations that a basic user really may not need. A user shouldn't have to go to any of the other tabs unless they want to explore. I'm trying to avoid intimidating non-technical users because the technical ones will go exploring anyway. I understand that this may not be ideal for every user and situation, but I would prefer to cater to the lowest common denominator and have the plugin be accessible to as many users as possible instead of throwing a wall of options at everyone and forcing them to sort through it. All of that is a big reason why you started with a 'default' config. The other, more practical reason, is backwards compatibility. Everyone with a previous version of the plugin had an existing config that needed to go somewhere, and I don't think anyone would have been thrilled with me if an update wiped it out 😅 Again though, I absolutely agree that configs are very confusing in their current form, so I have already made some small changes to the upcoming version that should help clear up how the configs work some for new users, and I will keep refining it as we move forward. I may move it to a different place at some point when the plugin state is a bit more stable, but for now I'm considering it an Advanced Setting and going to leave where it is. More or less for the reasons above. (Although I have some ideas on where to move it later once things are stable.) I'm not exactly sure what you mean by this? 🤔

@DZMM You create and manage the configs on the Manage Configs tab, but to edit them you choose the "current config" from the drop-down box on the Settings tab. From there you edit the config like normal. When you want to edit a different config, you go back to the drop-down box on the Settings tab. Also, default config cannot be renamed or removed, and will not show up on the Manage Configs tab. So you already start with that config active, but from the Manage Configs tab you can add/edit additional configs. In retrospect, it isn't very obvious how it works, so I'll make some adjustments in the next version.

Another big thanks to @sjerisman for adding this feature. Again, for anyone using the plugin, the features that get added here will make their way into the plugin before too long. v1.3.1 - 2020/01/21 So Say We All - added option to create a vm specific log in each vm's sub-folder. Script here: https://github.com/JTok/unraid-vmbackup/tree/v1.3.1 -JTok

My apologies for long delay since the last release, for those that care about why, I'll leave it in the postscript. In this version there are some major changes in addition to the usual bug fixes. A big bug fix, is that the plugin should now be able to handle parenthesis in your VM names (thanks to squid for suggesting a solution to that). New features include the ability to have multiple configs so that you can run backups on different schedules, as well as the ability to run pre and post scripts with those configs. Compression now uses pigz for multi-threaded compression (though a faster inline compression using zstandard is still coming). Full change-log below. v0.2.0 - 2020/01/21 The Resistance - fixed new config options not always getting added to user config. - fixed parenthesis in vm names. - added ability to run pre and post scripts. - added option to run backup without array started. - added confirmation dialogs to destructive or dangerous buttons. - added support for multiple configs/schedules. - added option to allow multiple configs to run simultaneously. - added option to set compression level. - added option to set the number of threads used during compression. - changed compression program from gzip to pigz to support multi-threading. - changed alert dialogs to use sweet alerts. - updated help for custom cron. https://github.com/JTok/unraid.vmbackup/tree/v0.2.0 -JTok The delay in this release was caused by me overhauling some of the code to prevent the pages from needed refreshed to update content, and I wound up needing to implement my own version of unRAID's markdown and inline-help to make that work the way I wanted.

I’m out of town this weekend and away from my server, so I’m not sure if there’s a better way (or if this is easily accomplished in unraid)... but off the top of my head, can you mount the share locally in unraid first, then backup to the mount point? Sent from my iPhone using Tapatalk

Big fat thanks to @sjerisman for doing all the work to get a zstandard compression option added to the script! This is going to be significantly faster than using the existing compression option, as well as much more efficient. However, it is not backwards compatible, so if you switch you will want to manually trim any old compressed backups. For those of you that are currently using the plugin, this is coming, but it will be a bit longer. With the plugin being so new, there are some additional kinks to work out as well. v1.3.0 - 2020/01/15 Better than standard - added option to use zstd inline compression. Script here: https://github.com/JTok/unraid-vmbackup/tree/v1.3.0 -JTok

@sjerisman I will take a look at the PR today, and follow up with you through there.

It's not your fault. It should work honestly, but it seems to be an issue with parse_ini_file in PHP as far as I can tell. I'm trying to find a workaround, but have been unsuccessful so far. At the very least, I'll try to make it clear they are causing an issue in a future update. At least until I can get it resolved.

Sorry, I wasn't clear. My tests were from an SSD cache array to the Parity Array. So that's also the bottleneck I was referring to. I was attempting to also point out that, to anyone interested in improving throughput, the biggest improvements will come from changing storage around to cut out the Parity Array. i.e. running the VMs on an NVMe unassigned device and backing up to an SSD cache array or vice versa. You're right though, that order of operations does seem a bit excessive doesn't it? lol This seems viable, but there are some issues that I would need to handle related to backwards compatibility before switching compression algorithms outright. I'm going from memory here, so possibly the details are wrong, but I believe it came down to being able to turn the VM back on sooner. Essentially I couldn't guarantee the speed of the system that unRAID would be running on, so I decided to compress after copy because it meant the VM might be able to be turned on sooner (though I honestly can't remember if I tested this or not). Esentially the logic was: Turn off VM -> Copy files -> Turn on VM -> compress files; would result in the VM being off for less time. With snapshots though, this is far less efficient. So I think it will be a good behavior to make configurable in the future. Thanks for looking into this btw!

Do you have parenthesis in any of your VM paths? I've run into some issues with that.

That's going to be a fun one. I don't have 6.8.1 yet because I am using the nvidia plugin, but I'll see if I can figure out what is going on anyway and get back to you.

In what way did it break it? It is difficult to fix bugs if I don't know what happened. Did you make sure to change the vdisk path in your VM config before using snapshots? -JTok

I looked into this a little today, but this is by no means conclusive. In my tests so far I/O has been the biggest bottleneck, not the compression algorithm or number of threads. So using the parity array vs the cache array, or an unassigned device, is probably going to have the biggest effect on performance. Honestly, all things being equal, I only saw about a 15-20% performance improvement with my test VM (though I understand that there could be more pronounced differences with other use cases). I tested using zstd, lbzip2, and pigz. That being said, since there are some performance improvements with a multi-threaded compression utility, I am looking into a good way to integrate something. I suspect, that at least initially, I will stick with pigz because of backwards compatibility issues. Though I may look into adding an option for the other two later on.

I'll have to try and replicate that and see what is going on. Thanks for letting me know

@jpowell8672 At first glance, it looks like it is not seeing any extensions for your vdisks. Can you confirm that they do have extensions, and if so, what are they?

Are you able to get the error message from the error log? It will be saved in the log folder inside your backup location. Thanks, JTok

Where are you seeing the warning, and what are you doing when it occurs? Does anything happen after you get the warning? Thanks, JTok Sent from my iPhone using Tapatalk

@queueiz @Stupifier With cron I have found it extremely difficult to validate for every possible input it will accept because there are so many. By default I use a more restrictive validation for user input to reduce errors. Since I realize that can prevent valid input that might be desired, there is an option in the Danger Zone tab to remove the validation. I will update the help to try to clarify what kind of input is accepted by default.

I love the flexibility to manipulate the OS with plugins and other things from the community. I would be really interested in seeing snapshots implemented for KVM. Sent from my iPhone using Tapatalk