Jorgen Posted May 20, 2018 Share Posted May 20, 2018 2) Run the script with the vdisks_to_skip field filled out for one of the vdisks so that only one gets backed up. Then rename the one vdisk you have a backup of and change the VM configuration.Cheers JTok, went with option two and it seems to be working as intended now.Sent from my iPhone using Tapatalk Quote Link to comment
DZMM Posted May 31, 2018 Share Posted May 31, 2018 (edited) A bug in your script just saved my bacon! My pfsense VM crashed and got corrupted and I'd lost all my config backups as I think they were in the same folder where the xml backup is done. Anyway, I luckily spotted that the script had backed up the pfsense VM image file even though I had set it not to! # list of specific vdisks to be skipped. use the full path. vdisks_to_skip=" /mnt/disks/sm961/domains/Woody/vdisk1.img " I didn't want to backup the image as I didn't want my whole network to go down and 'normally' restoring via the config is easy. Anyway, thanks for the bug as it saved my life. I've now made multiple copies of my config so please feel free to fix the bug - logs attached 20180528_0200_unraid-vmbackup.log Edited May 31, 2018 by DZMM 1 Quote Link to comment
Ruthalas Posted June 10, 2018 Share Posted June 10, 2018 Hello folks, I have been using this to back up a VM, and a recent power outage corrupted the VM. I copied over the img, but that was insufficient to restore the VM. I assume this means the configuration is damaged, and while I believe that is part of the backup (the xml file?) I am not sure the proper way to utilize that in the restore process. Can someone guide me in properly restoring from the backup? Quote Link to comment
dadarara Posted September 8, 2018 Share Posted September 8, 2018 Is there a way to shutdown OSX VMs by the script (or remotely in general)? as far as I know I cant right now. i need to cleanly shutdown manually from within or schedule a shutdown just before the script runs. Quote Link to comment
Stupifier Posted September 8, 2018 Share Posted September 8, 2018 Is there a way to shutdown OSX VMs by the script (or remotely in general)? as far as I know I cant right now. i need to cleanly shutdown manually from within or schedule a shutdown just before the script runs. Not sure what you're talking about. When I run this script (with the proper settings)....each of my VMs automatically shut down before they backup in sequence.Recheck your settings in the script....there are MANY settings Quote Link to comment
dadarara Posted September 8, 2018 Share Posted September 8, 2018 What I mean is that the Windows based VMs do shutdown. the Apple OS HighSierra and even before , can not. its not working if I try to stop the VM. the only way is to manually shutdown from within the VM (inside the OSX). Quote Link to comment
Stupifier Posted September 8, 2018 Share Posted September 8, 2018 2 minutes ago, dadarara said: What I mean is that the Windows based VMs do shutdown. the Apple OS HighSierra and even before , can not. its not working if I try to stop the VM. the only way is to manually shutdown from within the VM (inside the OSX). No idea. I don't use Mac OS. You can try using the force shutdown option in the settings. Other than that, I dunno. Good luck. Quote Link to comment
bastl Posted September 8, 2018 Share Posted September 8, 2018 Force shutdown is exactly what he is complaining about. It will be forced after some time if the clean shutdown isn't working. Except from a sheduled shutdown inside the VM i have no idea. I quickly tested it and it doesn't work for me either. Sorry Quote Link to comment
Alphahelix Posted October 1, 2018 Share Posted October 1, 2018 First thank you for a good script! I hope it is OK to suggest improvements. I noticed that with time stamp my backup location quickly ran into space issues. So my suggestion is to add some sort of clean-up before backing up. I have done some google search myself and found this command: find ./{backup location} -mtime +{days} -type f -delete Now I am far from being able to edit this script. But it has not held me back from trying to analyse it a bit. But to me it seems like backup location can re-use the backup location already needed to be specified in the script. The amount of days a backup should be kept safe (not deleted) looks like it could be set as a variable too. I don't know if it is possible to use a derivate of this script to only check files backed-up by the script, or maybe just xml and vdi files. Again I am not an expert here. But maybe one out here with the skills can and want to take a look at it? /Alphahelix Quote Link to comment
Alphahelix Posted October 6, 2018 Share Posted October 6, 2018 (edited) Ok I think I have found a way to have the script delete old files. Please correct me if I have pasted the addition a wrong place, or if there is a better way to do this. But here it is: autkvmbackup-with deletion.sh Edited October 7, 2018 by Alphahelix Quote Link to comment
DZMM Posted November 24, 2018 Share Posted November 24, 2018 (edited) On 10/6/2018 at 8:46 PM, Alphahelix said: Ok I think I have found a way to have the script delete old files. Please correct me if I have pasted the addition a wrong place, or if there is a better way to do this. But here it is: autkvmbackup-with deletion.sh There's already a variable to cover this Edit: actually 2 # default is 0. set this to the number of days backups should be kept. 0 means indefinitely. number_of_days_to_keep_backups="0" # default is 0. set this to the number of backups that should be kept. 0 means infinitely. # WARNING: If VM has multiple vdisks, then they must end in sequential numbers in order to be correctly backed up (i.e. vdisk1.img, vdisk2.img, etc.). number_of_backups_to_keep="2" Edited November 24, 2018 by DZMM Quote Link to comment
Alphahelix Posted December 1, 2018 Share Posted December 1, 2018 On 11/24/2018 at 6:03 PM, DZMM said: There's already a variable to cover this Edit: actually 2 # default is 0. set this to the number of days backups should be kept. 0 means indefinitely. number_of_days_to_keep_backups="0" # default is 0. set this to the number of backups that should be kept. 0 means infinitely. # WARNING: If VM has multiple vdisks, then they must end in sequential numbers in order to be correctly backed up (i.e. vdisk1.img, vdisk2.img, etc.). number_of_backups_to_keep="2" I don't seem to be able to find them... I even tried to download the script file again. from https://github.com/danioj/unraid-autovmbackup/blob/master/unraid-autovmbackup-0-4.sh But I could have overseen er newer revision than the 3 year old one from the link. Quote Link to comment
bastl Posted December 1, 2018 Share Posted December 1, 2018 @Alphahelix Jtok improved the original script. You can find his script with the above mentioned feature on his github page. Quote Link to comment
Alphahelix Posted December 1, 2018 Share Posted December 1, 2018 49 minutes ago, bastl said: @Alphahelix Jtok improved the original script. You can find his script with the above mentioned feature on his github page. Thank you! now it makes sense to me. Quote Link to comment
Lebowski Posted January 12, 2019 Share Posted January 12, 2019 On 5/31/2018 at 11:45 AM, DZMM said: A bug in your script just saved my bacon! My pfsense VM crashed and got corrupted and I'd lost all my config backups as I think they were in the same folder where the xml backup is done. Anyway, I luckily spotted that the script had backed up the pfsense VM image file even though I had set it not to! # list of specific vdisks to be skipped. use the full path. vdisks_to_skip=" /mnt/disks/sm961/domains/Woody/vdisk1.img " I didn't want to backup the image as I didn't want my whole network to go down and 'normally' restoring via the config is easy. Anyway, thanks for the bug as it saved my life. I've now made multiple copies of my config so please feel free to fix the bug - logs attached 20180528_0200_unraid-vmbackup.log This is still broken from what i can tell, is this still being maintained? Quote Link to comment
danioj Posted January 12, 2019 Author Share Posted January 12, 2019 20 minutes ago, Lebowski said: This is still broken from what i can tell, is this still being maintained? Nah. I did this and got really sick. My priorities have changed and tbh I have abandoned this. It was a good idea when i had time but even I back up manually now. Quote Link to comment
Lebowski Posted January 12, 2019 Share Posted January 12, 2019 @JTok Took this over it seems, the script works fine but the skip disk is not functioning. Quote Link to comment
JTok Posted January 13, 2019 Share Posted January 13, 2019 So... I honestly I haven't looked at this for more than a few months since it has been working for my needs and my personal life has been rather busy. However, I did start working on (and am planning to eventually finish) some updates to this script as well as a version that works with btrfs snapshots. Things have slowed down a little lately, so I've started getting back into these kind of projects. I will take another look at the vdisk skipping, plus I'm working on adding the ability to parse the vdisk names directly from the xml file as well as a few other things. Unfortunately I don't have a firm timeline right now, but I expect to be able to devote a decent amount of time over the next month or so. -JTok 1 Quote Link to comment
Lebowski Posted January 13, 2019 Share Posted January 13, 2019 Appreciate it, love the script. Nice work. Quote Link to comment
Dikkekop Posted February 4, 2019 Share Posted February 4, 2019 Hi JTok First of all big thnx for the script. Im using your script and want to add a feature which allows to backup running vm's. From just a few tests it seems to work but i have to adjust your script for it to be compatible with your bultin functions like stopping the vm before backup and starting it after. For now i outcommented and added some lines of code to make it work. Backup running vm's: https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit Quote Link to comment
Stupifier Posted February 5, 2019 Share Posted February 5, 2019 2 hours ago, Dikkekop said: Hi JTok First of all big thnx for the script. Im using your script and want to add a feature which allows to backup running vm's. From just a few tests it seems to work but i have to adjust your script for it to be compatible with your bultin functions like stopping the vm before backup and starting it after. For now i outcommented and added some lines of code to make it work. Backup running vm's: https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit Wow....I always thought it was only possible to safely/correctly backup VMs if they are shutdown. Your method is safe!? Quote Link to comment
Dikkekop Posted February 5, 2019 Share Posted February 5, 2019 Don't know yet its stil a w.i.p. and needs testing. From what i understand its safe if guest additions are installed in a vm: NOTE-1: Above, if you have QEMU guest agent installed in your virtual machine, try '--quiesce' option with virsh snapshot-create-as[. . .] to ensure you have a consistent disk state. Quote Link to comment
JonathanM Posted February 5, 2019 Share Posted February 5, 2019 9 hours ago, Stupifier said: Wow....I always thought it was only possible to safely/correctly backup VMs if they are shutdown. Your method is safe!? It should correctly backup the DISK properly and safely, but the running OS will not be given the chance to commit any pending changes. Picture this scenario. You have a document open for editing in the VM. The OS has the file open for writing, but keeps your edits in RAM for speed. The VM is backed up, your open file is backed up in the open pending state. You use this backup to restore, and your document is corrupted because it is restored as being open but no application is going to close it properly. Now, what happens if that same thing happens to critical system files? It may work well 95% of the time, but backing up a running OS without the OS itself being informed or involved in the backup is not safe. If the OS is aware and can prepare properly, it will work fine. 4 hours ago, Dikkekop said: From what i understand its safe if guest additions are installed in a vm: Do you have a list of supported OS versions? Quote Link to comment
Dikkekop Posted February 5, 2019 Share Posted February 5, 2019 This is an w.i.p. i dont have an answer for that. About your "document open for editing". Correct me if i'm wrong Word creates a temp file for editing leaving the original file as it is (maby its marked or something so no other user can edit the file while its open..but i dont think this "mark" is saved in the file itself). When you hit the save button its writes the changes to file. So if you backup while file is open it saves the original file as it is before editing and before saving. If you save the file after backup is done, the next backup wil contain changes made while file was open. I wil test this..... the --quiesce option can be used if guest additions are installed inside the vm and will flush the memory buffers to disk. But i have to test this... again W.I.P. This is al theory.. i want to test things first.. Quote Link to comment
Dikkekop Posted February 8, 2019 Share Posted February 8, 2019 Oke i have done some testing with libvirt commandline tool virsh... and made a basic script.... seems to work. Almost done merging this with: https://github.com/JTok/unraid-vmbackup Version v1.1.4 - 2018/05/19. So the orginal script is still functional with all its parameters/functionality and i will add one to enable "Live backup". Just to be clear this wil not include current RAM state into backup. This will backup only the disk at the current state, it will flush latest i/o to disk before snapshot/backup to get a consistent disk state (requires guest agent inside vm, script will check if it is installed..). If VM is restored it will act like its been forced shutdown. Tested: - backup of windows server 2016 is restorable, also tested an open word file wich does after a restore is still accesable. - backup of an debian vm is restorable while running dd if=/dev/zero of=/root/testfile bs=1G count=1 oflag=direct wich creates a 1 gb file. libvirt is waiting for it (if guest agent is installed inside vm) to finish before creating snapshot/backup so this is really nice. I will post updated script i a couple of days and would like to ask you guys to test. Maby do not test this on your critical vm's/unraid box but thats up to you I will add marks in scipt to mark what i added or edited. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.