VM Backup Plugin


Recommended Posts

Snapshots don't work for me.

 

in logs after a successful backup;

2019-12-23 12:51 information: commited changes from snapshot for /mnt/user/domains/Linux/vdisk1.img on Linux.
2019-12-23 12:52 information: forcibly removed snapshot /mnt/user/domains/Linux/vdisk1.snap for Linux.

 

Backups successful, check attached device is still called snap;

# virsh domblklist 6
 Target   Source
-----------------------------------------------
 hdc      /mnt/user/domains/Linux/vdisk1.snap

Looks like this.

 

# ls -la
total 13294048
drwxrwxrwx 1 nobody users          76 Dec 23 12:52 ./
drwxrwxrwx 1 nobody users          26 Dec 23 12:04 ../
-rw------- 1 root   users     1441792 Dec 23 12:52 .fuse_hidden0002bcca0000044a
-rwxrwxrwx 1 root   users 17179869184 Dec 23 12:46 vdisk1.img*

If you then stop the vm, and try to start it, it will fail as the .snap file doesn't exist. If you rename it to .img it's been rolled back to the pre-backup state.

 

Additionally the extensions to skip is case sensitive, I suspect this is not the intention as there is some setopts -u nocasematch in the code;

I had install media attached to a windows vm; after the above backup it left the .ISO attached device as a .snap too.

 

2019-12-23 12:15 information: able to perform snapshot for disk /mnt/user/recovery/Operating System ISO/9200.16384.WIN8_RTM.120725-1247_X64FRE_SERVER_EVAL_EN-US-HRM_SSS_X64FREE_EN-US_DV5.ISO on WinPrint. use_snapshots is 1. vm_state is running. vdisk_type is raw
2019-12-23 12:15 information: qemu agent found. enabling quiesce on snapshot.
2019-12-23 12:15 failure: snapshot command failed on 9200.16384.WIN8_RTM.120725-1247_X64FRE_SERVER_EVAL_EN-US-HRM_SSS_X64FREE_EN-US_DV5.snap for WinPrint.
2019-12-23 12:15 failure: snapshot_fallback is 0. skipping backup for WinPrint to prevent data loss. no cleanup will be performed for this vm.

 

running unraid 6.7.2

 

Edited by dibbz
woopsie
Link to comment

This version fixes some major issues and adds some advanced functionality.

If you were running a weekly schedule, it will actually be running daily using the previous version of the plugin. To fix this, just disable and re-enable the schedule.

 

v0.1.8 - 2019/12/24
Penny Wise and Pound Stupid

- fixed weekly cronjob.
- fixed validation to allow spaces in paths.
- fixed cronjob not being removed during uninstall.
- fixed extension matching case sensitivity.
- added fix for stuck snapshots.
- added ability to abort running scripts.
- changed default folder start location based on restrictive validation setting.
- updated help.
- updated lists to be sorted alphabetically.

 

https://github.com/JTok/unraid.vmbackup/tree/v0.1.8

 

-JTok

Link to comment

Hi,

Maybe I don't understand how snapshots work. 

I understand you need QEMU Guest Agent in order to be able to create snapshots. So I tried with a Windows 10 VM and it creates a snap file but also create a full vdisk copy in backup directory. Every backup it creates another full disk image. Is that how it works or I'm doing it wrong. I tought that a snapshot saves just the data that's different from the latest vdisk.

Link to comment

@andreidelait

Unfortunately, the snapshot mechanism is just for allowing backups to be performed relatively safely while the virtual machine is running.

 

I'm over-simplifying this a little bit, but essentially, when a VM is running, it is making changes to the disk image. In order to safely make a backup of a disk image and reduce the chance of data loss, we must stop the VM from making changes to the disk image. We can do that two ways currently. The first is by turning off the VM and making a copy of the disk image while the VM is not in use. The second is to use snapshots.

Snapshots work by creating a second disk image for the VM to start writing to instead of the original disk image. Which is why it is just the changes and not a full copy, just like you were thinking. However, since the snapshots are live disk images, we can't safely back them up for the same reason we couldn't back up the original disk image before. But, after making a snapshot, we can safely make copy of the (now static) original disk image.

 

So, you are right that the snapshot itself is the changes to the original disk image, but sadly, we can't safely make backups from them. What you are describing is a delta backup.

 

It is possible I might look into how to perform more complicated backup schemes that use delta backups like incremental or differential backups, but that will be a little ways down the road.

 

-JTok

 

P.S. A note about the QEMU Guest Agent. This is entirely unnecessary information, but since we are on the topic of snapshots, and in case anyone is wondering why it is a good idea to use the guest agent, here is an explanation. Again, I am simplifying a bit.

We can actually create snapshots without the guest agent installed. One of the things the guest agent does is allow the guest OS to be informed of things that we are doing at the host level, such as making a snapshot.

When a snapshot is performed *without* the guest agent installed the guest just keeps writing to the "hard drive" without being aware that the disk file it is writing to has been changed to a snapshot image file. This can leave the data on the original disk image in an inconsistent state, meaning that if you try to restore to it there may be issues with some of the information resulting in data loss or corruption. Certain applications are particularly sensitive to this, such as those with databases like an SQL server or Active Directory.

When a snapshot is performed *with* the guest agent installed, the guest OS is asked to quiesce. Meaning that it will flush any write caches to disk, and let all current writes to the "hard drive" finish while preventing new writes from starting. This prepares the original disk image for a backup by ensuring that its data is in a consistent state. Once the original disk image is in a consistent state, the guest is informed it can start writing again, only this time the writes are going to the snapshot.

Edited by JTok
post script
Link to comment

Thank you so much for this fantastic plugin!!

Crazy, I just wrote down my biggest wish to be a vm-backup solution in the Unraid Forum 100K Giveaway thread.... and now it's coming!

My tests all fished flawlessly and without any problems.

 

I found a very minor error in the help section of "Set backup location":

  Point 3: "To choose a folder in /mnt/ instead of /mnt/users/, disable restrictive validation."

  In fact I chose an unassigned disk (/mnt/disks/Backup/) and I had not to disable the "restrictive validation".

 

I also have a little wish list / ideas...

I slightly modified your old script and added these features:

  • My server beeps when finishing backup steps
    To do this I use the "beep" command with different parameters, so I know if thomething went wrong or right.
    Warning: beep -f 350 -l 700
    Error: beep -f 250 -l 600
    Success: beep -f 530 -l 300 -D 100
    Perhaps you can integrate this feature as an option?

     
  • I mount my unassigned disk before the backup and unmount it afterwards
    So nothing can happen to the backed up data (my vm's and documents) if a virus/ransomware hits my server.
    To achieve this have these steps:
    1. find out what drive has to be mounted and if it's mounted
    2. mount it if it's not mounted
    3. send a notification
    4. backup starts
    5. unmount the disk
    6. send a notification
    Perhaps you coud integrate a "pre-script" and a "post-script" feature, so one can make "things" before and after the vm-backups?

 

Here are my script snippets

  • 1. - 3.
    How it works:
    To be sure that the correct drive is mounted I use the UUID of the disk that I found out before manually with "ls -l /dev/disk/by-uuid".
    Then I get the device name (e.g. /dev/sdd1) on the basis of the UUID and write it in the variable "UDBACKUPDEV".
    Then I define a second variable "UDBACKUPSIZE" to store the size of the disk.
    Finally I check if the size is correct and then mount the disk to start the backup. 

    As I'm writing this I really ask myself why I do so complicated (useless?) tests (in particular concerning the size), but I do remember that I had problems mounting the right disk because sometimes I change the disks in my removable disk bay... 😕
    export BACKUPDISKSHARE='/mnt/disks/Backup'
    
    if ! [ -d "$BACKUPDISKSHARE" ] ; then
        ### Mail Notification
        /usr/local/emhttp/webGui/scripts/notify -e "VM-Backup" -s "VM-Backup begins." -d "Backup disk not yet mounted" -i "warning" -m "`date` \\n The backup disk is not yet mounted. Trying to mount it as $BACKUPDISKSHARE in the system."
        # Ensure to mount the right disk!
        # 1. got the UUID beforehand with "ls -l /dev/disk/by-uuid"
        # 2. get the device name (e.g. /dev/sdd1) on the basis of the UUID and write it in a variable "UDBACKUPDEV"
        UDBACKUPDEV=`findfs UUID=29bb9c6d-a1ee-4189-8146-1c898e314cb2`
        # 3. write the size of the partition in a variable
        UDBACKUPSIZE=`lsblk --output SIZE -n -d $UDBACKUPDEV`
        # 4. if it is 5.5T then mount it
        if [[ "$UDBACKUPSIZE" == " 5.5T" ]] ; then
            /usr/local/sbin/rc.unassigned mount $UDBACKUPDEV
        fi
        sleep 5
        if ! [ -d "$BACKUPDISKSHARE" ] ; then
            # beep
            beep -f 250 -l 600
            sleep 1
            beep -f 250 -l 600
            ### Mail Notification
            /usr/local/emhttp/webGui/scripts/notify -e "VM-Backup" -s "VM-Backup stopped!" -d "VM-Backup-HDD mount $UDBACKUPDEV failed" -i "alert" -m "`date` \\n VM-Backup stopped, because the backup disk $UDBACKUPDEV  could not be mounted."
            exit
        else
            ### Mail Notification
            /usr/local/emhttp/webGui/scripts/notify -e "VM-Backup" -s "VM-Backup disk mounted." -d "VM-Backup-HDD mount $UDBACKUPDEV was successfull, $BACKUPDISKSHARE is ready to use!" -i "normal" -m "`date` \\n The VM-Backup disk $UDBACKUPDEV was mounted successfully!"
        fi
    else
        ### Mail Notification
        /usr/local/emhttp/webGui/scripts/notify -e "VM-Backup" -s "VM-Backup disk mounted." -d "VM-Backup-HDD mount $UDBACKUPDEV was successfull, $BACKUPDISKSHARE is ready to use!" -i "normal" -m "`date` \\n The VM-Backup disk $UDBACKUPDEV was mounted successfully!"
    fi

     

  • 5. - 6.
    ######
    # unmount VM-Backup HDD
    /usr/local/sbin/rc.unassigned umount $UDBACKUPDEV
    # hdparm -y : Standby Mode of the disk
    hdparm -y $UDBACKUPDEV
    #check if drive mounted
    if ! [ -d "$BACKUPDISKSHARE" ] ; then
        ### Mail Notification
        /usr/local/emhttp/webGui/scripts/notify -e "VM-Backup" -s "VM-Backup disk $UDBACKUPDEV unmounted." -d "The VM-Backup disk $UDBACKUPDEV was successfully unmounted." -i "normal" -m "`date` \\n The VM-Backup-Routine is finished and the Backup disk was successfully unmounted and removed from the system."
    else
        ### Mail Notification
        /usr/local/emhttp/webGui/scripts/notify -e "VM-Backup" -s "VM-Backup HDD $UDBACKUPDEV could not be unmounted." -d "The VM-Backup disk $UDBACKUPDEV could not be unmounted." -i "warning" -m "`date` \\n The VM-Backup-Routine has finished, but the Backup-HDD could not be unmounted and removed from the system. Pleas remove it manually."
    fi

     

I hope you dont' find too much spelling errors because I'm german and translated all the comments and notification texts in the script, so you understand what I'm doing...😊

 

Edited by vakilando
Link to comment
6 hours ago, vakilando said:

I found a very minor error in the help section of "Set backup location":

  Point 3: "To choose a folder in /mnt/ instead of /mnt/users/, disable restrictive validation."

  In fact I chose an unassigned disk (/mnt/disks/Backup/) and I had not to disable the "restrictive validation".

This is actually the intended behavior, but the way I worded the help wasn't very clear. I have already updated it in the next release to make it more apparent how the validation will affect the backup location textbox.

 

6 hours ago, vakilando said:

I also have a little wish list / ideas...

I slightly modified your old script and added these features:

  • My server beeps when finishing backup steps
    To do this I use the "beep" command with different parameters, so I know if thomething went wrong or right.
    Warning: beep -f 350 -l 700
    Error: beep -f 250 -l 600
    Success: beep -f 530 -l 300 -D 100
    Perhaps you can integrate this feature as an option?

     
  • I mount my unassigned disk before the backup and unmount it afterwards
    Perhaps you coud integrate a "pre-script" and a "post-script" feature, so one can make "things" before and after the vm-backups?

 

 

I hope you dont' find too much spelling errors because I'm german and translated all the comments and notification texts in the script, so you understand what I'm doing...😊

 

I think have a pre-script and post-script option is a great idea!

I'm not sure if I'll implement the beeping as a direct option, but you should be able to have it parse the log for errors/warnings/etc and then beep accordingly using a post-script (once I implement it).

 

Thanks for the translations 🙂 I had no problem understanding.

 

-JTok

Edited by JTok
clarity
Link to comment
5 hours ago, vakilando said:

Also great would be to choose between:

  • No notification 
  • detailed notification 
  • detailed summary at the end of a backup

I chose detailed an have a hole bunch of mails in my inbox... 

 

I will look into how to send a detailed summary... I think it would be useful, but I'm also not entirely certain how much information I can send through the unraid notification system, or if there is a way to just have it send an email. Either way, I will definitely see if it is an option.

 

-JTok

Edited by JTok
words are hard
Link to comment

This new update is a lot of code optimization to reduce the number of writes to the flash drive, as well as increase the speed that the page loads after the initial run. There are some additional features added as noted in the change-log.

 

 

v0.1.9 - 2019/12/31

Flame Broiled

 

- fixed snapshot extension form validation.

- added file versions to reduce writes to thumb drive.

- added option to run backup even during parity check.

- added alert if enable backups is set to 'No' when apply button is hit.

- added ability to view latest log file.

- updated buttons and pages to use XHR (ajax) for asynchronous loading to reduce page refreshes.

- updated code to reduce page load times.

- implemented md5 hash checks for package files during install.

- removed "vms to backup while running" option due to difficulties with the menu item.

 

 

https://github.com/JTok/unraid.vmbackup/tree/v0.1.9

 

-JTok

 

  • Thanks 1
Link to comment
10 hours ago, JTok said:

- updated buttons and pages to use XHR (ajax) for asynchronous loading to reduce page refreshes.

- updated code to reduce page load times.

First thing I noticed, the page is loaded really quick now. Great improvement 😀

 

Just an idea which might be possible to implement, maybe not. In case you passthrough a physical device (hdd, ssd, nvme), image the whole disk would be a nice feature. Snapshots won't work, sure, but shutdown the VM and make an image of the physical device would be neat. Did you ever thought about such a functionality and maybe played around with it already? I know thats a big thing I'am asking for, handling all the different kind of source devices and filesystems people maybe using might be a hassle.

 

See it only as an idea in case you have to much time left 😂

 

Thanks again for this great plugin and the effort you put into it. I wish you and your family and friends a great start into 2020 🥂

Link to comment

@queueiz @Stupifier

 

 

With cron I have found it extremely difficult to validate for every possible input it will accept because there are so many.

By default I use a more restrictive validation for user input to reduce errors. Since I realize that can prevent valid input that might be desired, there is an option in the Danger Zone tab to remove the validation.

 

I will update the help to try to clarify what kind of input is accepted by default.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.