January 24, 20251 yr I have a RAID1 BTRFS pool with COW enabled on the domains shares (all shares in fact). I do not wish to use reflinks and create a crash consistent backup. Instead I wish to create a snapshot similar to how VMWARE might backup a VM and then copy the snapshot and write back the changes after the process completes to the original disk. I am using a VM that has EFI and Secure boot - ie Q35 and OVMF as a template. The memory and disk is encrypted while running. The strategy I would like to deploy is as follows: root@Tower:~# virsh list --all Id Name State --------------------------------- 1 Rivendell running 2 GrimaWormtongue running 6 Pallando running root@Tower:~# virsh domblklist --details Pallando Type Device Target Source ----------------------------------------------------------------------------- file disk hdc /mnt/btrfsprotect/domains/Pallando/Pallando.qcow2 file disk hdd /mnt/btrfsprotect/domains/Alatar/alatar1.qcow2 root@Tower:~# cd /mnt/btrfsprotect/domains/Pallando/ root@Tower:/mnt/btrfsprotect/domains/Pallando# ls Pallando.qcow2* CREATE SNAPSHOT root@Tower:/mnt/btrfsprotect/domains/Pallando# virsh snapshot-create-as --domain Pallando --name PallandoBack --atomic --disk-only --no-metadata --diskspec hdc,file=/mnt/btrfsprotect/domains/Pallando/PallandoBackhdc --diskspec hdd,file=/mnt/btrfsprotect/domains/Pallando/PallandoBackhdd Domain snapshot PallandoBack created root@Tower:/mnt/btrfsprotect/domains/Pallando# ls -asxhl total 87G 0 drwxrwxrwx 1 share users 88 Jan 24 10:20 ./ 0 drwxrwxrwx 1 nobody users 76 Jan 20 20:40 ../ 87G -rwxrw-rw- 1 root users 87G Jan 24 10:20 Pallando.qcow2* 6.0M -rw------- 1 root users 6.0M Jan 24 10:20 PallandoBackhdc 196K -rw------- 1 root users 195K Jan 24 10:20 PallandoBackhdd *NOTE CHANGES HAVE OCCURED TO FILE ABOVE (6MB of changes) At this point copy the file Pallando.qcow2 to a remote backup location, which can complete after some time. Cleanup: root@Tower:/mnt/btrfsprotect/domains/Pallando# virsh blockcommit Pallando hdc --wait --active --pivot --delete Successfully pivoted root@Tower:/mnt/btrfsprotect/domains/Pallando# virsh blockcommit Pallando hdd --wait --active --pivot --delete Successfully pivoted root@Tower:/mnt/btrfsprotect/domains/Pallando# ls -asxhl total 87G 0 drwxrwxrwx 1 share users 28 Jan 24 10:21 ./ 0 drwxrwxrwx 1 nobody users 76 Jan 20 20:40 ../ 87G -rwxrw-rw- 1 root users 87G Jan 24 10:21 Pallando.qcow2* root@Tower:/mnt/btrfsprotect/domains/Pallando# virsh list --all Id Name State --------------------------------- 1 Rivendell running 2 GrimaWormtongue running 6 Pallando running root@Tower:/mnt/btrfsprotect/domains/Pallando# Is this method likely to create a successful backup of Pallando.qcow2 that can be copied back in case of a failure?
January 24, 20251 yr Community Expert 1 hour ago, PeteAsking said: I have a RAID1 BTRFS pool with COW enabled on the domains shares (all shares in fact). I do not wish to use reflinks and create a crash consistent backup. Instead I wish to create a snapshot similar to how VMWARE might backup a VM and then copy the snapshot and write back the changes after the process completes to the original disk. I am using a VM that has EFI and Secure boot - ie Q35 and OVMF as a template. The memory and disk is encrypted while running. The strategy I would like to deploy is as follows: root@Tower:~# virsh list --all Id Name State --------------------------------- 1 Rivendell running 2 GrimaWormtongue running 6 Pallando running root@Tower:~# virsh domblklist --details Pallando Type Device Target Source ----------------------------------------------------------------------------- file disk hdc /mnt/btrfsprotect/domains/Pallando/Pallando.qcow2 file disk hdd /mnt/btrfsprotect/domains/Alatar/alatar1.qcow2 root@Tower:~# cd /mnt/btrfsprotect/domains/Pallando/ root@Tower:/mnt/btrfsprotect/domains/Pallando# ls Pallando.qcow2* CREATE SNAPSHOT root@Tower:/mnt/btrfsprotect/domains/Pallando# virsh snapshot-create-as --domain Pallando --name PallandoBack --atomic --disk-only --no-metadata --diskspec hdc,file=/mnt/btrfsprotect/domains/Pallando/PallandoBackhdc --diskspec hdd,file=/mnt/btrfsprotect/domains/Pallando/PallandoBackhdd Domain snapshot PallandoBack created root@Tower:/mnt/btrfsprotect/domains/Pallando# ls -asxhl total 87G 0 drwxrwxrwx 1 share users 88 Jan 24 10:20 ./ 0 drwxrwxrwx 1 nobody users 76 Jan 20 20:40 ../ 87G -rwxrw-rw- 1 root users 87G Jan 24 10:20 Pallando.qcow2* 6.0M -rw------- 1 root users 6.0M Jan 24 10:20 PallandoBackhdc 196K -rw------- 1 root users 195K Jan 24 10:20 PallandoBackhdd *NOTE CHANGES HAVE OCCURED TO FILE ABOVE (6MB of changes) At this point copy the file Pallando.qcow2 to a remote backup location, which can complete after some time. Cleanup: root@Tower:/mnt/btrfsprotect/domains/Pallando# virsh blockcommit Pallando hdc --wait --active --pivot --delete Successfully pivoted root@Tower:/mnt/btrfsprotect/domains/Pallando# virsh blockcommit Pallando hdd --wait --active --pivot --delete Successfully pivoted root@Tower:/mnt/btrfsprotect/domains/Pallando# ls -asxhl total 87G 0 drwxrwxrwx 1 share users 28 Jan 24 10:21 ./ 0 drwxrwxrwx 1 nobody users 76 Jan 20 20:40 ../ 87G -rwxrw-rw- 1 root users 87G Jan 24 10:21 Pallando.qcow2* root@Tower:/mnt/btrfsprotect/domains/Pallando# virsh list --all Id Name State --------------------------------- 1 Rivendell running 2 GrimaWormtongue running 6 Pallando running root@Tower:/mnt/btrfsprotect/domains/Pallando# Is this method likely to create a successful backup of Pallando.qcow2 that can be copied back in case of a failure? Do you have the guest agent installed, if so you may want to freeze the guest using --quiesce, As the VM is running will there be anything cached in ram that is not written to disk? As part of the v7 options I can copy the memory also. What is the guest OS.
January 24, 20251 yr Author 4 minutes ago, SimonF said: Do you have the guest agent installed, if so you may want to freeze the guest using --quiesce, As the VM is running will there be anything cached in ram that is not written to disk? As part of the v7 options I can copy the memory also. What is the guest OS. The guest additions are not yet working for windows server 2025, however at some point they will... so this is not an immediate concern, and can be made to work eventually. I dont believe the RAM can be captured as virsh does not support memory capture of VM's that are secure boot with q35. Main reason for this mothod is it can ensure validity with the --atomic flag. A cp cannot and although extremely rare there is a technical possibility of modification during the reflinking process that can be avoided here as virsh can create a perfect snapshot and generate an error code if it cannot. This means I can be guaranteed a copy of the disk that can be booted later with certainty, at least I think so. Edited January 24, 20251 yr by PeteAsking
January 24, 20251 yr Community Expert I dont have a secure boot machine to test. You may need to take a copy of the nvram in /etc/libvirt/qemu/nvram/ file name uses UUID as part of the name rather than VM. I have started looking at VM Backups that may be included in future releases. This is test code where it takes disks for a non running machine and create a tar file. For running VMs I plan to snapshot and save then commit changes. Also will be a different process if snapshots exist to create a new image from the backing file chain. So this option is ShutdownNS(No Snapshots) $vm = $argoptions['vm']; $backupName= "B" . date("YmdHis"); $backup = "shutdownNS"; $backup = $argoptions['type']; echo "VMBackup $backup\n"; switch($backup) { case "shutdownNS": $disks =$lv->get_disk_stats($vm) ; $cdrom = $lv->get_cdrom_stats($vm) ; #var_dump($disks,$cdrom); file_put_contents("/tmp/filelist","/tmp/filelist\n"); foreach ($disks as $disk) { echo "{$disk['device']} {$disk['file']}\n"; if (isset($disk['file'])) file_put_contents("/tmp/filelist","{$disk['file']}\n",FILE_APPEND); echo "{$disk['device']} {$disk['partition']}\n"; if (isset($disk['partition'])) file_put_contents("/tmp/filelist","{$disk['partition']}\n",FILE_APPEND); } if (is_file("/etc/libvirt/qemu/$vm.xml")) file_put_contents("/tmp/filelist","/etc/libvirt/qemu/$vm.xml\n",FILE_APPEND); $dom = $lv->get_domain_by_name($vm); $uuid = $lv->domain_get_uuid($dom); echo "UUID $uuid\n"; file_put_contents("/tmp/filelist","/etc/libvirt/qemu/nvram/{$uuid}_VARS-pure-efi.fd\n",FILE_APPEND); #file_put_contents("/tmp/filelist","/tmp/filelist\n",FILE_APPEND); shell_exec("tar -cvf - -T /tmp/filelist | pv -ptre -s $(du -bc $(cat filelist.txt) | grep total | awk '{print $1}') | zstd -o /mnt/user/VMbackup/\"$backupName\".tar.zst"); Other options will be. case "shutdownWS": #qemu-img info vdisk1.S20240929191534qcow2 | grep backing /* # qemu-img convert -O raw root@computenode:/mnt/user/domains2/EncDebian# qemu-img convert -f qcow2 -O raw vdisk1.S20240928215938qcow2 vdisk1.bu root@computenode:/mnt/user/domains2/EncDebian# ls -l total 104016275 -rw-r--r-- 1 root root 770834432 Sep 28 22:02 vdisk1.S20240928215938qcow2 -rw-r--r-- 1 root root 53687091200 Sep 28 22:04 vdisk1.bu -rwxrwxrwx 1 root users 53687091200 Sep 28 20:21 vdisk1.img* root@computenode:/mnt/user/domains2/EncDebian# */ break; case "activeNS": break; case "activeWS": break; case "Full": break; shutdown With active nosnaps Active With Full
January 24, 20251 yr Author No you do not need a copy of nvram. Also it is trivial to repair a boot record or a windows machine using an installation iso of windows. It is literally a single cmd command to rebuild. The reason is that if the unraid machine is functioning and you restore the image it will already have a valid copy. If the entire unraid machine dies you will need to boot repair the disk image if you have not kept a copy but it is not really important to do so. I believe ca appdata backup backs up libvirt if you use that. Either way a disk file like what you copy in your script can be repaired and booted even if that is the only thing you have. Im not clear what you are showing me in your code. I agree it shuts down a vm and copies the disk but this is trivial if you are able to turn off the vm. The goal I have is to create a valid 100% safe backup of a running vm. Also unclear is this part “I have started looking at VM Backups that may be included in future releases.” Future releases of what? Do you create and maintain a backup program for unraid? One thing in your scripts is you do not use pipefail… how do you monitor if a script exits abnormally in this case and then subsequently notify and do something? With regards to the commands I am looking at using in the OP they generate fails if there is a problem that can then be detected with proper scripting. If nobody uses this method to backup vm’s I will probably have to just test it myself and check it works properly. Edited January 24, 20251 yr by PeteAsking
January 24, 20251 yr Community Expert 18 minutes ago, PeteAsking said: No you do not need a copy of nvram. Also it is trivial to repair a boot record or a windows machine using an installation iso of windows. It is literally a single cmd command to rebuild. The reason is that if the unraid machine is functioning and you restore the image it will already have a valid copy. If the entire unraid machine dies you will need to boot repair the disk image if you have not kept a copy but it is not really important to do so. I believe ca appdata backup backs up libvirt if you use that. Either way a disk file like what you copy in your script can be repaired and booted even if that is the only thing you have. Im not clear what you are showing me in your code. I agree it shuts down a vm and copies the disk but this is trivial if you are able to turn off the vm. The goal I have is to create a valid 100% safe backup of a running vm. Also unclear is this part “I have started looking at VM Backups that may be included in future releases.” Future releases of what? Do you create and maintain a backup program for unraid? One thing in your scripts is you do not use pipefail… how do you monitor if a script exits abnormally in this case and then subsequently notify and do something? With regards to the commands I am looking at using in the OP they generate fails if there is a problem that can then be detected with proper scripting. If nobody uses this method to backup vm’s I will probably have to just test it myself and check it works properly. It is just testing/prototyping not production PHP code , There is a possible integrated backups being planned for a future release.
January 26, 20251 yr Author The script I will test for the next 3 months is this. Seems to work fine and you can just copy the backed up disk back to unraid and replace the one that is already there. After a few months I will know if there are any issues. I imagine this is safer than using reflinks. virsh list --all virsh domblklist --details Pallando virsh snapshot-create-as --domain Pallando --name PallandoBack --atomic --disk-only --no-metadata --diskspec hdc,file=/mnt/btrfsprotect/domains/Pallando/PallandoBackhdc --diskspec hdd,file=/mnt/btrfsprotect/domains/Pallando/PallandoBackhdd virsh domblklist --details Pallando cp /mnt/btrfsprotect/domains/Pallando/Pallando.qcow2 /mnt/btrfsprotect/VMBackupFol/ cp /mnt/btrfsprotect/domains/Alatar/alatar1.qcow2 /mnt/btrfsprotect/VMBackupFol/ ls -asxhl /mnt/btrfsprotect/Pallando/ qemu-img info --force-share --backing-chain /mnt/btrfsprotect/domains/Pallando/PallandoBackhdc qemu-img info --force-share --backing-chain /mnt/btrfsprotect/domains/Pallando/PallandoBackhdd virsh blockcommit Pallando hdc --wait --active --pivot --delete virsh blockcommit Pallando hdd --wait --active --pivot --delete virsh domblklist --details Pallando ls -asxhl /mnt/btrfsprotect/Pallando/ virsh domblklist --details GrimaWormtongue virsh snapshot-create-as --domain GrimaWormtongue --name GrimaBack --atomic --disk-only --no-metadata --diskspec hdc,file=/mnt/btrfsprotect/domains/GrimaWormtongue/GrimaBackhdc virsh domblklist --details GrimaWormtongue cp /mnt/btrfsprotect/domains/GrimaWormtongue/grima1.img /mnt/btrfsprotect/VMBackupFol/ ls -asxhl /mnt/btrfsprotect/domains/GrimaWormtongue/ qemu-img info --force-share --backing-chain /mnt/btrfsprotect/domains/GrimaWormtongue/GrimaBackhdc virsh blockcommit GrimaWormtongue hdc --wait --active --pivot --delete virsh domblklist --details GrimaWormtongue ls -asxhl /mnt/btrfsprotect/domains/GrimaWormtongue/ virsh domblklist --details Rivendell virsh snapshot-create-as --domain Rivendell --name RivendellBack --atomic --disk-only --no-metadata --diskspec hdc,file=/mnt/btrfsprotect/domains/Rivendell/RivendellBackhdc virsh domblklist --details Rivendell cp /mnt/btrfsprotect/domains/Rivendell/rivendell1.img /mnt/btrfsprotect/VMBackupFol/ ls -asxhl /mnt/btrfsprotect/domains/Rivendell qemu-img info --force-share --backing-chain /mnt/btrfsprotect/domains/Rivendell/RivendellBackhdc virsh blockcommit Rivendell hdc --wait --active --pivot --delete virsh domblklist --details Rivendell ls -asxhl /mnt/btrfsprotect/domains/Rivendell/ virsh list --all ls -asxhl /mnt/btrfsprotect/VMBackupFol/ scp -r /mnt/btrfsprotect/VMBackupFol/ [email protected]:/mnt/314f2211-7128-498b-9d85-354132601877/Unraid/
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.