unraid-borg-vm-backup script

[Monster1290]

Hello UnRAID community! In the beginning of this year I started to use UnRAID as my main gaming-NAS rig. As careful user I started to back up important files of the system. Such as libvirt.img, docker.img, flash drive. And also I wanted to backup my main Windows VM (who knows when it will break after another update). All existing solutions are primarily script-based and generally they all just do copying files to array. But if you wanted to have multiple backups of a VM, say for the last 3 days, then you must have multiple copies of vdisks. In this example 3 days x 70GB vdisk = 210GB of storage occupied by backup copies. Sounds not great for me.

 

So I searched for an existing solution that support de-duplication and not found any of them. Then I decided to create my own solution. At that point in time I've been already using JTok script. Then I found program "BorgBackup", checked it and found it useful for my purposes. Based on JTok script and using BorgBackup I made my own script few months ago. I've done debugging and tested it for a few months, fixed few minor bugs and now ready to present it to community. You can find script and instruction on my GitHub repository. Shortly, you need install "BorgBackup" from Nerd Pack plugin, prepare Borg repository in the shell, copy the script to User scripts plugin, adjust script options, set schedule and you good to go. Hope you liked it

 

This script have all features that have JTok script and couple of mine:

• Choose VM's to backup
• Backup running VM using snapshots
• De-duplication using borg
• Full support of borg pruning logic per VM-basis
• Much faster backup time after first run
• Versatile loging and notifications

 

Below is my current state of Borg repository. Currently I have 4 VM's under backup. These VMs have 5 vdisks, 210GB in total size. Repo contains 18 VM backups. As you can see repo occupies only 248GB, but without de-dupliction it's 1,38TB of raw space. It's 5,7 times lower!

Quote

root@HomeNAS:/mnt/user/backups# borg info borg-backup-repo/
Enter passphrase for key /mnt/user/backups/borg-backup-repo:
Repository ID: ff5efcd60b2dc8553f1df28ed7e823bfa0e3b19bef3367dfa3e515ad88ceac77
Location: /mnt/user/backups/borg-backup-repo
Encrypted: Yes (repokey)
Cache: /root/.cache/borg/ff5efcd60b2dc8553f1df28ed7e823bfa0e3b19bef3367dfa3e515ad88ceac77
Security dir: /root/.config/borg/security/ff5efcd60b2dc8553f1df28ed7e823bfa0e3b19bef3367dfa3e515ad88ceac77
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
All archives:                1.38 TB            937.27 GB            248.01 GB

                       Unique chunks         Total chunks
Chunk index:                  125209               499542
root@HomeNAS:/mnt/user/backups# borg list borg-backup-repo/
Enter passphrase for key /mnt/user/backups/borg-backup-repo:
580ea083-974f-1bc6-347f-91fdc56c0bbe-Win10-shut_off-2021-09-30T02:30:02+05:00 Thu, 2021-09-30 02:30:03 [0cc30db1414e9559937d70698651c6d03e0083480c8f456838f3a23fe3131775]
580ea083-974f-1bc6-347f-91fdc56c0bbe-Win10-running-2021-10-31T02:30:12+05:00 Sun, 2021-10-31 02:30:12 [5a237ef16f11b06316a2e3623c47595fe9d790a7ff4c4c1eedce2fa2e70f42c4]
580ea083-974f-1bc6-347f-91fdc56c0bbe-Win10-shut_off-2021-11-07T02:30:02+05:00 Sun, 2021-11-07 02:30:02 [a1ecc497c8266372ec957a8ae5a88a50ae4e1cde9e0122bb67838a4b08627db4]
60eb6949-8ae9-4900-a8bf-6925a097fe08-Win10Copy-shut_off-2021-11-07T03:20:17+05:00 Sun, 2021-11-07 03:20:17 [bade0d7e7bf03849d79644310211f77d5923b853cd0f70346c5f0268e83438dc]
cc10b868-c6b1-e7f6-570f-88aafbe04c20-Fedora-shut_off-2021-11-07T03:25:32+05:00 Sun, 2021-11-07 03:25:32 [c0bc5b8cfdaa5deeb27846bd45e270afe0cc210b7a3dc319096cd7fd991a5982]
1972520b-e489-0186-5990-6d245f69de74-FedoraMATE-shut_off-2021-11-07T03:27:59+05:00 Sun, 2021-11-07 03:27:59 [187de7c3e27f62e2df6224d29cd4652fc3e86ed211f48bd619b2f434ed518f72]
580ea083-974f-1bc6-347f-91fdc56c0bbe-Win10-shut_off-2021-11-10T02:30:02+05:00 Wed, 2021-11-10 02:30:02 [07242b6fc96a4109eb020e1f30944c8348fe53fc6092e8e8128f436f60ee754e]
60eb6949-8ae9-4900-a8bf-6925a097fe08-Win10Copy-shut_off-2021-11-10T02:54:11+05:00 Wed, 2021-11-10 02:54:11 [64fb7a82bc2712e8cbb91cf9ad503a31171b2e09265a9d8b0e58fc2831f71ec2]
cc10b868-c6b1-e7f6-570f-88aafbe04c20-Fedora-shut_off-2021-11-10T02:59:26+05:00 Wed, 2021-11-10 02:59:27 [9623033626cd334650685067857729856f9928f14efb8c97d8dd2ec0d05f5ebc]
1972520b-e489-0186-5990-6d245f69de74-FedoraMATE-shut_off-2021-11-10T03:01:54+05:00 Wed, 2021-11-10 03:01:55 [aa81d0eae6e447cffe939a6589b852ff7aab457761906331d84e448c88a0cfd4]
580ea083-974f-1bc6-347f-91fdc56c0bbe-Win10-shut_off-2021-11-11T02:30:02+05:00 Thu, 2021-11-11 02:30:02 [0221f8f6d5544a9a6a817bbc2591cc0e269a52e7ff20f121fca41a77a78abda0]
60eb6949-8ae9-4900-a8bf-6925a097fe08-Win10Copy-shut_off-2021-11-11T02:40:37+05:00 Thu, 2021-11-11 02:40:38 [c5e795d6221358b363ded60ee65b3ad9a3e47cc2ce0b1b90c01ec9014bd27600]
cc10b868-c6b1-e7f6-570f-88aafbe04c20-Fedora-shut_off-2021-11-11T02:45:56+05:00 Thu, 2021-11-11 02:45:57 [e3b468c896dc894722c42ea31b339e50e339d6b72f7834502898d40fc872b0f6]
1972520b-e489-0186-5990-6d245f69de74-FedoraMATE-shut_off-2021-11-11T02:48:23+05:00 Thu, 2021-11-11 02:48:24 [f8bb5d0c84cfba422dae39d0c2a269441df665da55f49e72930572da450c06bd]
580ea083-974f-1bc6-347f-91fdc56c0bbe-Win10-shut_off-2021-11-12T02:30:02+05:00 Fri, 2021-11-12 02:30:03 [b5b466d96be7a137cce2c14884f7bd8c1b9315653a3ce7302f65b3c984f92e9b]
60eb6949-8ae9-4900-a8bf-6925a097fe08-Win10Copy-shut_off-2021-11-12T02:53:04+05:00 Fri, 2021-11-12 02:53:04 [87eef9c251c9241918bdf8bf58d8577be87d717cdfd5d0f7b538a34ced7bd39b]
cc10b868-c6b1-e7f6-570f-88aafbe04c20-Fedora-shut_off-2021-11-12T02:58:18+05:00 Fri, 2021-11-12 02:58:19 [cd6b2695a9de40ecbced39e8d3e11b900424828d2a464bb99a5ea831fd20b68c]
1972520b-e489-0186-5990-6d245f69de74-FedoraMATE-shut_off-2021-11-12T03:00:45+05:00 Fri, 2021-11-12 03:00:45 [802e5b3b0172f2a09df6217d0513931dc0eb35dc261b658658de138c6992fdb5]

 

 

[JonathanM]

2 minutes ago, Monster1290 said:

I started to back up important files of the system. Such as libvirt.img, docker.img, flash drive.

Don't bother with docker.img, it's recreated from scratch easily by using the previous apps function of CA. Focus on the appdata config folders for your containers instead.

[Monster1290]

6 minutes ago, JonathanM said:

Don't bother with docker.img, it's recreated from scratch easily by using the previous apps function of CA. Focus on the appdata config folders for your containers instead.

Of course, I use Backup/Restore Appdata plugin. It's already helped me once. Also I already encountered issues with BTRFS on docker.img couple of times. And I found it painful to recreate all docker containers (especially when I have 2 custom docker networks), but of course it's not critical.