ZFS BuddyBackup is a plugin that aims to make ZFS snapshot maintenance, and backups between two Unraid servers easier to maintain and more secure. 

Disclaimer: I take no responsibility for any harm caused by using this plugin. Please report any bugs or issues to me.

This plugin is fully open source, and I encourage you to review the source to ensure you understand how it operates and verify its security.

You can find the source code here: https://github.com/Piratkopia13/unraid-buddybackup.

 

Install the plugin through Community Applications.

 

I wrote this plugin to work for my own use case, but it's hopefully of use to others as well. I found it tricky to balance a simple setup with the requirements and customization that I wanted, so this is a short guide to how this plugin works and how it can be used.

 

Before starting, you need to run Unraid 6.12.0 or later and have the data you want to backup to a remote destination on an encrypted ZFS dataset. I enforce encryption to make security a must and not an afterthought. Even if you trust your backup buddy, using encryption is a good idea.

You also need a secure connection to your buddy's machine. This plugin uses SSH over default port 22. Never expose this publicly outside your network, instead set up a VPN connection like TailScale or WireGuard between the machines.

 

ZFS BuddyBackup will automatically set up a non-root ssh user that only has access to run the commands needed to perform backups and restores. The plugin also set up ZFS permission delegation to only allow access to the dataset you specify as the backup target.

 

The plugin settings are divided in 4 important sections: "Backup and restore", "Snapshot creation and pruning", "Buddy's Backups", and Log. Let's go through them one by one.

 

Backup and restore

This is where you set up your backups, or restore data from them. Click Add at the bottom of this section to add a new backup.

In the example image below I have two different backups set up, one with a local dataset as the destination, and one to my Buddy's Unraid server.

On the left side you specify source settings:

• Choose the dataset to backup - it must be encrypted. I also recommend enabling zstd compression on this dataset.
• Enable "Backup child datasets" if you want snapshots in child datasets to be included.
• Cron schedule specifies when the backup will run, see https://crontab.guru/.

On the right side you specify destination settings:

• Set type to either Remote or Local - If you use local backups you can use a non-encrypted dataset as source.
• If type is remote:
  • Specify the VPN IP address to your buddy (who also has this plugin installed), and send your SSH public key, shown the the top of the section, to your buddy. Your buddy should paste this into the "Buddy's SSH public key" field in the "Buddy's Backups" section.
  • The destination dataset is the name of the dataset on your buddy's server that will receive your dataset and snapshots. It must begin with what your buddy has set in the "Buddy's Backups" section, followed by a child dataset name that does not already exist. 
• If type is local:
  • The destination parent dataset is the name of the local dataset that will receive the source dataset and snapshots. Make sure it's set to a dataset that does not already exist.

 

To restore data from any backup, use the "Restore data from destination" button and a restore wizard will show up, looking like this:

Select the snapshot or dataset to restore and it will then prompt you for the destination.

 

Snapshot creation and pruning

If you already create snapshots some other way you can disregard this section. Under the hood the plugin uses Sanoid, so if you know what Sanoid is, this section is basically a GUI for the sanoid conf. If you don't know what Sanoid is, this is where you specify how many snapshots to automatically create and keep. You can add as many datasets as you want to this section by pressing the Add button at the bottom.

• "Recursive" specifies whether options should be applied to datasets under this one, including this one.
• Create/Prune snapshots automatically will create or prune snapshots following the retention settings.
• Snapshot retention specifies how many and how frequent snapshots should be created/pruned. In the example below I keep snapshots every day for the last week, one snapshot every week for the last 4 weeks and one snapshot every month for the last 3 months.
• Trigger backup after snapshot creation allows you to select a backup setup in the above section to be triggered after a new snapshot has been created.

 

 

Buddy's Backups

This is where you set up receiving backups from your buddy.

Enter your buddy's public key as explained in the first section.

Choose a local dataset where your buddy's dataset and snapshots will be placed under. Note that datasets are sent raw (and encrypted) so any settings you do on this dataset, like compression, will not have any effect on the data received.

Snapshot retention lets you specify how many and how long your buddy's backups are kept on your server. Make sure you set this in cooperation with your buddy to balance data usage with how far back they need backups.

 

 

Log

Shows the full log file. In case you need to clear it, the log location in the system is /var/log/buddybackup.log.

 

-----------------------------------------------------------------------

Advanced Settings

Here you will find optional settings for eg. using UTC timezone for logs and setting how many days the last backup was ran before coloring text in the dashboard panel as a warning.

 

Dashboard panel

Buddy backup adds this panel to the Unraid Dashboard. It shows stats on currently active backups, snapshot creation/pruning and a list of when each backup was last completed with their respective size on disk.

Note that the destination size is updated only after a backup is completed, meaning if you do any manual changes in the dataset, the destination size in this panel will not reflect that until the next backup is performed.

 

Future plans

Check out my development backlog here: https://github.com/users/Piratkopia13/projects/1 

Edited March 29, 20251 yr by Pirat

Different people have different setups and different needs for backup solutions. I know there are different solutions circulating already for Unraid and ZFS, but  many use root ssh connections to function which is not okay at all in my opinion. This plugin aims to solve that, among other things.

 

If you use ZFS and need a backup solution between two Unraid servers that this plugin does not satisfy, I would be interested to hear about your setup and why this plugin doesn't work for you. I might not be able to solve the problems, but hearing different needs could help guide the direction I take this plugin in the future.

 

For reference, since I wrote this plugin to be my main backup solution, the following is my setup:

I have a ZFS pool consisting of some nvme ssds where I store all appdata, VM disks etc. In my array I have a couple of HDDs of which one is formatted with ZFS. I perform daily snapshots on the nvme pool and sync those snapshots to the ZFS disk in the array. Every week I sync the same snapshots to a buddy's Unraid server. This follows the classic 3-2-1 backup rule - have three copies of your data, stored on two different types of media, with one copy kept offsite.

The example images in the guide above show how I have this setup in the plugin. It fills all my backup needs, from snapshot creation and pruning, to local backups, to remote offsite backup and restore.

Edited February 9, 20251 yr by Pirat
added my own backup setup for reference

Question/Possible Bug? Below is how I configured this to work, I want local backups from Source: cache/appdata to Dest: zfs/snapshots/appdata, so far so good. However your log is telling me something that doesn't quite add up with what Im doing. Clearly I want cache/appdata under the snapshots creation and pruning section otherwise there would be nothing to snapshot create/prune. Then for the backup/restore I clearly want to RESTORE my snapshots from the destination (zfs/snapshots/appdata)

 

If anything, maybe break the Backup/Restore function into two separate functions. As it is right now, If the user doesn't know any better, assuming most people who might use this are zfs noobs, there isn't any explicit "restore" options to choose. (which is why I think your log message is throwing me a warning because it thinks Im trying to backup a dataset to itself)

 

Also what is the cron schedule for taking the automatic snapshots? I prefer to run my schedule every hour but there's no option for that for snapshot creation? It says automatic but how do you run it and how often does it run?

 

 

 

Edited February 10, 20251 yr by MowMdown

@MowMdown

That warning is very much a bug. It's referring to destination in the "buddys backups" section, which you haven't even set. So it's some logic error, I'll get it fixed. Thanks for reporting! 

 

A button for restoring exists, but is currently available only when the destination is remote, as offsite backups was the focus of this plugin. That said, I might add a restore wizard for local backups in the future as well to make that part easier!

Yeah it's a really neat plugin that is sorely missing from the unraid experience. 

 

Wishlist/Wantlist:

 

- To run the automatic snapshot feature on a cron schedule 

- Ability to restore snapshots from local destination (host machine)

Many of us keep a local copy of snapshots on an parity protected array disk, being able to restore from that would be nice.

 

Thanks for sharing the plugin

@MowMdown

I just pushed an update that fixes the bug you found. I don't think Unraid allows auto updates of manually installed plugins, so you might have to uninstall it and then re-install it with the link from the first post. Your backup configs will be saved!

The title of your plugin is clear, but since this is the first attempt I have seen to make a plugin for buddy backup, is there any hope that BTRFS could be supported in the future as well? The much higher RAM requirements for ZFS and the risk of data-loss if not using a UPS to protect the RAM led me to using a BTRFS array for my setup among others. BTRFS supports snapshots as well, so I would hope it could become a target for this plugin as well in time.

First of all, to clear up possible confusion, ZFS uses the ARC (adaptive replacement cache) to speed up read operations. All write operations are fully protected in case of a power loss. Furthermore, as the name implies, the size of the ARC is adaptive depending on the available system memory. 

On the opposite, as far as I know, btrfs still has the known "write hole" issue in raid5/6 arrays which can cause undetectable data loss on a power loss. That's actually one of the reasons I switched from btrfs to zfs 😁

 

Regarding adding support for btrfs snapshots, I'm definitely not against it! I will put it in the backlog, but my time is limited so don't expect anything soon. 

Really nice plugin but I have problem, I did set up Snapshot creation and pruning mostly like you shown in screenshot in section "Snapshot creation and pruning" but no snapshot are created. I want to backup localy, do I have to setup manual cron to run config for sanoid?

 

Edited February 19, 20251 yr by adamzetpl

@adamzetpl

You just found another bug! For some reason I put the sanoid cron to be enabled only if you have enabled the  "Buddys Backups" section. You can enable that for now and that should set up the automatic sanoid cron for you. I will get it fixed in the next version! 

32 minutes ago, Pirat said:

@adamzetpl

You just found another bug! For some reason I put the sanoid cron to be enabled only if you have enabled the  "Buddys Backups" section. You can enable that for now and that should set up the automatic sanoid cron for you. I will get it fixed in the next version! 

Now cron is working but I get in log:

FATAL: cannot load /boot/config/plugins/buddybackup/sanoid.conf - please create a valid local config file before running sanoid! at /usr/local/emhttp/plugins/buddybackup/deps/sanoid line 917.

 

7 minutes ago, adamzetpl said:

Now cron is working but I get in log:

FATAL: cannot load /boot/config/plugins/buddybackup/sanoid.conf - please create a valid local config file before running sanoid! at /usr/local/emhttp/plugins/buddybackup/deps/sanoid line 917.

 

That's odd. Could you send the output of
cat /boot/config/plugins/buddybackup/sanoid.conf

as well as 

cat /boot/config/plugins/buddybackup/snapshots.cfg

 

The sanoid.conf is created from the snapshots.cfg when you modify anything in the snapshot creation/pruning setting section.

You can also manually force the update of sanoid.conf by running:

/usr/local/emhttp/plugins/buddybackup/scripts/rc.buddybackup.php update_sanoid_conf

but that really shouldn't be necessary.

[]
    hourly = 0
    daily = 7
    weekly = 4
    monthly = 3
    yearly = 0
    autosnap = no
    autoprune = yes
    recursive = yes

[cache/appdata]
    recursive = yes
    autosnap = yes
    autoprune = yes
    hourly = 0
    monthly = 2
    daily = 7
    yearly = 0
    weekly = 4

[7bef260e]
dataset="cache/appdata"
recursive="yes"
autosnap="yes"
autoprune="yes"
hourly="0"
monthly="2"
daily="7"
yearly="0"
weekly="4"
trigger="no"

 

@adamzetpl

Ah. I'm guessing sanoid doesnt like that empty first section (the []). It's there because you have enabled Buddy's Backup section, but not selected a destination dataset.

Try also setting any destination dataset in that section and see if sanoid starts working. (note that it will autoprune snapshots in the selected dataset)

I will get both these bugs fixes asap.

Edited February 19, 20251 yr by Pirat

Now everything is OK

@adamzetpl

I just pushed a new version that fixes the issues you found! After updating, you can disable the buddy's backup section again if you aren't using it.

Thanks for letting me know of the issues! 🏆

Edited February 19, 20251 yr by Pirat

I have little more feedback, i removed plugin (deleted files from boot, restart unraid). After installing fresh I need to enable Buddy's Backups for cron snapshot to be created (after this I can disable it), additional error I get after Backup and restore job runs: "Local backup failed. Error code 2."
It seems that everything is running fine, snapshot are created, dataset is mirrored but I get constant error "Local backup failed. Error code 2.".

I recommend installing the plugin from the apps tab, and updating the plugin should work. No need to manually uninstall/reinstall any more. 

 

Could you confirm with version of the plugin you are on? 

Also, do you get any other print in the log other than "Local backup failed. Error code 2."? This means that syncoid did not exit cleanly and the actual error should have been printed above. 

You could also try running the backup from the button in settings and see if you get anything more printed in the popup. 

Verision of plugin 2025.02.23 its from CA not manual install.

 

Running backup..
CRITICAL ERROR: 
Target backup_hdd/zfs_backup/appdata_backup exists but has no snapshots matching with cache/appdata!
Replication to target would require destroying existing
target. Cowardly refusing to destroy your existing target.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:30:10_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:30:18_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:30:26_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:23_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:01_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:16_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:14_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:30:01_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:30:07_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:06_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:26_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:08_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:29_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:11_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:15:19_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:30:20_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:30:04_daily on target. Nothing to do, not syncing.
INFO: no snapshots on source newer than autosnap_2025-02-24_00:30:15_daily on target. Nothing to do, not syncing.
Local backup failed. Error code 2.
[[rc.buddybackup finished]]

 

I will investigate the cron snapshot not being created on a fresh install, that shouldn't happen 🤔

 

Your syncoid error there looks to me like the destination parent dataset (cache/appdata) doesn't have any snapshots, but all the child datasets do. I think this could happen if you for instance created the destination dataset yourself instead of letting syncoid create it on the first backup. Please verify if that is the case for you, and if it is the easiest way to solve that is manually syncing the oldest snapshot for the parent dataset with `zfs send cache/appdata@name-of-oldest-snapshot | zfs recv -F backup_hdd/zfs_backup/appdata_backup`. Note the -F flag which is destructive and will rollback the destination to match the source.

You could also change the backup destination to a new dataset that doesn't exist and re-run the full backup.

I did run full backup and now everything is OK. Thank you.

Just pushed version 2025.03.02 with support for restoring snapshots from local backups

@MowMdown

Awesome, ill check it out

I have another question, when using local backup so "Backup and restore" part of plugin every snapshots ever created is copied, is there a way to Prune snapshots like "Snapshot creation and pruning"? Local backup is growing fast with all snapshots (even those that were prune in "Snapshot creation and pruning").

You can just add the local destination dataset to the snapshot creation and pruning section. Disable autosnap, enable autoprune and select retention.