Hello everyone,

 

I have some questions regarding a reasonable backup strategy. I would like to be able to restore my unraid system fast in case of critical failures (e.g. disk failures).

 

For me there are three reasonable ways:

 

1. Save only data from shares or whole data shares, e.g. movies, pictures, documents. However, in this case I would not be able to restore things like paperless tags or jellyfin metadata.
2. Save data and appdata share. Now "metadata" from paperless, jellyfin and other tools should be included in the backup. Would this also include my docker configurations?
3. Save everything, a full backup of the whole system.

 

 

Since I would like to save the "metadata" of my installed apps, only strategy two and three seem reasonable for my use case. However, which is more advisable in your optinion and which tool should I use?

 

I found the following backup apps that could be suited:

• Appdata Backup/Restore v3
• borgmatic
• duplicacy
• duplicati
• kopia
• luckyBackup
• urbackup
• vorta
• "rsync Incremental Backup" script (writte by mgutt)

 

I should also mention that I want to use external hard drives as backup storage, connected to unraid only for backup. So the backup is mostly separated from unraid. I currently have two 4TB hard drives, one parity and one storage. In the near future, I would like to upgrade to a second 4TB storage drive, and perhaps a third and fourth later.

To keep the backup system (i.e. the hard drives) physically protected and easily accessible, I thought of buying an external case with 2 or 4 bays. That way, all I have to do is take the case, power it up, conenct it to my server and start the backup.

 

In summary:

• What are your thoughts about the different backup strategies?
• Which backup tool is most suitable for the described use case?
• What are your thoughts about the backup system using 2-bay / 4-bay external drives?

 

Thanks for your input.

6 hours ago, stwx said:

in case of critical failures (e.g. disk failures).

Typically disk failures are dealt with by rebuilding from parity, in the case of array disks. You can also have mirrored pools outside the parity array (usually SSDs) for things that need faster access.

 

But, neither parity or mirrors are a substitute for backups. Plenty of ways much more common than failed disks to lose data, including user error.

 

6 hours ago, stwx said:

Appdata Backup/Restore v3

You will use this to backup appdata (docker application working storage), and also libvirt.img (VM definitions). Typically you would back these up to the array, and you could include those backups as sources in your backup plan to external disks.

 

If you have VMs you will probably want to backup their vdisks, there is also a plugin for that.

 

I use rsync script to backup important things to externals for storage offsite, and also to backup less important things (media) to my backup server. My backup server is just parts and disks leftover from previous hardware upgrades, usually powered down.

 

Another very important thing you must always have a current backup of, your flash drive. None of the OS or your configuration is stored on your data disks.

 

You don't need to worry about backing up the OS, it is easily replaced. The OS is unpacked fresh from archives on the flash drive, into RAM, at each boot, and it runs completely in RAM. Think of it as firmware. These archives can be easily replaced (and are when you upgrade).

 

Your configuration is in the config folder on flash. This contains your license, your disk assignments, any and all settings from the webUI, including docker templates.

 

My Servers plugin will keep a current backup of your flash drive on the Unraid cloud. You can also manually download a zipped backup of flash at Main - Boot Device - Flash - Flash Backup.

 

 

 

 

I did set up my backup strategy in the last couple of days, and had the same thoughts that you're having.

As a rough strategy I went with your second option, but with some specifics:

Container configurations

Wherever possible, I try to use the backup functionality build into the application running in the container.

For paperless this is a scheduled user script running:

docker exec -i paperless-ngx document_exporter ../export --use-filename-format --delete

this exports all the documents including the tags and the metadata to a share which is mounted as /export into the container. See the documentation for more details.

For other tools I'm mostly using the webGUI provided backup schedules, backing up to a backups share.

This step is not strictly necessary, because I also backup the appdata share. However, especially for paperless, I do want all the documents in a readable format in my backup.

Appdata backups

As not all tools in the containers provide a native backup functionality I additionally backup the appdata share using yet another user script, which stops all containers, rsyncs /mnt/user/appdata to a backups share and afterward restarts the containers.

I didn't want to use the Appdata Backup/Restore v3 plugin, as it didn't provide me with all the functionality I required, but you may as well just use it.

The important part is, that you should stop your containers, before backing up, and in case of any issues send a notification to unRAID which then can send you an email or any other notification (the plugin does all of that for you).

Flash backups

I'm using this script:

If you're ok with the data from the flash drive being stored unencrypted on unRAID's servers, you can also use the MyServers plugin.

Share backups

I backup all my "backup-worthy" shares (including the previously mentioned "backups" share), using restic to Backblaze B2. Restic comes as a single binary which I just download using wget, when needed.

From your list, and if you prefer a GUI I tested Duplicacy, and it worked quite well.

 

On 2/1/2023 at 8:51 AM, stwx said:

To keep the backup system (i.e. the hard drives) physically protected and easily accessible, I thought of buying an external case with 2 or 4 bays. That way, all I have to do is take the case, power it up, conenct it to my server and start the backup.

For me, the most important thing was that the whole backup process is automated and scheduled, otherwise I cannot be sure that a backup is up-to-date when I really need it.

In your case you'd need to open up e.g. Duplicacy and start the backups manually after connecting your external drive.

I like duplicati a lot, I use it for unraid and my mail server. The only thing I don't like about it is you can only have one destination for each backup set, so if you want data to end up in 2 offsite locations you have to roll your own. It has a "run-script-after" feature, which I use to rclone many of my backup sets from their primary location in s3 to backblaze b2. It works well but I'd be keen to evaluate something that simply allows multiple destinations.