Request: Definitive Backup & Restore Tools and Guide


Recommended Posts

Unraid is a great tool, making very complex system administration functions seem like playtime. This is wonderful, and every release keeps adding great new features. One thing that I think is sorely missing is an official backup & restore tools and guide. I know there are many forum posts and guides and plugins, and even a flash backup tool built in. None of this makes it easy or clear on how to properly backup ALL the configuration and data in a simple and safe way. Obviously, the shares are critical, and backup of this data should be handled by standard backup tools. What I am talking about is a way to export/import/migrate:

 

  • Configuration Settings
  • Installed Plugins and their configuration
  • Docker settings
  • VM settings
  • Users and Share settings per user

 

I think it would be useful and provide peace of mind if there was an official tool (or at least an official updated guide on how) to backup and reproduce an unraid system upon failure or desire to migrate hardware. Unraid is a digital hoarders paradise. But how to we make a copy of paradise! :)

 

Clearly, there is a need for this, something that covers everything, OFFICIALLY. Here is just a sampling of people trying to figure out how to do this with a hodgepodge of solutions:

 

https://forums.unraid.net/topic/72090-backing-up-the-unraid-server/

https://forums.unraid.net/topic/86303-vm-backup-plugin/

https://forums.unraid.net/topic/61211-plugin-ca-appdata-backup-restore-v2/

https://forums.unraid.net/topic/81233-unraid-671-easy-way-to-backup-and-restore-specific-dockers

https://forums.unraid.net/topic/102669-unraid-backup-strategy

https://forums.unraid.net/topic/81740-a-reliable-backup-solution-with-gui-ca-backup-restore-or-duplicati/

https://forums.unraid.net/topic/79539-best-way-to-backup-unraid-config

https://forums.unraid.net/topic/100202-latest-super-easy-method-for-automated-flash-zip-backup

https://forums.unraid.net/topic/68970-kvm-backups-snapshots

https://www.reddit.com/r/unRAID/comments/dcuxqc/community_backup_methods/

 

There are probably 100+ threads like this all over the internet.

 

 

Edited by ethanpil
Add more info
  • Like 3
  • Thanks 2
Link to comment
30 minutes ago, itimpi said:

Other than the VM settings all the other information is contained in the config folder on the flash drive and this is part of the zip file created by the built-in unRaid flash backup tool.

 

This proves my point. Why can't we have a simple built in solution that covers "everything" without exception? How do we backup and restore VM settings and why does this need to be a separate process?

Link to comment
9 hours ago, ethanpil said:

 

This proves my point. Why can't we have a simple built in solution that covers "everything" without exception? How do we backup and restore VM settings and why does this need to be a separate process?


Not disagreeing - just pointing that most of what you mentioned is already covered in the current flash backup capability.

 

I do not see any reason why at least the .xml files for VM definitions should not also be included in that zip file as they would be very small.   Not sure if there is anything else VM configuration related that should be included?   Obviously the vdisk files and iso images would not be part of such a backup.

Link to comment
  • 2 weeks later...
  • 2 weeks later...

Hi there

Totally understand your request and why you would want a fully documented backup solution. There are plenty of different tools out there to utilize for backups. There have been a number of different guides written from different individuals on how to do this.  We even wrote a blog post recently detailing one (https://unraid.net/blog/unraid-server-to-server-backups-with-rsync-and-wireguard).

 

The reality is that backup is a business in and of itself. There are plethora of different providers out there that create various software solutions that work with various platforms such as Unraid. If we ever do create a first-party solution for backups for unraid we will obviously document it but until then we do rely on the community and other backup software solution providers to provide guidance.  That being said, we may end up creating a guide under our wiki in the future, but no ETA on when that will happen just yet.

  • Like 2
Link to comment

Thanks for the "official" answer. I understand that "backup is a business in and of itself" and certainly a fully integrated backup/restore solution is a complex feature to add and might take some time. In my humble opinion, keeping our data and configuration safe this should be a priority for a company whose core product is a data storage and archiving tool. Unraid is an excellent, mature tool that has an amazing combination power and flexibility. This is the last weakness that needs to be addressed.

 

You may have missed my core point. Its easy to backup array data, I think everyone gets that. The real impetus here is that we should be informed/able to completely backup and restore all of our configurations for Unraid INCLUDING VM and Docker settings and data even if it has to be done manually until the official tool is developed. Otherwise our work and efforts setting up and configuring Unraid can go to waste. Even the blog post you mentioned only covers backup of array data. There is a glaring omission everywhere in regards to docker and VMs.

 

Just look at my original post, search reddit, etc. This is a common question without any official or reliable solution. I don't think anyone needs another tutorial on backing up array data, its the other proprietary Limetech magic that we need to be able to backup/restore. The USB image backup is 90% of the way there.

 

Thanks for listening. So, how can I backup and restore my Dockers and VMs? :)

  • Like 1
Link to comment
  • 4 weeks later...
On 3/22/2021 at 8:22 PM, ethanpil said:

Thanks for the "official" answer. I understand that "backup is a business in and of itself" and certainly a fully integrated backup/restore solution is a complex feature to add and might take some time. In my humble opinion, keeping our data and configuration safe this should be a priority for a company whose core product is a data storage and archiving tool. Unraid is an excellent, mature tool that has an amazing combination power and flexibility. This is the last weakness that needs to be addressed.

 

You may have missed my core point. Its easy to backup array data, I think everyone gets that. The real impetus here is that we should be informed/able to completely backup and restore all of our configurations for Unraid INCLUDING VM and Docker settings and data even if it has to be done manually until the official tool is developed. Otherwise our work and efforts setting up and configuring Unraid can go to waste. Even the blog post you mentioned only covers backup of array data. There is a glaring omission everywhere in regards to docker and VMs.

 

Just look at my original post, search reddit, etc. This is a common question without any official or reliable solution. I don't think anyone needs another tutorial on backing up array data, its the other proprietary Limetech magic that we need to be able to backup/restore. The USB image backup is 90% of the way there.

 

Thanks for listening. So, how can I backup and restore my Dockers and VMs? :)

 

Forgive me if I am missing something, but can you not backup and restore your dockers using the plugin:

CA Backup / Restore Appdata

 

You can choose it to automatically backup your appdata folder?

 

I am not sure about VM's yet I am yet to run one (only recently set my server up 1-2 days ago) but will run one soon and I agree with your points on being able to backup all the configurations to be able to easy restore them if something happened but I thought this plugin backed up your libvert folder as well, is that not enough to restore from there?

 

Since it backs it up to the array and you have parity on your array, aren't you all good in terms of backing up? I think I might be missing something here though

  • Like 1
Link to comment
13 hours ago, Arby said:

Since it backs it up to the array and you have parity on your array, aren't you all good in terms of backing up?

Parity is realtime, so any changes made are committed immediately. It protects against device failure, but not accidental deletion or corruption. So, no, parity is not backup. In the context of this thread, where files are being copied to a separate device on a schedule with the option of keeping aged copies, that is a backup, but adding parity doesn't make it a backup.

 

Parity is not a backup. A versioned copy of your files kept on a different device, that is a backup. The array is a good place to keep a backup, but it's not a backup of itself.

 

Maybe you already understand all that, but your wording concerned me that you may not have.

 

Backing up VM's is a little trickier, at the moment I recommend treating their backup just like you would a standalone machine on the network, utilizing tools inside the VM. The tricky bit is concerning open files, if you are ok with shutting down the VM to do the backup it's easy to do with tools in Unraid, if you need to keep the VM running 24/7 then it's easier to use something like acronis inside the VM. This is very general advice, there are so many different ways to set up and configure VM's it's difficult to cover all the scenarios. If you are using a vdisk image file for your VM, then backing up the XML and the vdisk file while the VM is shut down is all that is needed.

Link to comment

There's a pretty big reason that there can't (well... "won't" would probably be more accurate) be a single all-inclusive backup of everything in unraid - different applications/files have differing requirements for ensuring file consistency. A good starting point for your research would be looking for information regarding application consistent vs crash consistent snapshots/backups.

 

In short, a crash consistent backup will just ensure that the filesystem of the OS itself is intact. An application consistent backup is one from which you can restore the application's specific data in a manner it can utilize, which means that your backup has to take into account that application's specific requirements. Each application's consistency requirements are unique to that application. The only way to get around it easily would be to, at the start of the backup, ensure all applications, docker containers, and VMs are shut down, and remain shut down until the backup completes.

 

As one example:

Lets take a relational database, as these are fairly commonly used on unraid; at a basic level, this type of database has two components, the tables (db file), and the transaction log, both of which are required in order to use the db, and are interdependent upon eachother... If you were to take a snapshot of the filesystem that the db resides on without first ensuring that the data between the db and transaction log is consistent on the disk (meaning no pending actions in memory, which there pretty much always are unless you have either a very low resource use case, or have stopped the application entirely), you'll end up losing data. Usually application downtime isn't an option, so backup tools instead use other mechanisms to quiesce IO to the DB to first create a snapshot, then back it up.

 

While the filesystem itself would be consistent if you were to use just any backup tool, the application may well not as the application depends on the OS for it's own consistency, and that backup doesn't include anything in memory, and as far as the OS is concerned, 'the stuff that's on the disk is good'.

 

The level of complexity involved in backing up everything in a virtualized environment is pretty significant... That's why companies like cohesity, rubrik, veeam, etc charge such big bucks, because they have to integrate with a million different things (VSS for microsoft stuff, inc server OS, SQL, exchange, vix api to backup vmware virtual machines, various oracle components in order to understand RMAN, nutanix API to backup AHV snapshots, and so on) to ensure that each one is backed up in a way that, when you go to restore, you get what you expected. 

Link to comment

Unraid is obviously a tool we rely on for data storage and integrity. Parity and regular array data backups are also necessary and self-explanatory on how to perform. Even the flash backup is basically perfect.

 

The issue is all of the other stuff we spend hours setting up, debugging, tweaking, etc. Particularly, Dockers and VMss. There is no official way to backup and restore all of this configuration. We rely on third party plugins. That is what I think is unacceptable. 

 

There should be a path to backup and restore ALL data in my unraid box. Even if there isn't an official tool, at a distant second best, there should be at minimum, official documentation for every release on how to do this. So I can make sure I have ALL of my data and configurations safe from disaster, with a path to recovery. Honestly, I am shocked that the community wouldn't demand this, and is happy to rely on unofficial third party tools which can become instantly unsupported or abandoned...

Link to comment

We understand it... it just isn't something that's reasonable to ask of a hypervisor - I tried to explain the reason for this in the post above, that many applications require their own method of backing up in order to ensure they can be restored in a consistent manner. If you look at any other hypervisor, you'll see they operate similarly. Not because it's just never been asked of them, but because what's being asked of them is unreasonable and unrealistic.

Link to comment

Maybe you misunderstand. It doesn't need to be a live backup, but it does need to be automated.

 

If I need to shutdown the hypervisor or docker engine to make a reliable backup, by all means, we can do a shutdown, backup and restart. Very reasonable. I dont need VMWare enterprise level hot backups for Unraid. But even the free VM tools all allow import/export of VMs...

Link to comment

You can do that with as little as a copy/paste... I guess maybe that's why I'm not following.

 

In any case, I have enough data that id have to take the system down for multiple days if I were to copy everything out as a backup at this point, even over a 10Gb network. I'd rather use tools that were designed to back up explicitly what I need backed up, and do so without taking down applications I have people relying on for services I host on unraid.

Link to comment

Where do you learn to do this backup "as little as a copy/paste"

 

Did you see it in official documentation? As I said above AT MINIMUM i'm requesting/stating that Limetech needs to add a page to their wiki that is tested and maintained with each release so we have a (semi?) guaranteed disaster recover path.. Example below.

 

Better yet, some OFFICIALLY SUPPORTED tools to make these things easier. We already have some of them officially (flash tool) and some unofficially (CA Appdata Backup, etc)

 

"How to Backup and Restore Your Unraid Box"

 

1. Flash Drive

 Backup - Use the backup tool

Restore - ...

 

2. Array Data

Backup - make a copy of everything in /mnt/user

Restore ...

 

3. Plugins and Configuration

Included in flash backup

 

3. Dockers

...

 

4. VMs

..

 

Edited by ethanpil
  • Like 1
Link to comment
14 hours ago, ethanpil said:

"How to Backup and Restore Your Unraid Box"

 

1. Flash Drive

 Backup - Use the backup tool

Restore - Use the USB Creator tool

 

2. Array Data

Backup - make a copy of everything in /mnt/user

Restore - copy back to /mnt/user

 

3. Plugins and Configuration

Included in flash backup

 

3. Dockers

...

 

4. VMs

..

 

 

I like this topic. I've edited your post a little to include some things. 

 

I for one would like to know where VM XML lives for example, and if the only things needed to restore VMs are the vDisks and XML.

Edited by -Daedalus
Link to comment
  • 2 weeks later...

I also struggle with how to properly backup a VM.  The CA Backup/Restore AppData plugin is great for Dockers, would love to see something similar for VMs or as asked by Newbie - a clear guide on what files/folders to backup in order to do a proper config/settings etc backup including instructions on how to restore if needed.  

 

I recently had some errors on my cache drive and I am troubleshooting, and have backed up my appdata, but would like to be able to ensure my Home Assistant VM is recoverable as well.

 

Really appreciate everyone's contributions by the way - love Unraid!!

Link to comment
35 minutes ago, Jonwork88 said:

I also struggle with how to properly backup a VM.

I suggest a belt and suspenders approach.

 

For the day to day automatic backup, use software inside the VM appropriate to the OS, for example a windows VM could use acronis backup or whatever else you would normally use for a desktop with that OS.

 

For disaster recovery, periodically shut down the VM, make a copy of the VM's raw XML, you can just copy and paste to a text editor and save it, then make a copy of the vdisk.img file and keep it someplace safe, preferably other than on the same server.

 

That way, for typical restore you just roll back using the OS based software, but if the VM won't even start, you can create a new VM with the vdisk file and XML, then apply the latest restore from the OS.

  • Like 1
Link to comment

Xml is in libvirt.img, you can browse the contents with your preferred archival/compression tool to back them up individually (handy as it enables you to script backing up the xml specific to a given VM at the same time you're running your OS's native backup application), or en mass by simply copying the img entire file out.

  • Like 1
Link to comment
  • 2 weeks later...

Any instructions/documentation to make things easier for the end user is always better, especially if that can be -inside- the software itself.  We're covering unRaid for a NAS-building episode, and how to back up an array is actually pretty daunting even with the Rsync setup- how about an easy backup to local array button? I'm no good with Linux at all, so simplifying backups/making local backups easy would be great for the end user.  Personally, we'll use some Win10 sync software over the network with a GUI, it won't operate at on-server speeds but it's a cinch to use.  Hopefully the unRaid team can look at this, and this functionality can be improved going forward!  :)

-Rick

  • Like 1
Link to comment
  • 1 month later...

Well, all of the above are very nice, unofficial suggestions and recommendations that may work. I'm simply asking for some OFFICIAL, definitive documentation (or better - tools) to ensure our reliance on Unraid for all personal data are safe and sound.

Link to comment
On 4/19/2021 at 5:14 PM, ethanpil said:

make a copy of everything in /mnt/user

Make a copy of it where? That is where the main differences will be between different user scenarios, and why a simple approach that works for everyone is not possible. There are already a lot of different ways to approach this with dockers, scripts, etc. already discussed on the forum.

Link to comment
5 hours ago, trurl said:

Make a copy of it where? That is where the main differences will be between different user scenarios, and why a simple approach that works for everyone is not possible. There are already a lot of different ways to approach this with dockers, scripts, etc. already discussed on the forum.

 

I don't agree. Unraid users are power users. We wouldn't be here otherwise. We run duplicity, duplicati, s3cmd, etc and all kinds of other configurations to backup array data and everything else we can rig up.

 

We are capable, but we need to have an official and updated DOCUMENTATION of what needs to be backed up, how to back it up and how to restore it. What is needed is the knowledge to be able to backup and restore (and test the system). Really, without official backup and restore Unraid becomes a toy.

 

I dint understand why there is resistance to this notion that we should be informed and prepared for the inevitable. Limetech claims on the homepage that Unraid allows users "to have ultimate control over their data" and thats nice, and Unraid is a great product.

 

Ultimately, however, if Limetech doesnt share and maintain the knowledge of how we can backup and restore our systems perfectly, or perhaps doesn't provide that, then we have no control at all. Documentation is a minimum, import/export tools/scripts will go the extra mile. Silence and third party support are cop outs.

  • Like 1
Link to comment

Flash drive is your configuration, including disk assignments, license, plugins, and docker templates. Flash is the main thing you need to backup to get your system going again. In addition to those docker templates, you need appdata to get your dockers going again as they were. I don't use VMs seriously enough to know the details for backing those up, but there is a plugin for that.

 

Everything else is your data, and your data (/mnt/user) is what I was referring to. Lots of different requirements for that for different users. On site (to what?), cloud (which?), all of the user shares or only some of it. Maybe most people don't backup all of their user shares because they don't have the capacity and cloud can be slow and expensive for the volume many of us have.

 

I back up the important user shares to externals for storage offsite with rsync script. I have a backup server just because I had the hardware leftover after upgrading my main server, and I power it up to get some of the less important user shares rsynced to it occasionally. And some user shares I just don't bother with.

 

Other users will have other requirements. A general purpose way of meeting the requirements of everyone probably isn't going to happen as part of core Unraid. There is already cloud backup for flash provided by limetech in the form of a plugin provided by limetech that uses limetech provided cloud storage.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.