June 21, 20251 yr Hi,my unraid server becomes more and more how I like it to be, thus I soon have to set up some backups. The data stored on my unraid are personal files like documents and stuff, large amount of photos and a few media files. On top I run a docker container with paperless, which contains a large amount of important documents, as well as a virtual machine with HomeAssistant. I don't have a lot of changes, most are coming from paperless, editing a bunch of photos once in a while and obviously bazillions of events and measurements within homeassistant. My curent thoughts on this are:24 hourly and 7 daily ZFS snapshots for file history (i.e. against personally induced f... ups)ZFS send 7 daily and 4 weekly snapshots to a small backup server in another part of my house (which is fireproof)rsync (or rclone?) a weekly backup to a cloud storageThere are still a few questions though:Does this make sense after all?How do I back up things like the paperless container and the HomeAssistant VM due to the built in databases?Which backups should I run as full backups, and which as incremental ones?How do you do it, what are your concepts and why? What tools are you using?regards,Klayman
June 21, 20251 yr Community Expert Review the 321 rule of thumb...https://www.techtarget.com/searchdatabackup/definition/3-2-1-Backup-Strategy#:~:text=The%20basic%20concept%20of%20the,data%20is%20sent%20off%20site.Most often do 221...Ideally Clustering with a similar Hardware, same disk configuration and essential you duplicate the current unraid machine. then you rsync content between them.Raid is not a backup it is redundancy and helps with maintaining uptime.I make a share called Backup then use this folder on unraid on disk for a staging area. Once I ahve all my backup targets I have this one folder to send to another machine... I also use zfs and use snapshots and in a sftp ring I send my zfs snapshots as a backup...https://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.htmlI have a cron script... ZFS utilizes snapshots and the zfs send command for efficient backups and replication. Snapshots capture a point-in-time view of a dataset, and zfs send creates a stream of the snapshot's data, which can be sent to another location (another pool, a file, or another system) using zfs recv. This allows for incremental backups, sending only the changes between snapshots, significantly reducing backup time and storage space. ....I also implement and utilzes a few CA Apps and plugins.Such as Appdata backup, zfs backup buddy, unraid replication. rclone...There is alot of this talk and examples of script and commands you can do to automate and backupexample:https://forums.unraid.net/topic/126028-simple-automated-offline-backup-script/https://unraid.net/blog/unraid-server-to-server-backups-with-rsync-and-wireguardandhttps://forums.unraid.net/topic/97958-rsync-incremental-backup/do yo have any specific question?a backup is only good on how you handle your recovery plan and known it can be restored from the files you have backed upThe main tool I use that does just about everything to backup locally is the appdata backup plugin.Unraid flash dirve, dockers, vm templates...https://forums.unraid.net/topic/137710-plugin-appdatabackup/some side Info That I have found over the years:VMsWith VM snapshot this has gotten tricky...https://forums.unraid.net/topic/169220-vm-snapshots/Unraid libvirt image needs backup ed.Snapshot Database needs backed up. Usually separate boots to clonezilla is needed to image a the entire vdsik and a copy of the vms xml code is copied to be resued for a full restore of a vdisk image and configurations...Plugins exsit for vm backup...I use app data plugin to grab the VM metdata and xml configs...vm backup plugin is ok you really have to watch the use of vm snapshots and bring vms to 1 vdisk file...https://www.youtube.com/watch?v=ntjQphOSPPIhttps://www.youtube.com/watch?v=rDGSS1iEtq4https://forums.unraid.net/topic/86303-vm-backup-plugin/Dockersapp data backup plugin.sometimes the docker vdisk needs backuped. (Only if docker paths and data is misconfigured. thus, docker data is saved within the docker image and not a volume set to a path to store on unraid...snapshots of the appdata folder with zfs I used to send and backup... zfs snapshots are automated 1 a month and i trim them to have 12 so 1 year worth of backups.with appdata I can configure what i'm backingup on the docker and have it tar the docker image contents and grab some volumes for it. *-Not ideal for things like plex/jellyfin/embry etc... as it will try to backup the entire movie library and may need additional configurationLXCs. Ich777 has a settings area and each lxc you can runa backup but this is more like snapshoting...-Orgianly was using the backup snapshot esk feature to share lxc before learngin about how he is trying to implement and use lxc tempaltes...LXC are simlar to docker and VMs. as you run a VM OS but setup a docker service. more for dev and needing more granual control that a docker can bring.General file backup?copy paste... unasnged disk cont to remote devcie.rsync / cp (copy command) / zfs send .... etc etc...I'm partial to runign ssh and using ssh as a sftp server.I then run fiel zilla (docker or cleint) and conect to my machein and use samba to the remote and file zilla tranfer files.This is becaue I can leverage samba to write as nobody and surs group for file access wiht the samb connetion but that a bit advace configuration...-This also give me a change to scan a new file being written to storage for virus / malware checks.. aslo a bit advance setup...other use unasigned disk and conec a usb passport device and use udev rules and backup scripts when the device is conected...so with that siad lets look at some Q/A:24 hourly and 7 daily ZFS snapshots for file history (i.e. against personally induced f... ups)ZFS send 7 daily and 4 weekly snapshots to a small backup server in another part of my house (which is fireproof)rsync (or rclone?) a weekly backup to a cloud storage24 hours backups don't make sense unless you have space to burn. weekly and monthly... are the way to go... if you need more intermediant i would recomend one every 3 days...You would befit fomr a snapshot creation and purne setup ... keep x days in the past...for zfs some more info found here:using the zfs send feature Again unless you have space to burn... weekly and or monthly...some more info found here with a zfs send and prune by sending the zfs snapshots:https://forums.unraid.net/topic/191128-should-i-be-worried/#findComment-1561313rclone is running rsync...Rsync and rclone are both powerful tools for synchronizing files and directories, but they cater to different needs. Rsync is traditionally used for local and remote file synchronization, particularly over SSH, and excels at transferring only the changed portions of files (delta transfer). Rclone, on the other hand, is primarily designed for cloud storage synchronization and supports a wide range of cloud providers, but it generally transfers entire files rather than delta updates. Key Differences: Cloud Storage Support:Rclone shines when working with cloud storage services like Google Drive, Dropbox, and S3, offering a vast array of supported providers. Rsync primarily focuses on local and SSH-based remote transfers. Transfer Method:Rsync uses delta transfers to minimize bandwidth usage by only transferring changed parts of files, making it efficient for large files with small changes. Rclone, in contrast, typically transfers entire files. However, some rclone commands can do a file by file transfer of differences. Synchronization vs. Copying:Rsync excels at bidirectional synchronization, ensuring both source and destination are identical, while rclone is often used for unidirectional backups or simple copies to the cloud. Portability and Shells:Rsync's reliance on SSH can limit its portability, while rclone's support for various cloud services makes it more versatile for accessing data from different locations. Rsync can also be used with different shells, while rclone is primarily command line based. Multi-threading:Rsync is generally single-threaded, while rclone can utilize multiple threads for concurrent transfers. Resumable Transfers:Rsync can resume interrupted transfers, which is useful for large files over unreliable connections. Rclone, in its default configuration, may not resume partial downloads. However, some rclone commands can be configured to resume transfers. Use Cases:Rsync:.Opens in new tabIdeal for backing up and mirroring data between local and remote servers, especially when dealing with large files and bandwidth constraints. Rclone:.Opens in new tabBest suited for synchronizing data with cloud storage services, creating backups to the cloud, and transferring data between different cloud providers. Regarding the on papaer your bakup consious which is good but over zelous on what your backup ing...time frams space, time all matter here how fast are how long are you willing to wait... example a movie is 1 GB these days if not more if 10 of file is compresed and sent that 10 GB ish being copoed and transferd mutiple times That can be days worth of tryign to copy and not have a backup as tranfer can fail backups sytems fial script fail... thus 3 day min du to copy start, time to make the tranfer, and confirm check...I went monthly and have stuf to mainting space by pruning...as mentinoed earlier vms are a bit tricky... they usualy requrie a siamlr methd of ghost/imageing the vdisk. If you used snapshots on the VM it can be tricker to recover... Sicne the database is on the VM you would backup the enteir vm vdisk. and restroe by rempalte teh entire vdisk... snapshot break here... as a snapshot restor doent' clear the database and moving to a earlier sanpshot erase to incremental...DON'T USE SNAPSHOTS FOR PRODUCTION VMs! to use plugins script and utites for backing up a VM...the full vs incremental...based on your time frams of stuff that woul be a full at first of month incremental till end of month... (that alot of data)...I would recomend a monthly full backup for yarly archival and backup recover...but full vs incremental has caveot...Full backups create a complete copy of all your data, while incremental backups only copy the changes made since the last backup (full or incremental). Full backups are comprehensive but time-consuming and resource-intensive, while incremental backups are faster and use less storage but require restoring multiple backup sets for recoveryFull Backup:What it is:Creates a complete copy of all selected data, essentially a snapshot of your system at a specific point in time. Pros:Fastest recovery time as only one backup set needs to be restored. Simplest to manage. Cons:Most storage space required, slowest backup speed, and can be time-consuming. Incremental Backup:What it is:Backs up only the data that has changed since the last backup (full or incremental). Pros:Faster backup times and smaller storage requirements compared to full backups. Cons:Slower recovery times as you need to restore the full backup and all subsequent incremental backups to get a complete copy. In essence:Full backups are like taking a complete photograph of your data, while incremental backups are like taking pictures of only the parts of the scene that have changed. Think of it like this: restoring from a full backup is like having one perfect image; restoring from an incremental backup is like needing to assemble a puzzle using multiple images. Choosing between them depends on your priorities: if you need the fastest recovery, full backups are better. If you need to minimize backup time and storage, incremental backups are a good choicee Edited June 21, 20251 yr by bmartino1
June 29, 20251 yr Author Thank you for your extensive answer. I have now implemented automated ZFS Snapshots with the "ZFS Buddy Backup" plugin. So far I do only use it for snapshots as I don't have another ZFS system available yet to also send snapshots to. The snapshots serve as a first safety net for my own mistakes... For Appdata and VM metadata I use the plugin "appdata backup" and back it up to my pool. It automatically stops and starts the containers to have a consistent backup. Eventually the important parts of the pool (i.e. not movies and such), including the appdata backup folder are backed up daily with backrest to a Hetzner S3 compatible storage with 30 daily and 12 monthly snapshots. Also 10% of the data on the S3 storage are checked for integrity every sunday night.For now I am happy with it, except backing up the virtual machines. But these run databases within the guest OS (e.g. HomeAssistant) and I am not sure how to back them up. Is there a way within unraid, or does it HAVE to be initiated from the VM itself?Thanks,Klayman
June 29, 20251 yr Community Expert Good question. I left Unraid VM for this issue and went to proxmox and proxmox backup server.@SimonF thoughts? vm backup solutions for Unraid.There are ways I'm not familiar with them any more.
June 30, 20251 yr Community Expert Only backups for VMs is via the VM Backup plugin. Appdata backup does backup the libvirt parts. But you need to be saving the domains folder also.Plans for integrated backups are being looked at. VM Backups will be added to this but not sure if for inital release. I have been testing processes to create backups, but with options if snapshots exist but only as a PoC at present.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.