Redundant backup

[steve1977]

I have a very large Unraid array, which is protected through parity.

I am considering to create a redundant backup for some folders, which are of particularly importance (family photos, etc.). A few questions:

 

1) What software best to use? I read about rsync, but seems very commandline heavy? Anything in docker or Windows?

 

2) Better to do the redundant backup within the array or have the other disk as unassigned drive outside the array?

 

3) Also thinking to backup VM image file, which are already on unassigned drive? Anything to consider?

 

4) I never understood why nobody ever speaks about backing up the Unraid OS / config. Why is this and something worth doing?

 

 

Thanks in advance!!!

[itimpi]

13 minutes ago, steve1977 said:

4) I never understood why nobody ever speaks about backing up the Unraid OS / config. Why is this and something worth doing?

Not sure why you think nobody ever talks about this!   It is a actually recommended that you do this any time you make a significant configuration change.  You can do this either by clicking on the flash drive on the Main tab or by using something like the CA backup plugin.

[DavidInDE]

On 3/28/2020 at 4:12 AM, steve1977 said:

I have a very large Unraid array, which is protected through parity.

I am considering to create a redundant backup for some folders, which are of particularly importance (family photos, etc.). A few questions:

 

1) What software best to use? I read about rsync, but seems very commandline heavy? Anything in docker or Windows?

 

2) Better to do the redundant backup within the array or have the other disk as unassigned drive outside the array?

 

3) Also thinking to backup VM image file, which are already on unassigned drive? Anything to consider?

 

4) I never understood why nobody ever speaks about backing up the Unraid OS / config. Why is this and something worth doing?

 

 

Thanks in advance!!!

The closest question I could find to my situation! Not sure if it helps, but I'm running UnRAID and thinking of building a FreeNAS box to backup to. I'm looking into software and will let you know my findings (currently looking at urbackup). 

[testdasi]

On 3/28/2020 at 8:12 AM, steve1977 said:

1) What software best to use? I read about rsync, but seems very commandline heavy? Anything in docker or Windows?

2) Better to do the redundant backup within the array or have the other disk as unassigned drive outside the array?

3) Also thinking to backup VM image file, which are already on unassigned drive? Anything to consider?

1. I used to use Syncthing and can recommend it. It is also available cross-platform.

You can set up a docker on the server as the main destination and install it on each device (including VM) as source.

 

2. There is no "better". It depends on what you find to be a reasonable level of protection. Think of your data in "Four-O"

• Original copy
• Online backup - having redundant backup on the array would be online; unassigned device that is an INTERNAL HDD would also be online. It won't protect you if, for example, there's a power surge.
• Offline backup - a device that is disconnected and turned off except when data is being transferred e.g. a USB external HDD. It won't protect you if, for example, there's a house fire.
• Offsite backup - can be the cloud or even an external HDD that is at a relative's home. It won't protect you if, for example, COVID-19 wipes out the human race.

 

3. I use bash script (i.e. command line) for this so I don't have any GUI-based stuff to recommend.

Edited March 30, 2020 by testdasi

[primeval_god]

4 hours ago, testdasi said:

1. I used to use Syncthing and can recommend it. It is also available cross-platform.

I would not recommend Syncthing as it is a synchronization tool not a backup tool. I do use it myself for synchronizing files across my devices and for that purpose it works great. It does have some versioning capabilities but they are fairly basic.

 

I would recommend Duplicati, which is cross-platform and I believe available in Docker. I use Duplicati on my Windows machines and back up to a Minio Docker container on my unRAID server (Minio presents an S3 compatible storage target).

[testdasi]

5 minutes ago, primeval_god said:

I would not recommend Syncthing as it is a synchronization tool not a backup tool. I do use it myself for synchronizing files across my devices and for that purpose it works great. It does have some versioning capabilities but they are fairly basic.

 

I would recommend Duplicati, which is cross-platform and I believe available in Docker. I use Duplicati on my Windows machines and back up to a Minio Docker container on my unRAID server (Minio presents an S3 compatible storage target).

I disagree then.

 

A one-way periodic sync is a backup. That's why people use rSYNC as a backup tool.

In fact, a synced copy is an excellent backup because restoring does not require any additional software. Drag-and-drop in Windows Explorer would work. cp -ar in Linux console would work.

 

In fact, I would definitely NOT recommend Duplicati because:

• It doesn't scale well at all. It worked originally when I only had a few GB to backup my documents. It crapped itself once things got into the hundred GBs range.
• It requires technical skills to restore. My granny certainly can't use Duplicati but she understands "drag-and-drop".
• A corrupt incremental volume makes subsequent incrementals untrustworthy to restore. You have to go back to the last full backup, which means you might as well have full backups all the time, which means you might as well have a synced copies.

Its other bells and whistles are not as important it seems:

• Compression is good to have but I would argue that most things that need to be backed up (e.g. photos) don't really compress well at all.
• Incremental is good to have but see my above point about the disadvantage of incremental.
• Encryption is good to have but only if you backup to the cloud (and only if you don't have any other way to encrypt e.g. rclone encrypted volume). For most people, I would argue that an encrypted backup adds an unnecessary layer of complexity.
• Deduplication is good to have but how much data is dup in a typical user's backup?

 

I guess can I can TL;DR my response as: when it comes to backup, "fairly basic" is a complement.

 

[primeval_god]

6 hours ago, testdasi said:

• A corrupt incremental volume makes subsequent incrementals untrustworthy to restore. You have to go back to the last full backup, which means you might as well have full backups all the time, which means you might as well have a synced copies.

• Deduplication is good to have but how much data is dup in a typical user's backup?

Duplicati 2.0 uses a block based backup scheme that allows keeping multiple incremental versions where each version is essentially a full backup but leverages duplication. Basically the same thing that Crashplan or Veeam uses (both of which i am also a fan of). 

 

6 hours ago, testdasi said:

• It doesn't scale well at all. It worked originally when I only had a few GB to backup my documents. It crapped itself once things got into the hundred GBs range.

My total backups are in the hundreds of gigs, though no single backup set is over 100G. At this scale at least i have never had any performance issues.

[testdasi]

8 hours ago, primeval_god said:

Duplicati 2.0 uses a block based backup scheme that allows keeping multiple incremental versions where each version is essentially a full backup but leverages duplication. Basically the same thing that Crashplan or Veeam uses (both of which i am also a fan of). 

 

My total backups are in the hundreds of gigs, though no single backup set is over 100G. At this scale at least i have never had any performance issues.

We should just agree to disagree then. At least the OP gets to see it from both sides to make a better decision. 😉

[DavidInDE]

Thanks everyone for your input - at the risk of starting things up again, would your views change based on the fact that the files I intend to back up are on a media server predominantly, and therefore change hardly at all? There are SOME document files, but really not that many. So I'm assuming a sync rather than an incremental backup would suffice?

 

Thanks again