Recommendation for Docker Backup Utility

[mikesp18]

I hope I've placed this question in the correct forum.  Please let me know if there was a more appropriate place.

 

Use case:

Intermitted Full Backup of 650Gb of Documents and Picture (probably either weekly or monthly) 

Incremental Backups between Fulls Backups (Nightly)

Space isn't an issue

Backup locations: 1) Local on the same Unraid installation, 2) Local onto a second separate Unraid Installation, 3) Remote offsite backup

On site backups will not need encryption. Offsite backups will need encryption

I'd like to be able to easily access the backup fills from onsite locations without using restore functions that would need to recreate the entire database.

 

Free is nice, but I'm willing to pay if necessary.

 

I have tried Duplicati repeatedly, and while I like the interface, it crashes multiple Unraid installations and I've never been able to iron out the kinks. This may be me personally messing it up, but I've stopped messing with it after 2 years of repeated crashes. I recently tried Duplicacy, and it's OK, but I'm not in love with it.  I see a number of different Backup Apps under the Backup section in COmmunity Applications, but before I start installing all of them and trying them, I was looking for personal recommendations. 

 

 

[ppaul]

Hi,

actually I too am searching for a good solution to manage backups and also archive a part of my data.

 

I tried a lot of different apps and wasn't that convinced by something yet. So I'm interested in suggestions too.

On my machines I use Vorta/Borg and maybe I'll give it a try directly on unraid too.

 

The question that came up for me is: How to archive stuff for long term storage and keep track of it?

So I would like to burn all my Pictures and Homevideos for example to M-Discs directly from within unraid.

 

I really am astonished that there is no solution out there that handles all this stuff in one place.

I mean organize and check the sync to another Unraid/NAS/External Drive and let you chosse stuff for archiving too.

 

Maybe someone has a suggestion to handle this in a way that keeps track of the archived stuff too.

 

Sorry if I hijacked your thread but maybe you would like this archiving feature too.

 

[TimTheSettler]

3 hours ago, ppaul said:

On my machines I use Vorta/Borg and maybe I'll give it a try directly on unraid too.

 

I researched this myself and in the end I chose Vorta (Borg) because of a recommendation from a friend who is a Linux administrator.  He uses it in a production setting so how can you beat that?  Anyway, Borg is free and it's pretty powerful but it's a command line utility.  I prefer a GUI interface which is where Vorta comes in.  Vorta is a docker app you can download in unRAID and it's a GUI interface to Borg.  Although Vorta is a bit kludgy it does the trick.  You can create one or more archives and each archive is encrypted.  One other feature I like is the ability to mount an archive point.  So if you need a file from a week ago then you mount that archive date and pull the file out.

 

The decision for you is, do you create one archive or many?  Each archive benefits from deduplication but the more you put into an archive means that it's harder to manage that archive.  I have multiple archives so that I can control the schedule and docker.  Each archive has it's own docker app because I found that trying to create multiple archives in a single Vorta docker didn't work as I expected.  Within each archive I keep many archive points (dates/snapshots) because there's no reason not to with deduplication.

 

3 hours ago, ppaul said:

I mean organize and check the sync to another Unraid/NAS/External Drive and let you chosse stuff for archiving too.

 

Can you clarify what you mean here?  In Vorta you can see the progress and check the archive.

 

19 hours ago, mikesp18 said:

Intermitted Full Backup of 650Gb of Documents and Picture (probably either weekly or monthly) 

 

In your case you just need one archive.

[ppaul]

Like I said Vorta and Borg are also my preferred solutions.

 

3 hours ago, TimTheSettler said:

7 hours ago, ppaul said:

I mean organize and check the sync to another Unraid/NAS/External Drive and let you chosse stuff for archiving too.

 

Can you clarify what you mean here?  In Vorta you can see the progress and check the archive.

 

I mean besides backing up with vorta I want to archive long term. I mean most of the families pictures and stuff like that. Data like that doesn't change anymore and I like to burn it on M-Disks and keep track what I have where and when I last archived. It would be great to do that and have an solution to segment to 25/50 or 100GB media and have a database that has information about what is where. So that I can verify old archives and maybe rearchive stuff from time to time.

 

A real strategy to keep data for a lifetime or longer and in a way that is transparent to others in the family. I mean why backup/archive in the first place. redundancy an normal backups sure for day to day stuff. But most of the data I have is only interesting in the long run. All HDD solutions are prone to bitrot so thats not good. Amazons Glacier would be good but having everything in two places I own is more to my liking.

 

[TimTheSettler]

On 3/9/2023 at 1:05 PM, ppaul said:

I like to burn it on M-Disks and keep track what I have where and when I last archived.

 

There was a time when I saw these used by companies (as replacements to tape backups) but no one does that anymore.  It was always expensive technology and is still so.  However, I loved the idea that seldom used stuff (based on modify date or access date) would be archived off but can be brought back automatically without human interaction.  The OS or software would know where that file is.  I think this is what you're referring to but as I mentioned, almost no one uses it and it's expensive.

 

I use a CD suitcase and burn things like pictures to DVDs (or now Blu-Ray) from time to time.  It's a manual process but pretty easy.  Keep an Excel file with the disc number, it's contents, and date it was burned.  If the disc gets old (10 years?) then burn a new copy of the files that were on it and update the speadsheet.

 

[ppaul]

 

On 3/10/2023 at 8:51 PM, TimTheSettler said:

I use a CD suitcase and burn things like pictures to DVDs (or now Blu-Ray) from time to time.  It's a manual process but pretty easy.  Keep an Excel file with the disc number, it's contents, and date it was burned.  If the disc gets old (10 years?) then burn a new copy of the files that were on it and update the speadsheet.

sure thats what I'll do but I was wondering if there isn't something out there better than Excel.

 

Maybe I should write the solution to my problem 🙂

[Michael_P]

rsync - it won't encrypt the data for you but you can send to an encrypted volume

[TimTheSettler]

On 3/14/2023 at 4:56 AM, ppaul said:

Maybe I should write the solution to my problem 🙂

 

The ultimate solution would be an app where you specify a folder and tell it to create a disc archive.  It then reads the directory structure into memory and splits up the files so that they fit optimally on a bunch of discs.  You then insert the discs and it burns them and records a serial number which can be used to link back to the disc's contents.  Since this process is a manual one you would need to run the archival process manually.  Each time it runs it collects the directory structure and compares it to the last one and creates new discs for changed and new files.  Similar to a backup app but it would need to know which disc the proper version of the file is on.  It would also need a feature where a new disc can be created which can replace an existing but lost or destroyed disc (same content as previous disc but with new serial number recorded in the app).

 

A simpler version is that you use Brasero to burn a disc and then separately have a script that reads the disc, records the files on it, links them to an ID, and puts all the data into a database (or Excel).

 

Let me know when you have either of these built.