Moving from CrashPlan to Duplicati - Requesting Guidance


jdag

15 posts in this topic Last Reply

Recommended Posts

I need to come up with an alternative to CrashPlan ASAP as my contract expires later this month.  I am not going to continue with their business plan.  I've decided to use Duplicati and I believe that I've narrowed my options to 2 alternatives, and I was hoping from input from those here.
 
In terms of background: I have 3 Mac laptops and 1 unRAID server.
 
Virtually all of the laptop data is recoverable in the event of an issue.  Each laptop is backed up locally to a Time Capsule, and even if those backups are compromised, nearly all data/files are kept within cloud-based email and/or Dropbox (yes, I understand Dropbox is a sync service and not backup).  I am only saying that there are multiple layers/copies of files already, so I am not overly concerned about the laptops.
 
On the other hand, I am very concerned about getting off-site storage for my unRAID server.  In total, the unRAID server holds ~8TB of videos/music and ~1TB of photos.  The videos/music are copied off to external hard drives that I keep at my parent's house (and of course don't often change).  The photos are my #1 priority, and I add to and edit those photos very often, so a constant backup solution is important.
 
My 2 alternatives are:
 
1) Backup each computer to unRAID, then unRAID to cloud (I would start with the folders that hold the laptop backups as well as the folders holding photos.  I may eventually add the video/music folders to the backup, but again not urgent).
 
2) Backup each laptop plus server individually to cloud (similarly, not the videos/music to start).
 
The reason #1 is more attractive to me is that I then have a "central connection" to the cloud storage.  I would not have to worry about multiple cloud accounts, checking each laptop to assure connectivity, etc.
 
Thoughts or suggestions would be very appreciated!
 
Thanks in advance, John
Link to post

I went through exactly the same thought process as you when I first read about CrashPlan halting their residential service offering, and in the end wound up selecting option #1. I chose not to backup any of the videos/movies/etc I have on my unRAID server, since I can easily re-rip or re-download them as necessary if they were to be lost. But I do have several terabytes of photographs on my array, as well as a significant music collection, both of which would be practically irreplaceable. So I installed Duplicati on each of my family's 4 PC's which back up to a "PC-Backups" share on unRAID via FTP (making use of  SlrG's ProFTPd plugin). Then I created two jobs on the unRAID Duplicati docker, one for PC backups and one for array backups, each with it's own separate bucket in Backblaze B2, and on its own schedule; PC's every other day, unRAID shares on a weekly basis.

 

My main justification for backing the PCs up to unRAID instead of directly offsite is the significantly improved time-to-recover. I figured it was unlikely for both a PC and unRAID to fail simultaneously, so being able to restore data over a local 1Gbps LAN connection will be much faster than trying to restore a PC over the internet. In the event of something more catastrophic (like a house fire) where the unRAID server is lost too then I still have my offsite backups, it'll just take longer to recover.

 

I've had it running this way for a couple months now and so far I'm pleased. I'm paying way less than I was with crashplan too. Since Backblaze B2 is priced purely on the basis of storage consumption, in my particular case it works out way better than when I was being charged per-PC. I think my bill last month for 4 PCs and several shares in unRAID itself was around $4.

 

YMMV, but it's working well for my needs.

 

-A

Link to post

Thanks a ton, Ambrotos.  It certainly sounds like you and I are on the same page.  And I agree that using the offsite sources to restore would only be needed in the most extreme situations, in which case I'd likely have a lot of other pressing issues on my mind!

 

I do have a question for you.  You said "So I installed Duplicati on each of my family's 4 PC's which back up to a "PC-Backups" share on unRAID via FTP (making use of  SlrG's ProFTPd plugin)".  Why did you do that as opposed to just backing up using Duplicati's default "Local Drive" option?

 

As for Backblaze B2, I hadn't looked into that option.  I was leaning to Amazon Cloud Drive or MS OneDrive.

 

 

Edited by jdag
Link to post

We must be having a miscommunication or something. :P

 

The PCs are using a share on unRAID as a backup destination. They connect to these shares via FTP (SMB isn't a supported option in Duplicati). The built-in unRAID FTP server isn't configurable enough for my liking, so I used the ProFTPd plugin on unRAID to allow better user account control, etc. I'm not sure how you could use "Local Drive" as a backup destination for the PCs, unless you were planning to map a network drive or something like that, but I've never had much success with persistent network mapping in Windows.

 

Cheers,

 

-A

Link to post

Interesting.  I am using a Mac, so maybe it is different.

 

Within Duplicati, I am able to select "Local folder or drive" and then select the volume and share in the "Folder Path" box.  I have been testing it, and it does work.  Although, maybe the FTP method is a better one?

Link to post

Ah, interesting. I don't have any Macs, so I couldn't tell you how well that'll work. Personally I try to use as few plugins as possible on my unRAID box, so if it works reliably as you have it now then I wouldn't mess with a good thing ;) I could be wrong, but I doubt you'll see a noticeable speed difference between the two protocols. Honestly, I would prefer it if Duplicali supported SMB, and it seems that you Mac users have it a bit easier in that regard than Windows users.

 

https://github.com/duplicati/duplicati/issues/1257

 

Cheers,

 

-A

Edited by Ambrotos
Link to post
  • 3 months later...

Ok, so here is what I am doing to replace Crashplan and I want to make sure Duplicati is the best docker for this.

 

Previously, each PC in the house we simply use Google Drive to save anything truly important.  In most cases, this is free anyway each family member.

On the server, I want to backup my important files so I was using a personal crashplan account to backup those files only.  I also backed up my boot USB.

 

We now also have a family office 365 account which gives each of us 1TB of storage so I am going to use Duplicati to backup to OneDrive since I already pay for that account.  My important stuff is only about 500GB total.

 

Anyone using Duplicati for backup to OneDrive?  Is there a better option?  Does Duplicati only backup changes?

Link to post
1 hour ago, rgeorge123 said:

Ok, so here is what I am doing to replace Crashplan and I want to make sure Duplicati is the best docker for this.

 

Previously, each PC in the house we simply use Google Drive to save anything truly important.  In most cases, this is free anyway each family member.

On the server, I want to backup my important files so I was using a personal crashplan account to backup those files only.  I also backed up my boot USB.

 

We now also have a family office 365 account which gives each of us 1TB of storage so I am going to use Duplicati to backup to OneDrive since I already pay for that account.  My important stuff is only about 500GB total.

 

Anyone using Duplicati for backup to OneDrive?  Is there a better option?  Does Duplicati only backup changes?

 

This is precisely what I am doing.

 

I recently subscribed to the Office 365 family plan (I needed to upgrade from Office 2011 for Mac anyhow).  I set up each family member (3 of us) with a MS account, and each person now has 1TB of OneDrive space (as well as the latest Office version).  I then installed Duplicati on each of our 3 computers, each backing up to a personal OneDrive location.

 

I then created a generic MS account for my unRAID server.  I set up the Duplicati docker, and used that generic login to use its OneDrive destination.  More-or-less, I am sending my photos from unRAID up to OneDrive.

 

I still have 1 more Office 365 account I can eventually use.

 

For $100/year, it is quite a good value.  In fact, I am saving $50/year over what I was spending with CrashPlan, albeit with a more complex setup.  But so far it is working very well.

Edited by jdag
Link to post
  • 1 month later...
On 11/6/2017 at 1:58 PM, Ambrotos said:

I went through exactly the same thought process as you when I first read about CrashPlan halting their residential service offering, and in the end wound up selecting option #1. I chose not to backup any of the videos/movies/etc I have on my unRAID server, since I can easily re-rip or re-download them as necessary if they were to be lost. But I do have several terabytes of photographs on my array, as well as a significant music collection, both of which would be practically irreplaceable. So I installed Duplicati on each of my family's 4 PC's which back up to a "PC-Backups" share on unRAID via FTP (making use of  SlrG's ProFTPd plugin). Then I created two jobs on the unRAID Duplicati docker, one for PC backups and one for array backups, each with it's own separate bucket in Backblaze B2, and on its own schedule; PC's every other day, unRAID shares on a weekly basis.

 

My main justification for backing the PCs up to unRAID instead of directly offsite is the significantly improved time-to-recover. I figured it was unlikely for both a PC and unRAID to fail simultaneously, so being able to restore data over a local 1Gbps LAN connection will be much faster than trying to restore a PC over the internet. In the event of something more catastrophic (like a house fire) where the unRAID server is lost too then I still have my offsite backups, it'll just take longer to recover.

 

I've had it running this way for a couple months now and so far I'm pleased. I'm paying way less than I was with crashplan too. Since Backblaze B2 is priced purely on the basis of storage consumption, in my particular case it works out way better than when I was being charged per-PC. I think my bill last month for 4 PCs and several shares in unRAID itself was around $4.

 

YMMV, but it's working well for my needs.

 

-A

 

Do you foresee/have any problems with Duplicati-ing (to B2) a bunch of Duplicati backups (from your PC's)? 

 

I assume that to recover your PC's in the event of say, a fire where the server and all computers are destroyed (or a flood, etc.), you'd have to rebuild the server first then restore the PC's from the newly re-minted server?

Link to post
  • 2 weeks later...
On 3/20/2018 at 4:19 PM, ksignorini said:

Do you foresee/have any problems with Duplicati-ing (to B2) a bunch of Duplicati backups (from your PC's)?

 

I don't have any problem with backing up a repository of duplicati backups to B2. They are after all just a bunch of zip files. ;)

 

In fact, while troubleshooting an unrelated backup completion problem I was having a few months ago I asked the very same question in passing on the Duplicati forums. Here, if you're curious.

 

Anyway, as long as you don't mind the lack of additional de-duplication, you're correct in that the only downside to this plan is that due to the nested nature of the backups you'd have to restore the entire outer backup even if you're only looking to recover a single file from the inner one.

 

-A

Link to post

Apologies to the OP, but I was hoping to bend @Ambrotos ear briefly.  Learning UnRaid by the day but pretty lost on how to set up an FTP server and then have a Windows 10 machine use Duplicati to back up to the UnRaid FTP server.  Any chance you could point me in direction that I can read up on how to accomplish this setup?

Link to post
19 hours ago, ChappyEight said:

Any chance you could point me in direction that I can read up on how to accomplish this setup?

 

No problem.

 

I'm assuming you're mostly asking about the unRAID-side of the equation, since from the Duplicati side of things it's really just setting the backup destination storage type to FTP and filling in the address ;)

 

unRAID has a built-in FTP server, but it's not configurable, so I turned that off and installed SlrG's ProFTPd plugin (just search the Apps tab in the unRAID UI for it to install). Check the ProFTPd page in unRAID"s Settings menu to make sure it's Enabled and "RUNNING". Instead of having its own separate user database, this plugin uses the built-in unRAID user profiles. Basically, any configured unRAID user with the description "ftpuser" will be added to the ProFTPd user database. If you also add a storage path to the user's description then that will be set as the FTP user's home directory. So I created a new unRAID user called "backup" with the description "ftpuser /mnt/users/Backup". Then just create the unRAID share called "Backup" (for backups I un-check use of the cache disk), and you're done. You can test that your FTP user has been created properly by using Chrome/Firefox to browse to ftp://<username>:<password>@<unraidIP> and see if the resulting page lists the contents of the folder you specified as the FTP user's home directory.

 

Hope that helps.

 

-A

Edited by Ambrotos
  • Like 1
  • Thanks 1
Link to post

Sincerest thanks @Ambrotos for your patience and for taking the time to write that out.  I'm going to try it first thing tonight!  

 

ETA: Searched the forum for ProFTPd and it's got a really great readme for anyone looking on here:  https://raw.githubusercontent.com/SlrG/unRAID/master/Readmes/ProFTPd-README

 

 

Edited by ChappyEight
Added readme.
Link to post

@Ambrotos - Great description!

 

I did mine a little differently. Rather than using the ProFTPd plugin, I've used DocGyver's SSH Config Tool (which still seem to work on 6.4.0) to allow my unRAID users to have SSH access with individual passwords. (I don't know how long SSH Config Tool is going to stay working, so I may end up switching to ProFTPd some day, anyway.)

 

I then created a /mnt/user/Backups share and a directory called /mnt/user/Backups/username-duplicati for each user. I then set the ownership for each of those directories to username:users with: chown username:users /mnt/user/Backups/username-duplicati

 

I have Duplicati on the user computers connecting with SFTP rather than FTP this way. 

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.