What is the best way to keep 2 unRaid servers in sync?

[De1taE1even]

I just got finished building a small unRaid box (build thread coming soon) to be used primarily as a backup for my media collection, which has become quite extensive (located on my primary unRaid machine).  My question is, what would be the best way to "sync" my media library between the two unRaid servers?

 

I had a few thoughts in my head, but I wanted to make sure I wasn't missing out on a better way.

1. I could write a simple script, running on a periodic cron

2. Maybe Crashplan could do it?

 

It'd be really nice if there exists software that would allow me to pick-and-choose folders/shares to backup, but I'm not sure such a feature exists.

 

Thanks everyone!

-Dave

[Kir]

I'd say rsync in a script.

[garycase]

rsync can do it from Linux

 

... personally, I use a set of SyncBack profiles to backup my two primary UnRAID servers to my backup server.  The profiles are set to run weekly -- a schedule task runs that turns on all the UnRAID servers (via WOL), and then 3 minutes later SyncBack is invoked to do the backups.  Works perfectly ... but of course it requires a Windows machine to run SyncBack  [i know at least a couple UnRAID users who do the same thing, but have SyncBack running in a Windows VM on one of their UnRAID systems to avoid the need for another system.]

 

[Leifgg]

As garycase is pointing out SyncBack as an easy way of doing it. There is a free version you can download and try.

 

CrashPlan could possible work but your backup would be stored in their backup format and you need to figure out how to mount a share on the remote server. Could possible be done with plugin Unassigned Devices using SMB Mount.

 

rsync is perfect for the task but has a steep learning curve and requires a lot of time to setup if you aren’t already familiar on how to use it. Have a look here for my setup :

http://lime-technology.com/forum/index.php?topic=38554.msg358683#msg358683

 

There is also the Syncthing Docker that you could look at. I haven’t used it between two servers myself but it might work.

 

Edit: Forgot to mention the Websync Docker. This is rsync with a GUI. Haven’t tried but looks promising.

[foo_fighter]

Isn't this what BT sync is for?

[De1taE1even]

rsync, syncback, websync, and bittorrentsync so far.  Thanks for the suggestions!  I'll do some research tonight.  Pre-clears all finished finally, so I'm just about ready to get this going.

 

Thanks again!

[De1taE1even]

Right now I'm experimenting with BTSync (PHaZe plugin) and it's working really well.  I'm currently syncing 8TB worth of media at about 25MBps.  Not terribly fast, but as long as it's dependable, it'll work for me.  Once the initial backup is finished, syncing as I add media should be quick.

 

I should note that BTSync is indexing the folder at the same time as transferring.  Once the indexing is done, I might see quicker transfer speeds.

[WeeboTech]

BTSync is be good for the initial and incremental sync.

 

Later on if you choose to use something like bitrot or bunker to store hashes in the extended attribute, you may need to resort to rsync.

I'm not sure if BTSync will keep those attributes.