Linking Multiple unRaid Servers

[nacat78]

A way for one unraid server to communicate with another to either combine data from both to share on network as one share and/or a way for one server to communicate to another for mirror backup.

[bnevets27]

I have been wondering if there was away to have a second unraid box setup as a seamless redundant backup of the first,  plug-ins and all so if the first one goes out of service for whatever reason all access is moved to the backup server without any hiccups

 

Sent from my SM-N9005 using Tapatalk

 

 

[jonp]

This is a great concept and one that we have discussed internally.  It's a bit tricky but possible.  Definitely not something for the short term...

[wmcneil]

I would be very interested in simply having two systems, which are located in different houses, mirrored.  The auto failover feature is a good feature idea, but allow me to expand a little on just the mirroring function......Having a different physical location for backup is a big and obvious advantage.....While remote access is getting better all the time, there are still issues that can only be solved by having a local server at each house. An example of this is that Plex does not support adaptive bit rate video streaming, which results in a very widely varying remote video streaming quality, due to the large latency swings on the internet during peak usage periods. I would love to add a second unraid server at my second home, and it would be very useful to have a mirroring function.

[storagehound]

I have to "ditto" those other options.  However those ideas are implemented having a friendly way to move/copy data from one unRAID server to another is a great idea.  Especially if it has a robust GUI opiton.

[wmcneil]

  I ended up implementing a sync of array data between machines at two different houses using scripts and cron jobs based on the rsync command. I am using the bandwidth limit feature of rsync, as well as QOS on my routers, so that the sync does not interfere with the "normal" use of the network at each house. This approach is for the data only, and does not cover the unRAID OS or plugins, but having "just" the data sync is a huge benefit. The mirroring of my media libraries at my second home, and off-site backup of critical data are both very useful/valuable.

  Because my bandwidth is somewhat limited at my houses (12dn/5up at primary residence, 8dn/1up at second home), turning the bandwidth used for the sync, as well as using QOS on the routers is a critical part of having a scheme that does not consume the available network entirely when it is desired to use the network for other "normal" uses. I can certainly see that such a sync feature would be a great feature to add to unRAID down the road , but when the sync is remote, and bandwidth is limited, such a feature would have to have lots of option settings (and disclaimers to the users about potential issues) to produce the desired result.

[jonp]

  I ended up implementing a sync of array data between machines at two different houses using scripts and cron jobs based on the rsync command. I am using the bandwidth limit feature of rsync, as well as QOS on my routers, so that the sync does not interfere with the "normal" use of the network at each house. This approach is for the data only, and does not cover the unRAID OS or plugins, but having "just" the data sync is a huge benefit. The mirroring of my media libraries at my second home, and off-site backup of critical data are both very useful/valuable.

  Because my bandwidth is somewhat limited at my houses (12dn/5up at primary residence, 8dn/1up at second home), turning the bandwidth used for the sync, as well as using QOS on the routers is a critical part of having a scheme that does not consume the available network entirely when it is desired to use the network for other "normal" uses. I can certainly see that such a sync feature would be a great feature to add to unRAID down the road , but when the sync is remote, and bandwidth is limited, such a feature would have to have lots of option settings (and disclaimers to the users about potential issues) to produce the desired result.

 

wmcneil, thank you for sharing this.  This is EXACTLY what we would have to do to make this work in unRAID (support the advanced "tweaking" of syncing so that we could control bandwidth throttling).  QoS is another way to go to control this at the router level, but even still, users should have the option to configure bandwidth throttling and other settings for this outside of using third party equipment (e.g. a router).  This is also why this feature is probably a little ways out from being a reality just yet.

 

Again, thank you for your post as this was very helpful and calls out several of the details that need to be considered if we were to implement something like this natively into unRAID.

[caseyparsons]

  I ended up implementing a sync of array data between machines at two different houses using scripts and cron jobs based on the rsync command. I am using the bandwidth limit feature of rsync, as well as QOS on my routers, so that the sync does not interfere with the "normal" use of the network at each house. This approach is for the data only, and does not cover the unRAID OS or plugins, but having "just" the data sync is a huge benefit. The mirroring of my media libraries at my second home, and off-site backup of critical data are both very useful/valuable.

  Because my bandwidth is somewhat limited at my houses (12dn/5up at primary residence, 8dn/1up at second home), turning the bandwidth used for the sync, as well as using QOS on the routers is a critical part of having a scheme that does not consume the available network entirely when it is desired to use the network for other "normal" uses. I can certainly see that such a sync feature would be a great feature to add to unRAID down the road , but when the sync is remote, and bandwidth is limited, such a feature would have to have lots of option settings (and disclaimers to the users about potential issues) to produce the desired result.

 

Care to do a detailed write-up? Many other users could benefit from your experience.

[wmcneil]

Care to do a detailed write-up? Many other users could benefit from your experience.

 

I've attached my notes on setting up rsync. This should be a fairly complete "how-to".

readme_rsync.txt

[caseyparsons]

Care to do a detailed write-up? Many other users could benefit from your experience.

 

I've attached my notes on setting up rsync. This should be a fairly complete "how-to".

Cool! Thanks.

[ootuoyetahi]

Would this be applicable to backup an UnRAID server to another unRAID server?

[gundamguy]

Would this be applicable to backup an UnRAID server to another unRAID server?

 

Yes, it can be done with rsync but it's takes a bit of command line set up first.

[Porterhouse]

I'd like to add my support for this feature. This will benefit both users and product maturity/appeal/take-up. I also have a Thecus (and other NAS) which have GUI functionality to stack and rsync multiple devices. On a personal level it will save me a lot of manual setup (and worry in it's absence) to get a second (or third) unRAID server up and running ( in a safe location with a secure VPN open) and all shares fully in sync - to me, this is EXACTLY where I want to get to closer to unRAID nirvana .. and note that I recently had a laptop stolen from inside my house  less than 6 feet from my unRAID NAS with 6x 4Tb disks in hot swap bays they could also have pulled .. and not yet rsync'd to my 'backup' unraid svr.

[TheBuz]

+1 i like everything mentioned above.

 

Or maybe even just an Unraid manager, one UI for managing multiple servers

[nicksphone]

Reviving this old topic anything since 2015 to do this yet?

[IFC2000]

I've recently been using rsync and syncthing to do syncing between remote unraid servers and local backups devices which works pretty well.  It would be great if there was a plug and play way to do this built in to unraid, but until then there are some really good alternatives out there.

[Abe677]

Adding my interest here. I would like to be able to backup one Unraid server to another over the internet. Being able to throttle bandwidth use would be a must, and a bonus would be the ability to schedule the activity in some way. For example, there may be certain times of the day when I wouldn't want the backups to consume any bandwidth, but when everyone is sleeping "release the hounds" (with possible bandwidth limits even then). (Thanks to the person who mentioned the rsync throttle feature, which I was not aware of.)