Rsync via a dedicated NIC to another unRAID server NIC (peer-to-peer)


TechMed

Recommended Posts

Hello –

 

Anyone have experience with getting rsync to use a dedicated NIC  to another dedicated NIC (rsync-ing peer-to-peer)?

 

Configuration:

unRAID Server #1 – basic Gb network (eth0 & eth1 bonded) with a 192.168.1.x schema with a gateway

AND a 10Gbe NIC with a 10.10.10.X schema and NO gateway.

 

unRAID Server #2 (same config as above) with different values for “X” of course.

 

My goal is to get rsync to exclusively use the 10Gbe NIC’s to do my backups. So, the rsyc action would only occur on the 10.10.10.X network.

 

----------------------------------------------------------------------------------------------

Here is what I have been messing around with:

rsync -u -v -r -P --address=10.10.10.221:/mnt/user/* 10.10.10.225:/mnt/user/vault

---------------------------------------------------------------------------------------------

 

If anyone can help me get this first part to work, I can figure there rest out from other posts I’ve read so far.

 

Thanks in advance everyone!

Link to comment

You just need to ssh to the dest address using the 10GbE IP, e.g., this is what I used:

rsync -av /mnt/path/to/source/ 10.10.10.1:/mnt/path/to/dest/

 

Note however that rsync isn't really made for performance, I could never get it over 150MB/s, even using faster ssh ciphers, that's one of the reasons I stopped using it.

Link to comment

Hi @johnnie.black,

 

Appreciate the reply. And @tr0910 beat me to the obvious question!

 

I did a bunch of digging on the whole rsync backing up, including the post by tr0910 and his remote to China (there has to be a story there)

 

I, more or less, get the concept of SSH, but have not used it yet.

So, let me start by asking: does SSH-ing in 'make the connection' between the two machines?

In other words, does using SSH give me the ability to specify that I want to connect from NIC-A (local) to NIC-B (remote)?

Link to comment
2 minutes ago, TechMed said:

So, let me start by asking: does SSH-ing in 'make the connection' between the two machines?

When using an IP address on the command line rsync will default to ssh connection, and as long as you use the correct destination 10GbE IP it will use that NIC, and use it also on source since normally it's the only one that can connect to the other.

Link to comment

 

 

3 minutes ago, johnnie.black said:

rsync will default to ssh connection

Okay, I thought so.

 

I did not bring any of this up (in original post) as I wanted to make sure I had the basics down.

 

I did lay my rsync string out (very similar to what you suggested) with the assumption data would flow OUT the 10Gbe as it was the only NIC with that addressing schema. It just did not seem like it was working as the speed was atrociously slow.

 

So, based on what your saying...

On 3/20/2019 at 3:45 AM, johnnie.black said:

I could never get it over 150MB/s

The 150 MB/s is the best I can hope for?

 

 

Link to comment

<Sigh>

 

I've been looking around too and cannot find an alternative backup solution for unRAID.

I went down this whole 10Gbe road just to try to p/u some speed (have about 20Tb I need to backup).

I couldn't be more disappointed; I wish I would have saved my money.

 

I really appreciate your chatting with me about it.

Props Bud... you help a lot of people. Thanks!

 

Link to comment
46 minutes ago, TechMed said:

I went down this whole 10Gbe road just to try to p/u some speed (have about 20Tb I need to backup).

If this is for the initial backup you could disable parity and rsync multiple disks at the same time, I did this when I switched from rsync to send/receive, I could do 3 to 4 disks at the same time, it maxed out at around 375MB/s for multiple transfers, in my case it appeared to be the CPUs couldn't handle more, still initial backup was much faster, now with parity it it's only worth doing one disk at a time, but incremental backups are much smaller so I can live with that.

Link to comment

@johnnie.black ran a test and had an interesting outcome.

Any thoughts on why I went from +/- 115Mb/s to 50Mb/s?

Thermal?

Test bkup disk is an - HGST HUS726060ALA640 Ultrastar He6 (Helium Platform) 6TB 7200RPM 64MB Cache SATA 6.0Gb/s

NO cache, but parity is still enabled...

64GB ram and processors did not seem to be working hard at all.

Link to comment

Hmm okay.

So to disable parity, temporarily, just stop the array, 'remove' the parity drive (remove it from the current config), restart the array, ignore the warnings and roll with a test?

Do you think that will at least get me a steady +/- 115Mb/s?

 

After the test just re-enable parity by stopping the array, 're-installing' the parity drive (adding it back to the config), restarting the array and let parity run (for days and days and days... 😵)

 

**I am being verbose in case someone else needs this info**

Link to comment
6 minutes ago, TechMed said:

So to disable parity, temporarily, just stop the array, 'remove' the parity drive (remove it from the current config), restart the array, ignore the warnings and roll with a test?

Yes.

 

6 minutes ago, TechMed said:

Do you think that will at least get me a steady +/- 115Mb/s?

It should, unless you're using very old and slow disks.

 

7 minutes ago, TechMed said:

After the test just re-enable parity by stopping the array, 're-installing' the parity drive (adding it back to the config), restarting the array and let parity run

Yes, just re-assign it and start the array to begin the parity sync.

Link to comment
3 hours ago, TechMed said:

I did a bunch of digging on the whole rsync backing up, including the post by tr0910 and his remote to China (there has to be a story there)

Not much of a story, other than I started using unRaid for storing massive raw camera image data, and got so accustomed to it that I decided to use it in a business solution where web server backups needed to flow internationally.  (give a bloke a hammer and from then on, every problem is a nail)

 

I don't doubt that other solutions might be better for that use case, however unRaid was already in place and it scaled to meet the need.  The only problem I have is that the community of awesome volunteers here are expert on media hoarding, but use cases like mine are on the fringe to the general community.  It wouldn't matter so much if I were a bit more of a Linux gearhead.  I only hope my adventures might spur some more capable coders to clean up my mess and make something that isn't so hacked together and difficult to replicate for future users.

Link to comment

Hi @tr0910 - it is a story nonetheless and I thank you for sharing it (and all of your rsync work). Who knows, in some far off day maybe many others will discover the utility of unRAID for something other than being a media server. At present, I am guilty of the latter, but do have designs on being a "Contributor" someday!

 

 

@johnnie.black - Worked like a charm!

One thing was rather concerning though, during the 'sync' CPU 8 and 12, out of 24, were pegged at 100% usage while the others were only 10-20% usage.

Normal? Is there a way to spread the love around?

 

(And for those wondering, yes, I DO own the discs! ;) )

 

image.thumb.png.53b59d07fb93ef72b9179fe4df499bc1.png

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.