TechMed Posted March 19, 2019 Share Posted March 19, 2019 Hello – Anyone have experience with getting rsync to use a dedicated NIC to another dedicated NIC (rsync-ing peer-to-peer)? Configuration: unRAID Server #1 – basic Gb network (eth0 & eth1 bonded) with a 192.168.1.x schema with a gateway AND a 10Gbe NIC with a 10.10.10.X schema and NO gateway. unRAID Server #2 (same config as above) with different values for “X” of course. My goal is to get rsync to exclusively use the 10Gbe NIC’s to do my backups. So, the rsyc action would only occur on the 10.10.10.X network. ---------------------------------------------------------------------------------------------- Here is what I have been messing around with: rsync -u -v -r -P --address=10.10.10.221:/mnt/user/* 10.10.10.225:/mnt/user/vault --------------------------------------------------------------------------------------------- If anyone can help me get this first part to work, I can figure there rest out from other posts I’ve read so far. Thanks in advance everyone! Quote Link to comment
JorgeB Posted March 20, 2019 Share Posted March 20, 2019 You just need to ssh to the dest address using the 10GbE IP, e.g., this is what I used: rsync -av /mnt/path/to/source/ 10.10.10.1:/mnt/path/to/dest/ Note however that rsync isn't really made for performance, I could never get it over 150MB/s, even using faster ssh ciphers, that's one of the reasons I stopped using it. Quote Link to comment
tr0910 Posted March 20, 2019 Share Posted March 20, 2019 What did you replace it with?How fast can you transfer now?Sent from my chisel, carved into granite Quote Link to comment
JorgeB Posted March 20, 2019 Share Posted March 20, 2019 I'm using btrfs send/receive, get up to 300MB/s, also has other advantages for me, like only metadata is updated on for example a folder rename/move, though obviously only works for btrfs filesystems. Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 Hi @johnnie.black, Appreciate the reply. And @tr0910 beat me to the obvious question! I did a bunch of digging on the whole rsync backing up, including the post by tr0910 and his remote to China (there has to be a story there) I, more or less, get the concept of SSH, but have not used it yet. So, let me start by asking: does SSH-ing in 'make the connection' between the two machines? In other words, does using SSH give me the ability to specify that I want to connect from NIC-A (local) to NIC-B (remote)? Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 2 minutes ago, TechMed said: So, let me start by asking: does SSH-ing in 'make the connection' between the two machines? When using an IP address on the command line rsync will default to ssh connection, and as long as you use the correct destination 10GbE IP it will use that NIC, and use it also on source since normally it's the only one that can connect to the other. Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 3 minutes ago, johnnie.black said: rsync will default to ssh connection Okay, I thought so. I did not bring any of this up (in original post) as I wanted to make sure I had the basics down. I did lay my rsync string out (very similar to what you suggested) with the assumption data would flow OUT the 10Gbe as it was the only NIC with that addressing schema. It just did not seem like it was working as the speed was atrociously slow. So, based on what your saying... On 3/20/2019 at 3:45 AM, johnnie.black said: I could never get it over 150MB/s The 150 MB/s is the best I can hope for? Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 I could never get it faster than that, and after googling it found various users complaining of the same, only way I found to speed it up was by running multiple instances, but that's not a good option with Unraid because of how parity works. 1 Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 <Sigh> I've been looking around too and cannot find an alternative backup solution for unRAID. I went down this whole 10Gbe road just to try to p/u some speed (have about 20Tb I need to backup). I couldn't be more disappointed; I wish I would have saved my money. I really appreciate your chatting with me about it. Props Bud... you help a lot of people. Thanks! Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 46 minutes ago, TechMed said: I went down this whole 10Gbe road just to try to p/u some speed (have about 20Tb I need to backup). If this is for the initial backup you could disable parity and rsync multiple disks at the same time, I did this when I switched from rsync to send/receive, I could do 3 to 4 disks at the same time, it maxed out at around 375MB/s for multiple transfers, in my case it appeared to be the CPUs couldn't handle more, still initial backup was much faster, now with parity it it's only worth doing one disk at a time, but incremental backups are much smaller so I can live with that. Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 You mention 5 minutes ago, johnnie.black said: I could do 3 to 4 disks I get the disabling Parity, but you mention "disks" instead of files/folders. If you have the time, would mind elaborating or pointing me to a post(s) that explain? Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 I was using btrfs send/receive from disk to disk, but you can do the same with rsync, disk shares must be enable, then e.g. rsync -av --progress /mnt/disk1/share/ 10.10.10.1:/mnt/disk1/share/ Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 got it... thanks! Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 @johnnie.black ran a test and had an interesting outcome. Any thoughts on why I went from +/- 115Mb/s to 50Mb/s? Thermal? Test bkup disk is an - HGST HUS726060ALA640 Ultrastar He6 (Helium Platform) 6TB 7200RPM 64MB Cache SATA 6.0Gb/s NO cache, but parity is still enabled... 64GB ram and processors did not seem to be working hard at all. Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 Just now, TechMed said: Any thoughts on why I went from +/- 115Mb/s to 50Mb/s? That's about right with parity enable and default writing mode, it should be faster with turbo write (only relevant while using parity) Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 Hmm okay. So to disable parity, temporarily, just stop the array, 'remove' the parity drive (remove it from the current config), restart the array, ignore the warnings and roll with a test? Do you think that will at least get me a steady +/- 115Mb/s? After the test just re-enable parity by stopping the array, 're-installing' the parity drive (adding it back to the config), restarting the array and let parity run (for days and days and days... 😵) **I am being verbose in case someone else needs this info** Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 6 minutes ago, TechMed said: So to disable parity, temporarily, just stop the array, 'remove' the parity drive (remove it from the current config), restart the array, ignore the warnings and roll with a test? Yes. 6 minutes ago, TechMed said: Do you think that will at least get me a steady +/- 115Mb/s? It should, unless you're using very old and slow disks. 7 minutes ago, TechMed said: After the test just re-enable parity by stopping the array, 're-installing' the parity drive (adding it back to the config), restarting the array and let parity run Yes, just re-assign it and start the array to begin the parity sync. Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 Cool. Thanks! I'll give it a go... Quote Link to comment
tr0910 Posted March 21, 2019 Share Posted March 21, 2019 3 hours ago, TechMed said: I did a bunch of digging on the whole rsync backing up, including the post by tr0910 and his remote to China (there has to be a story there) Not much of a story, other than I started using unRaid for storing massive raw camera image data, and got so accustomed to it that I decided to use it in a business solution where web server backups needed to flow internationally. (give a bloke a hammer and from then on, every problem is a nail) I don't doubt that other solutions might be better for that use case, however unRaid was already in place and it scaled to meet the need. The only problem I have is that the community of awesome volunteers here are expert on media hoarding, but use cases like mine are on the fringe to the general community. It wouldn't matter so much if I were a bit more of a Linux gearhead. I only hope my adventures might spur some more capable coders to clean up my mess and make something that isn't so hacked together and difficult to replicate for future users. Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 Hi @tr0910 - it is a story nonetheless and I thank you for sharing it (and all of your rsync work). Who knows, in some far off day maybe many others will discover the utility of unRAID for something other than being a media server. At present, I am guilty of the latter, but do have designs on being a "Contributor" someday! @johnnie.black - Worked like a charm! One thing was rather concerning though, during the 'sync' CPU 8 and 12, out of 24, were pegged at 100% usage while the others were only 10-20% usage. Normal? Is there a way to spread the love around? (And for those wondering, yes, I DO own the discs! ) Quote Link to comment
JorgeB Posted March 21, 2019 Share Posted March 21, 2019 27 minutes ago, TechMed said: Normal? Yep, not much you can do about that. Quote Link to comment
TechMed Posted March 21, 2019 Author Share Posted March 21, 2019 Good to know. Looked at the 'parallel' command, but that is a bit over my head so I will stick to the basics for now. Thanks again! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.