Rsync slow between unRaid and second server

[JoshFink]

Hi, I've spent the better part of a day working on this and I can't seem to nail it down. If I was only transferring a few files I'd be fine but I'm transferring about 40TB so I'd like to get it as fast as possible. Maybe this is as fast as possible but I hope not. 

 

I'm trying to rsync some directories between my old Synology and my new unRaid machine. I'm currently stuck at about ~30 - 45 MB/s

 

The command I'm using is 

rsync --ignore-existing -rahv --progress -- ./ /mnt/user/Media/Movies

• I've mounted the SMB share via the Unassigned Devices Plugin (I can also mount it from the Synology and run rsync from there and get the same speed)
• iPerf3 between the two devices sits around 940 Mbits/sec
• Most of the files are large, anywhere between 1gb and 40gb each. 
• Currently no parity or cache setup
• According to the GUI in the Synology when transferring the same file I'm getting anywhere between 60 - 95 MB/s. Not sure how much I believe this but it is definitely faster. 
• Turbo Write mode (reconstruct_write) is enabled
• nothing else is running on the unRaid. I'm still in the process of getting things copied over so it's not in use yet. 

 

ethtool on unRaid

Settings for eth0:
	Supported ports: [ TP ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Supported pause frame use: No
	Supports auto-negotiation: Yes
	Advertised link modes:  10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Advertised pause frame use: No
	Advertised auto-negotiation: Yes
	Speed: 1000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 1
	Transceiver: internal
	Auto-negotiation: on
	MDI-X: on
	Supports Wake-on: g
	Wake-on: d
	Link detected: yes

 

ethtool on Synology

Settings for eth0:
	Supported ports: [ TP ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Supported pause frame use: No
	Supports auto-negotiation: Yes
	Advertised link modes:  10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Advertised pause frame use: No
	Advertised auto-negotiation: Yes
	Speed: 1000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 1
	Transceiver: internal
	Auto-negotiation: on
	MDI-X: on (auto)
Cannot get wake-on-lan settings: Operation not permitted
	Current message level: 0x00000007 (7)
			       drv probe link
	Link detected: yes

Thanks for the help

[1812]

 

 

 

-nevermind-

Edited July 16, 2017 by 1812

[JoshFink]

Just now, 1812 said:

Search for "turbo write"

 

 

 

 

Thanks. However Turbo Write is already enabled (second to last bullet above). :-) 

[1812]

Just now, JoshFink said:

 

Thanks. However Turbo Write is already enabled (second to last bullet above). :-) 

 

Yeah, I missed that on the first read through.

[JoshFink]

Just now, 1812 said:

 

Yeah, I missed that on the first read through.

 

No worries.. 

 

Any other suggestions?

[ryan]

I am actually in the process and have spent most of the weekend debugging and looking into this. 

 

rsync will use the encryption cipher that ssh uses by standard.. to remedy this, i have enabled all cyphers on the synology box in DSM (choose Terminal > SSH > Advanced > Low Encyption Ciphers), also this on the unRAID box and then used the rsync command with ssh to force to use a different encryption algorithm.. the result?

 

I get the full 115MB/s over rsync to the Syno box, by running two and three of these on a trunk ethernet, i am sending at very high speeds. 

 

rsync -av -e 'ssh -c arcfour' /mnt/user/Media/ [email protected]:/volume1/Media/ --progress

Give it a try,

[JoshFink]

ugggh.. Speeds initially up and then dropped down to 35. I would say no increase here for me. 

[JoshFink]

@ryan I'm not sure what's going on. I'm glad you're at least getting the good speeds. 

 

I just created a new folder of files on the unRaid and copied them to the Synology and I'm getting the same speeds as when I do from the Synology to the unRaid. I'm assuming that if I didn't enable the Cipers correctly for arcfour then I would be getting an error. 

Not sure why it's not working. 

[ryan]

5 hours ago, JoshFink said:

I'm assuming that if I didn't enable the Cipers correctly for arcfour then I would be getting an error. 

 

 

No need to assume! you can check with the -vvv option in the ssh commands to determine which cipher is used. 

 

Have you tried with FTP for comparison? FTP doesnt transfer all the good stuff (ownership, timestamps etc) but it's good for just comparing speeds to those of rsync. Furthermore, rsync is not a performace transfer tool rather a tool used to sync data and check if all is OK. 

 

[Naed]

If encryption isn't important then you could just use netcat instead.  It should be faster.