Hi, I thought I'd share my success getting rsync faster. I almost doubled the transfer speed.

 

I found that the bottleneck was SSH's CPU usage on the target box, due to decrypting being expensive. You can improve performance easily by using blowfish encryption, which is less secure than 3DES, but significantly less computationally expensive.

 

Put this in your rsync to make that happen:

rsync --rsh="ssh -c blowfish" ....

 

This got me from 15MB/s to about 20-25MB/s.

 

I got more gain by ditching ssh for rsh. RSH is completely insecure. Do not use it over the open Internet. If you're on a private home network, then it completely removes the SSH overhead.

 

Getting it to work on unraid took a little fiddling. You can get an rshd package from http://pkgs.org/slackware-13.1/slackware-i486/netkit-rsh-0.17-i486-1.txz.html

 

Edit

/etc/inetd.conf

to uncomment the rsh.d (starts shell) line, and run

kill -HUP `pgrep inetd`

so inetd reloads its config, and knows to listen for rsh connections, and call rshd to handle them.

 

Edit

~/.rhosts

(create it, if it doesn't exist) to list the IP addresses that are allowed to connect to the system.

 

When debugging,

tail /var/log/syslog

to see what's not working.

 

Using RSH got me from 15MB/s to 25-30MB/s.

Remember any edits to /etc/inetd.conf and ~/.rhosts will be lost on a reboot.  You need to setup those edits to happen from your go script if you want to preserve your work.

Try "arcfour" instead of "blowfish" as the kind of encryption.

I use it for large database files all the time.

Seems to be the fastest from my tests.

YMMV though.

Thanks for the extra infos