Recently setup a backup Unraid server, and moving files to it. 

 

Ive noticed that the copy speed is around 250Mbps (I have an SNMP app running detecting the speed of the interface)

 

If I run iPerf between the 2 hosts, its full gig speed:

 

[SUM]   0.00-10.00  sec  1.09 GBytes   935 Mbits/sec   71             sender
[SUM]   0.00-10.00  sec  1.08 GBytes   924 Mbits/sec                  receiver

 

My scripts are basic:

 

echo -e "\n"
echo "<b>***Backing up Appdata***<b>"
rsync -avh /mnt/user/sysbackup/appdata/ [email protected]:/mnt/user/appdata/ --delete
echo "<b>Finished Backing up appdata<b>"

 

Any way to speed up rsync copies across a network?

 

Backup server only had a single 4TB disk for now, no parity. 

 

I tried the same transfer with SSDs in both devices, and the speed did double, but that's still half of what gigabit is capable of (the data was scp'd from the ssd manually from the cli) 

Edited February 25, 20251 yr by rorton
Update/amended info

Can you read and write to both servers at full speed using your desktop?

need to setup some cabling, just wifi from laptop to network - will setup a gig switch port in same vlan and try it. 

ok, done a test

Reading from my primary machine to laptop over gig is at full line speed, no problems

 

Writing from laptop to the backup server starts at full line speed, then after a while, the CPU gets unto high utilisation (its only an old HPN40L as a backup server with a Turion AMD Dual core 1.5g CPU), after 3 mins, the transfer drops to about 600Mbps

 

So I know I can get line speed writing to the Backup Server not sustained, but it starts off at that at least.

 

if I run the original test then, from Primary server to backup, using ssd drives at each end, then its stays around 500Mbps

 

So could be the CPU is the factor at the remote end - but saying that, we can get 500Mbps ssd to ssd. That's 62.5MB/s, and they suggest a 3.5" spinning HD, 5400rpm should sustain 150 - 200MB/s

 

When im copying from the Primary Server to laptop over gigabit, the data is on the parity array, so its reading from the array at gig line speed

 

So im still thinking should be faster with a spinning disk? TBH, id he happy with 500Mbps from the spinning disks if they would do it. 

SSH will also add some overhead, you could try mounting the other share using the UD plugin then do rsync over SMB, if it's local LAN no need for SSH.

 

Also keep in mind, that small files, like typical appdata, will always be much slower than large files.

thanks, will try that, the sustained test was a 40GB mkv - im new to the backing up stuff and rsync in general, so I went reading and ssh seemed to be how a lot of people were doing it, ill see what happed with UD later on!

 

Just looking at smb copy. My concern is that I need to mount the shares for the destination host. 
 

my backup server is switched off most of the time, just switched on to run backups. 
 

how will the unassigned devices cope with the share disappearing and then reappearing? The beauty of doing this via ssh is it makes the connection as it needs it?

9 hours ago, rorton said:

how will the unassigned devices cope with the share disappearing and then reappearing?

I think it can, but best to ask in the UD support thread.

think I have the solution! The backup server is an old chassis but a brand new install of Unraid with new disks. I was still getting it configured so hadn't installed plugins like fix common problems. 

 

Installed that, ran it, and it mentioned write cache not enabled on the hard disks!

 

Switched it on, and boom, 650Mbps. nearly 3 x faster. 

 

 

ah, spoke too soon once the disk cache (I assume) is full, speed drops back down again

 

hmm, looks like write cache isn't staying when rebooted. 

 

I tested last night, and was good, powered back up today, and not enabled, have enabled it again and transfer is now at around 600 - 650Mbps

I just need to fix write cache so its on all the time, strange as my primary server doesn't have this problem