Slow transfer from headless linux server to Unraid

[xalten]

I'm having extremely slow transfer speeds going from my headless Fedora server to my new unRaid box.  The only thing I know I may have done wrong was to set up a parity drive before the initial upload but even still, I'm only getting ~10MB/s copying over 6TB of data which seems extremely slow. 

 

My setup:

 

Source is a Fedora Server, 3 drives mounted in a LVM.  Drives are all WD reds.  I've SFTP'd files from this server to my windows box in the past and have done 20 gig files in 5 - 10 mins so I'm sure it can supply faster than what my current set up is providing

 

Destination is a unRaid box, 2 x Xeon E5-2660, 64Gb Ram, 3 x 4 TB HGST NAS drives (1 is parity),  and a 120Gb SSD cache drive

 

Network is full gigabit with gig nic cards in both servers and a gig switch connected all with Cat 6

 

On my source I mounted my main share (/mnt/user/Storage) via NFS and now I'm using rsync to copy the files over, using the --progress flag I'm seeing around 10MB/s transfer speeds.  Is this not the way I should be doing this.  Is there anything I can check or do to get these speeds up?  If I stop my rsync and disable my parity drive will it make that much difference?  I'm reading of users who with parity were experiencing 40MB/s transfer speeds and I'm only 1/4th that so I'm not sure what's going on here

[itimpi]

10MBps sounds remarkably close to what you would get if the network is running at 100Mbps rather than the 1Gbps you think it is.    If you are sure all the hardware is capable of gigabit LAN then it might be worth checking the cabling as if any connector in the cable is broken then it would automatically drop the speed to 100Mbps.

[xalten]

That would make sense.  Both adapters are reporting at 1000Mb/s, even if a cable was bad, would they still report at those speeds?

 

Destination Server

Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: on (auto)
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

 

Source Server

Settings for enp3s0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: Symmetric
        Link partner advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: on
Cannot get wake-on-lan settings: Operation not permitted
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes

[xalten]

I did a test and set up a SMB share and copied a 13Gb file from my windows box to my Unraid, it was transfering at 100+MB/s

[xalten]

And I've tested my transfer speeds to the source box, using SFTP I was getting ~60MB/s which is still 6 times faster than what I'm getting from my source to my Unraid box

[ken-ji]

How fast does plain scp run for you?

rsync can be very slow (depending on the flags used) because it might be asking the unraid server to supply the source files for comparison before it sends the deltas.

 

Try running this:

#unRaid
nc -v -l -n 2000 >/dev/null

#Fedora
dd if=/dev/zero bs=1000M count=10 | nc -v -v -n <<unraid hostname/ip>> 2000
10485760000 bytes (10 GB) copied, 18.2118 s, 576 MB/s

 

then try again, changing the unRaid one to:

nc -v -l -n 2000 > /mnt/cache/dummyfile

 

This will send 10GB down the wire from the Fedora server to the unRaid server (RAM in #1; cache SSD in #2)

and you'll get a how fast after.

 

[xalten]

I'm not sure how to answer that as I don't know how to test a SCP transfer?  My buddy told me that rsync can be slow when running over ssh and suggested I set up rsync-daemon on my fedora server so I did that and got it running, it was copying at ~60mb/s which is much faster than the 10.  Unfortunately my session closed in the middle of the night and this morning I continued my sync and now its running at ~12MB/s using the exact same command.  I'm using

rsync -az rsync://[email protected]:xxxxx/storage /mnt/user/Storage/ --progress

 

as for the commands you wanted me to try, I tried running the NC command on my unRaid box but it's not available.  I looked into that and I believe it is netcat? So I installed netcat but the NC command still isn't available.

[xalten]

so I read up on netcat and found that I could run it using the netcat command instead.  I tried multiple different ways but from my Fedora box all I get is connection refused.  I think there is something (firewall?) blocking it from the UnRaid box.  I'm think that it might not be worth it troubleshooting much more as I only have 500Gb or so left to transfer

[John_M]

It might be worth trying the rsync command without the -z option.

 

[ken-ji]

My bad - nc (netcat) is in an additional slackware package nc-1.10-x86_64-1.txz (I think nerdtools has this)

in any case scp is as simple as

scp <localfile> root@<unraid>:/mnt/cache/<somedir>

or

scp <localfile> root@<unraid>:/mnt/user/<somedir>

for single file copies and

scp -r <localdir> root@<unraid>:/mnt/cache/<somedir>

or

scp -r <localdir> root@<unraid>:/mnt/cache/<somedir>

for directory copies ( -r being recursive )