Yet, another slow network transfer post

[russo]

Ok, i've been troubleshooting this for the past 2-3 days and Im now at my whitts end. Im summoning your help.

 

Backstory:

for the past 2-3 years I've been running (2) Truenas servers.  One was a backup and the other is (still is) my main media sever.

 

What I need to do is transfer all of my content (~50TB) from my main Truenas server over to my new Unraid.

When i did transfers in the past between my 2 Truenas servers, I would get 80-90MB/s (almost hitting my 1 gig max.) My method of transfer between the servers has always been SFTP over ssh via Flashfxp. However, when i try to transfer from my Truenas to my Unraid, I can not achieve more than 21-22MB/s (usually much lower)

 

 

Things I have tried:

 

*allowed my data to pass through my cache drive first.

 

*activate turbo write

 

*checked and double check my cables connected to this server (all of CAT5e or 6) - NO CAT5 in system-

 

*Started out using the onboard Realtek 1gig nic, swapped out to 

  Intel Gigabit CT PCI-E Network Adapter EXPI9301CTBLK, then swapped out to a 

  Broadcom NetXtreme 10/100/1000Mbps BCM5751 NIC Nic

(As you can see, no improvement with the 3  different cards.

 

*Iperf3 test from the Truenas sever to the Unraid seems to check out "OK"

https://imgur.com/qmQ4SPw

 

*ethtool seems to show no issues:

Quote

Settings for eth0:
    Supported ports: [ TP ]
    Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Supported pause frame use: No
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Speed: 1000Mb/s
    Duplex: Full
    Auto-negotiation: on
    Port: Twisted Pair
    PHYAD: 1
    Transceiver: internal
    MDI-X: off (auto)
    Supports Wake-on: pumbg
    Wake-on: g
        Current message level: 0x00000007 (7)
                               drv probe link
    Link detected: yes

 

Now for desperate times:

* removed ALL hard disc drives from the unraid sytem except for (1) 16 TB drive.  No improvement with this. Transfer rates still bad. There doesnt seem be to a bottle neck.

 

attached are my diagnostics.

 

Anything thing else I should try?

 

 

 

 

tower-diagnostics-20230529-2016.zip

Edited May 30, 2023 by russo

[JorgeB]

Is a local copy fast, i.e., is the problem only when using the network?

[russo]

It was pretty instantaneous for a 2gig file.  I did this in Krusader.

 

[russo]

I just copied a 900gig folder that took less than 2 min.  Seems pretty fast with copying.

[JorgeB]

Looks like it's network related then, do you have another PC you could use as source, like a Windows PC?

[russo]

im on a win 10 laptop on wifi . I can hook it up with an ethernet cable.

[russo]

I hard wired my windows 10 laptop, opened up a sftp over ssh  connection and transferred a 2 gig file.  Transfer was under 21MB.

 

I did confirm my laptop does have a 1 gig nic card. 

 

 

 

[JorgeB]

Are you using any kind of compression? Try transferring a file over SMB, using Windows explorer.

[russo]

No compression that I know of.

 

However, now we are making progress!!!!!!

 

Here are my SMB transfers.

 

 

 

I actually was averaging about 85-93MB/s on this transfer (below)

 

 

 

 

So what does this tell us? I mean, we are practically getting 1 gig transfers via SMB in all directions?  Just  a reminder, when I did SFTP transfers from

TRUENAS ---->TRUENAS (BACKUP) I would generally saturate my network connection. I cant do this with TRUENAS to UNRAID.

 

 

thanks for helping me.

[russo]

I just did a reverse SFTP transfer from UNRAID to my TRUENAS.  Never go more than 24MB/s

[JorgeB]

Try transferring from TrueNAS to Unraid using rsync over SSH, make sure no compression is used with rsync (-x), for example, from the Unraid side:

 

rsync -av root@TrueNAS_IP:/mnt/pool/dataset/ /mnt/user/share/

 

if you get permission errors, which is normal with TrueNAS and rsync add --no-perms to the command.

[russo]

awesome!  I'll try this in a bit and report back

 

thanks

[russo]

rsync -av root@TrueNAS_IP:/mnt/pool/dataset/ /mnt/user/share/ --no-perms

 

is that the right command for no perms?

[JorgeB]

rsync -av --no-perms /path /path

 

Though that might also work.

[JorgeB]

Ohh, and also add --progress to see the speed:

 

rsync -av --progress --no-perms /path /path

 

 

[russo]

Ok, Rsync shows that im hitting the 1gig transfer rate.  Im going to guess that there's a setting somewhere within my FLashFXP client that is causing the slow down.  I didnt experience this slowdown when i transferred between the 2 Truenas servers.

I think i'll use this Rsync for now on since it consistently hits over 100MB/s

 

Dont judge me by my file names 🤪

 

again, a big thank you to YOU