Decent iperf3 speeds but very subpar rsync/scp/samba transfer speeds with site-to-site VPN

[ogi]

Hello folks,

 

I could use some assistance, as I am lost as to where to go from here.

 

I am quite some distance from my unraid server.  I have a site-to-site VPN configured so I can access the unraid server from my LAN with ease.  With the site-to-site VPN, using iperf3, I'm able to do about 150 Mbps transfer in either direction.

 

Accepted connection from 192.168.3.215, port 62663
[  5] local 192.168.1.32 port 5201 connected to 192.168.3.215 port 62664
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   716 KBytes  5.86 Mbits/sec    0    146 KBytes
[  5]   1.00-2.00   sec  12.4 MBytes   104 Mbits/sec    0   2.83 MBytes
[  5]   2.00-3.00   sec  17.5 MBytes   147 Mbits/sec    0   11.6 MBytes
[  5]   3.00-4.00   sec  17.5 MBytes   147 Mbits/sec    0   8.33 MBytes
[  5]   4.00-5.00   sec  17.5 MBytes   147 Mbits/sec    0   8.33 MBytes
[  5]   5.00-6.00   sec  18.8 MBytes   157 Mbits/sec    0   8.23 MBytes
[  5]   6.00-7.00   sec  17.5 MBytes   147 Mbits/sec    0   8.33 MBytes
[  5]   7.00-8.00   sec  18.8 MBytes   157 Mbits/sec    0   8.30 MBytes
[  5]   8.00-9.00   sec  17.5 MBytes   147 Mbits/sec    0   8.30 MBytes
[  5]   9.00-10.00  sec  17.5 MBytes   147 Mbits/sec    0   8.34 MBytes
[  5]  10.00-10.22  sec  3.75 MBytes   142 Mbits/sec    0   8.34 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.22  sec   159 MBytes   131 Mbits/sec    0             sender

 

When trying to transfer files via samba, rsync over smb, or rsync using SSH, or scp; at best I get transfer speeds of 90 Mbps.  I understand iperf3 will perform faster due to less overhead, but this still seems like a significant penalty. Unfortunately, that transfer rate is just slow enough that I can't stream some of the higher bitrate content on my server.

 

Any suggestions on how to debug where the bandwidth is being lost would be appreciated!

[Michael_P]

Not just overhead, latency is a transfer speed killer 

[ogi]

32 minutes ago, Michael_P said:

Not just overhead, latency is a transfer speed killer 

There is ...significant geographic distance.  Ping duration is 218 ms (wired on both ends).  I wouldn't have thought that to be a significant barrier to transfer speeds...

[Michael_P]

1 minute ago, ogi said:

I wouldn't have thought that to be a significant barrier to transfer speeds...

 

SMB doesn't like latency, at all. 90Mbps isn't bad considering

[ogi]

17 hours ago, Michael_P said:

 

SMB doesn't like latency, at all. 90Mbps isn't bad considering

So while file transfers are going fine, mpv streams are a good deal slower, best I can do via mpv stream is ~60 Mbps, which isn't quite good enough to keep up with higher quality media playback. I'm sturgling to think about what options I could change on the unraid samba side to support (slightly) faster streaming via mpv.

[Michael_P]

1 hour ago, ogi said:

I could change on the unraid samba side

 

If you're happy enough with the file transfers, any tweaking would need to be done in whatever streaming software you're using (ie: using buffers/caching).

[ogi]

23 minutes ago, Michael_P said:

 

If you're happy enough with the file transfers, any tweaking would need to be done in whatever streaming software you're using (ie: using buffers/caching).

yeah mpv doesn't give a lot of options here; I can specify a large cache but the rate of caching is just a wee bit slower than the rate of playback (and significantly slower than a file transfer).  Unfortunately I'm not seeing options in mpv to tweak anything in this regard