Any solutions for remote file access of home server while away from home?

[ericswpark]

Basically title. When I'm at home, I'm getting great performance. But when I'm away from my home network and access my server using a VPN, it adds latency and SMB slows to a crawl. Think fifteen seconds just pulling up a directory listing crawl. Or 30 seconds transferring a 2 MB file crawl.

 

How much latency? Not much. When I'm at home, I get <100 ms latency. When I'm out and about, the latency varies depending on the connection, but I've noticed the horrible performance even with 200 ms latency.

 

I wanted to use something like SFTP, but learned that UnRAID doesn't support SFTP unless you log in as the root user. Which is a huge pain because you need to manually set correct permissions and other members of my family obviously cannot use the root user to access everything on the server.

 

I previously tried Nextcloud, but found the experience very janky. The Nextcloud Docker tends to corrupt the DB, and having Nextcloud as a middleman still feels clunky compared to mounting the server directly using SMB/SSHFS.

 

Anybody have any tips on alternative file access methods? Maybe an FTPS/SFTP server via Docker? How do you access files on your server when you're away from home through VPN that has high latency?

[JorgeB]

Did you try Wireguard?

[ericswpark]

@JorgeB thanks, but I'm looking for alternative file transfer technologies, not VPN protocols. As far as I'm aware, Tailscale uses Wireguard, and I'm using Tailscale right now.

 

I'm looking for a way to get better speeds over VPN, as SMB doesn't perform very well over high-latency networks bridged via VPN.

[Michael_P]

12 hours ago, ericswpark said:

I'm looking for a way to get better speeds over VPN

 

Define 'better speeds', what are you seeing now? What's the link's speed? What's the VPN server's specs?

 

SMB over VPN tends to be slower, but it can also be plenty fast

[ericswpark]

3 hours ago, Michael_P said:

 

Define 'better speeds', what are you seeing now? What's the link's speed? What's the VPN server's specs?

 

SMB over VPN tends to be slower, but it can also be plenty fast

Sure! So on SMB over VPN, I'm getting around 1-5 MB/s on a good day, but with horrible latency. That means that file listings take a long time to load (around 10-20 seconds), transfers take a long time to start (around 30 seconds minimum), and the actual progress update is rather sporadic and "jumpy" (about once every 10-20 seconds).

 

For contrast, on LAN (again, 50 ms ping on average), listings take less than 2 seconds, transfers start in less than 2 seconds, and the actual progress update is done extremely rapidly (once every second or so).

 

I get 200 Mbps down/up (symmetrical), and the server has 100 Mbps down/up (also symmetrical). I know losses and overhead and all that, but I should be getting around 10 MB/s, not 1-5 MB/s. And I do get 10 MB/s if I transfer using UnRAID's built-in SFTP.

 

Tailscale uses DERP relays in case a direct connection cannot be established, but most of my transfers are done over a direct link, so VPN server speed shouldn't be an issue. On restrictive networks I take the DERP relay overhead into account.

 

I mean, 1-5 MB/s over VPN is also "fast", but compared to SFTP I think it's rather slow and high-latency to be usable. Sure, it works in a pinch, but actually navigating and accessing files on the NAS is painful, to the point where all of my family members complain whenever they have to do it away from home.

Edited October 18, 2023 by ericswpark

[Michael_P]

1 hour ago, ericswpark said:

so VPN server speed shouldn't be an issue.

 

It could be if it's a low end processor, especially using SMB

[ericswpark]

16 hours ago, Michael_P said:

 

It could be if it's a low end processor, especially using SMB

Tailscale's DERP servers are only used as a last resort, and when it is being used I don't fault the SMB speeds.

 

The server is running a 4600G, and the client device is an M1 Pro. I don't think encryption overhead is to blame here.

 

If SFTP over the same connection setting can do 10 MB/s, then there's no reason why SMB can do so as well, given improved latency. But VPN will always add latency, and SMB suffers over high-latency links.

 

Anyway, I think we've gone off topic. I'm looking for a protocol or something else to use as an alternative to SMB, only when I'm accessing the server away from home. UnRAID's built-in SFTP (and FTP) servers don't cut it because they only permit root user login, which I cannot give out to my users (and is generally more tedious as I have to go in and manually set correct permissions after the fact). At this point. I don't think trying to improve what is hitting against a theoretical limit of SMB is going to be beneficial here.

[Michael_P]

54 minutes ago, ericswpark said:

then there's no reason why SMB can do so as well

Exactly the point I'm trying to make - even with high latency, you should be able to saturate your 100Mb link - most people that have issues are on 1Gb or higher links. If you're hell bent on it, I'd set up a lightweight VM

[ericswpark]

1 hour ago, Michael_P said:

Exactly the point I'm trying to make - even with high latency, you should be able to saturate your 100Mb link - most people that have issues are on 1Gb or higher links. If you're hell bent on it, I'd set up a lightweight VM

Not really. When I said "there's no reason why SMB can do so as well", I meant that in terms of encryption overhead. I also put in the qualifier: "given improved latency."

 

However, when you put SMB and SFTP next to each other, you can see why it doesn't perform very well over VPN. SMB is a very chatty protocol, and the increased latency means slower transfer speeds and the general choppyness that I am experiencing. In terms of protocol overhead, SMB just cannot compete with SFTP (and/or other protocols).

 

I was hoping there would be some sort of Docker thing available, but I think I may have to resort to a VM, as you said. Thanks for the reply.

Edited October 19, 2023 by ericswpark

[strike]

36 minutes ago, ericswpark said:

I was hoping there would be some sort of Docker thing available,

There is, SFTPGO. I use it myself, you can set up how many users you want, don't need root access. A vm just for this is very overkill. Just remember to det the correct permissions using the extra parameters IIRC.

[ericswpark]

Just now, strike said:

There is, SFTPGO. I use it myself, you can set up how many users you want, don't need root access. A vm just for this is very overkill. Just remember to det the correct permissions using the extra parameters IIRC.

Great, thank you. I'll check it out.