I often FTP files from a VPS in Europe to my home in Canada.  I find that with applications like FileZilla I will cap out at about 2.5MBps per file (aka 20Mbps), which is well below my internet capacity of 250Mbps.  Under Windows I have been using Bitkenix which uses multi-segment downloads and I get speeds up to around 8-9Mbps - not bad but still using less than a third of my bandwidth.  But I am happy with that speed and would like to be able to transfer files directly into unRAID.

 

Are there any clients for unRAID that use multi-segments to speed up FTPs?  I believe lftp can do muti-segments but is it command line only or is there a front end for it?

Have you searched in Community Applications?

Yes I have and there are four hits in Network Services: FTP Clients / Servers.  But it isn't clear which ones, if any, support this functionality although I am pretty sure that Filezilla doesn't as its author is strangely hostile to the whole concept of multi-segment FTPs.

I don't know either. CA has links to the support threads, and those threads often have links to the application's support website.

Bump - anybody know the answer to this, or another way to be able to quickly transfer files from Europe to North America?  As I mention in the OP regular FTPs top out at 20Mbps, but I can get multi-segment FTPs up to about 80 Mbps - faster but still only less than 1/3 of my rated internet speed.

lftp works pretty well once you tune the config file, but as you mentioned, it's command-line only.  I just use a screen instance and let a mirror command run in the background overnight.  It doesn't saturate my connection but it's reasonable.

 

One thing I've had really good luck with from my seedbox is transferring files via HTTPS and a segmented downloader like Internet Download Manager.  For whatever reason I get much higher throughput that way vs (S)FTP(S).  May want to give it a shot.

 

Another possibility that may help is to route traffic through CloudFlare's CDN, which a lot of people do to make their Plex streaming more solid.  May or may not help for regular file transfers though.

Edited June 23, 20179 yr by DavejaVu

Thanks - any idea what config, if any, you have to do on the server side to facilitate HTTPS downloads?  My VPN server is running Ubuntu and I have Apache running on that VPN server as well.  Can I just then move the files to a directory that Apache has access to and then initiate a HTTP(S) download by clicking on the file?

 

But I guess this is still not a solution using docker as I will need to run IDM on Windows.  Bitkinex seems to work ok on Windows but, as I mention above, I am only able to use 1/3 of my 250 Mbps download speed.  I may need to do some turning of various parameters to get it a bit faster, or do the same with IDM or maybe that is simply the best that you can get given latency issues over transcontinental connections, interconnect speeds between various ISPs, etc.

 

I wonder why no one has ever written a GUI for lftp?  lftp is still being actively developed as there have been three updates in 2017.  It shouldn't be hard to take a GUI written for another ftp program, or even another file manager, and just change the cli commands to be lftp specific, while adding in a few options such as the number of segments to use when doing a pget in lftp.

As long as Apache can read/serve the directory the files are in, you should be good to go.  Just make sure it's using HTTPS and it requires authentication.  No need to get fancy with a real cert unless you really want to.

 

Yeah, I know IDM on Windows kinda defeats the purpose of Unraid-based dockerized apps, but after trying a LOT of download managers and FTP clients to deal with European peering, IDM outperforms everything else I've tried.  It's a dedicated server I'm hitting, so I run 32 threads against it and get pretty decent throughput without worrying about causing issues for the neighbors.  If your box is a shared one, may want to be a little more kind.

 

Can't really help you with the lack of a GUI for lftp, though it wouldn't be terribly hard to write a basic one I would think.  Personally, I don't mind CLI-based tools but I'm old and used to suffering.  

 

One other thing, speaking of peering...you may want to try some other providers to see if you can get better peering.  I've been through a few different shared and dedicated providers in NL, FR, and DE, and they definitely have different speeds to me.  Time of day seems to make a difference as well.