Stark

If I recall, the progress bar is per file in the directory. Not ideal I know but I think that explains it. Davos was designed more as a “set it and leave it” app so the progress bar was a bit of an afterthought.

What version of MySQL are you using? Is there anything in the logs?

The logs suggest that you have multiple schedules running with the same filters/target directory. I could be mistaken as I’ve only scanned through the log. Also it seems that it’s not comparing the files from the last scan to find new files (it thinks it’s a first-time scan each time). It may be worth checking the scan list history to check it’s actually saving that reference after each run.

Hi, Looking at the logs it seems that it did initially pick up the files based on the filter, but there was an error when attempting to download one of the metadata files. By default davos will try to continue with the workflow but will register all attempted files as having been actioned, so following runs of the schedule will ignore the files. That's why it appears the filter isn't working (it is, it's just ignoring them because it thinks it's seen them before). For some reason davos is unable to create the local directory for some sub folders when recursively downloading the files. If you look at line 1109 onwards in the logs you provided, you'll see the schedule picks up the files and then goes over each of the matching directories. In order to download the directories, it needs to traverse them over FTP and then generate them locally by manually creating them (you can't transfer directories over FTP using the Java libraries I use). However it seems that is is unable to create these directories locally "2019-11-29 08:51:54.107 - DEBUG - [FileUtils] - Directory was not created!". Check your directory permissions for /downloads/Downloads and make sure that is is a) a valid directory, and b) owned by the correct user. If you want to test the schedule out again, you can click on the history icon in the schedule summary and clear the list. This will force davos to fully match everything again.

The downloaded files should take the ownership of the process owner so it could be you're running davos under the wrong PUID/PGID. That said I can't remember what permissions it sets on the files after downloading them. Double check your run configuration for the container so that the PUID and PGID match the user you want to take ownership of the files.

@sickb0y It appears that there was a dependency change recently with OpenJRE which meant nss was no longer packaged up in its dependency tree. It has been added in manually now as part of the image build process. Hopefully this will fix your issue.

That's just the app telling you that you've tried navigating to a page that doesn't exist. I don't believe I created an endpoint for /error, so it's falling back to that "white label" page.

OK, we've figured out what went wrong. During the migration phase of our pipeline, the build command omitted a flag which set it to use production configuration. The result of this is that the latest image was build with a "staging" environment in mind and so had its database set in memory, not from a file. A new build has been pushed through the pipeline, so can you pull latest and check? You may find that now it looks for the database file correctly, it will pick up your old existing database, so will be a bit stale. Long story short: we dropped the ball a bit with the build, and I can only apologise for that.

@manofcolombia Sorry mate it's been a hectic week. I'll try and have a look at the latest tag in the next couple of days. In the meantime, switch back from ":latest" and use ":2.2.1" instead. That was the version of the app before we migrated our pipeline. Can you see if this helps; if so we know it's an issue with the latest build of the Docker image.

@drumstyx at this point if you're putting effort into using rclone instead, just keep using that. I'm afraid I don't have any more free time to give to this project as I recently took a new job which has effectively exhausted me. Parallel downloads won't be something I personally implement. @manofcolombia nothing has changed regarding the actual application but I will check to see if anything has changed regarding the image dependencies.

Hi @drumstyx, when you say you're getting a 24Mbit/s transfer speed, are you referring to actual speeds? I have a 74Mbit/s line, and when using davos, got ~6.5MB/s - which isn't too far off 80-90% of my available bandwidth. Admittedly I've not run the application on a faster line (they're just not available to me where I live). I appreciate that the slowness for some people is annoying, but my implementation of the FTP and SFTP libraries is incredibly basic (practically no internal library tweaking), as the davos application was initially designed as a personal project that ended up gaining traction with a decent number of people. Unfortunately due to other commitments, I don't have the spare time to run diagnostics to understand where the bottlenecks may be. Honestly, it could be one of any number of things that's causing your slow-downs: SSL handshaking delays, JVM heap space utilisation, system property interference etc. Hell, even the hardware could be a factor. I have to admit, the most likely culprit is quite likely my implementation of the libraries, for which I can only apologise.

Hi guys, sorry for the late reply. I have to admit this isn't something I've noticed. I haven't released to davos since 1st Oct 2017 so any differences in performance would be either externally instigated, or perhaps via the Docker image (although I can't see this being the cause). The best I can offer in terms of help is the suggestion to keep monitoring it and seeing how it goes. My implementation of the connection speed can be a bit flaky for SFTP connections so it may be that while it shows a slow speed, it's not necessarily that slow. Are you able to monitor the transfer speed using ntop or likewise? Thanks.

Hi @sinbrkatetete, there is currently no support for connecting to a host via proxies (SOCKS or HTTP). If this is something you would like to have, please add a request as an Issue in GitHub (https://github.com/linuxserver/davos/issues), with your requirements so I can see if this is something I can add in to a future version. Cheers!

If for example your host machine has the following directory structure: / home/ downloads/ tmp/ other/ And the docker container has the following internal structure: / download/ tmp/ app/ When you map your directories during the creation process ("-v /home/downloads:/download"), you do this: / / home/ downloads/ <-------------------> download/ tmp/ tmp/ other/ app/ That link (or, mapping) effectively means your "/home/downloads" directory is the "/download" directory inside the container. They get overlaid, meaning they are effectively now the same directory. This is an important concept to understand because any files placed in the "/download" directory are immediately visible to the host, via "/home/downloads", and vice versa. In order to access the files (or in this case, download them) you must specify "/download" so the files are visible in your "/home/downloads" directory. If you specify root "/", all that will happen is files will be downloaded internally in the container (and will fail due to permission errors). What you are describing is a fundamental misunderstanding of how docker overlays its directories when you map them. By mapping these directories together, you are not saying that the internal root directory ("/") is now your "/home/downloads" folder. If you want, feel free to map root "/" and see what happens (I definitely do not recommend this though). In the case of Davos, no the download location does not imply "/download" because it was designed to be flexible when running outside of docker. The path is ABSOLUTE, meaning you must specify the whole path, from root. If you do not do this, it will attempt to find the path relative to where the application is running, which is the "/app" directory. TL;DR: Use "/download" when specifying the local directory for davos to download to. It will not work with any other directory when running inside a container. It is by design, and will not change.

Hi @Nyghthawk, davos does not assume where your "download" folder is, as it was designed to be relatively flexible regarding which directories you wish for it to be aware of. Due to the way Docker containers work, it is important to remember that just by mapping /home/my/host/directory:/download you are not telling davos (the application) where you want downloaded files to go. All that does is tell the Docker container where you want files in the internal /download directory to be on your host. For example, if you decided to run davos natively on your machine, you'd tell it to download files to /mnt/usr/Seedbox because davos has no knowledge of your preferred download directory - you need to tell it explicitly. The same applies to running it within a Docker container. In this particular case, the Docker container has mapped the /download directory specifically for this use case, hence why you need to start your local directory with it. This is not something I plan on amending in future releases. Regarding your other feature request, please add it to the Issues list on the davos GitHub page. Cheers. Thank you for using davos! If you have any further queries, let me know.