[Support] ich777 - Application Dockers

[evakq8r]

3 minutes ago, ich777 said:

Prefill don't need it's own IP strictly speaking.

 

Did you set the "--dns=" parameter like mentioned in the description from the container?

 

True, however without the IP it defaults to DHCP, which was a lot lower in the IP range I have available, so kept them together.

 

Yes, I did:

 

 

5 minutes ago, ich777 said:

Do you need the lancache-dns at all or better speaking have you something like AdGuard or PiHole somewhere in your network? You can also use AdGuard, PiHole, Unbound or whatever as a DNS for lancache.

 

Based on Spaceinvader's tutorial video I linked earlier, yes it does seem to be needed. Happy to try an alternative method if you think one fits better. 

 

As for DNS, I use NextDNS (a variation of PiHole/Adguard etc - https://nextdns.io/)

 

6 minutes ago, ich777 said:

May I ask where you are located in the world?

 

Australia

 

6 minutes ago, ich777 said:

This could also be caused because of an unstable connection to the Steam CDN itself which maybe not be obvious while downloading through the Steam Client itself.

AFAIK many users get the message that it retries and it works after a short while again but that could be caused of many things.

Even I get the message sometimes but in my case that happens really occasionally.

 

Unfortunately for me it's quite often. Downloading direct from Steam CDNs outside of Lancache are always stable, so I find it odd for it to become unstable while using Lancache and stable outside of it.

 

7 minutes ago, ich777 said:

Do you have the data for your lancache on your Array? If yes, I would not recommend doing that it that way, rather I would recommend that you use a disk outside from your Array, maybe a spinning HDD or even better a SSD or NVME mounted through Unassigned Devices.

I could imagine that the prefill stresses your server too much and you maybe also get higher I/O Wait than usual when having the lancache on the Array.

 

I do, yes, and what you say makes sense. Unfortunately the server I have has all HDD bays populated and the NVMe adapter in use is on a PCIe bracket in the server (as the Poweredge does not support NVMe or M.2s natively). 

 

Interestingly enough, I was using Josh5's Lancache-Bundle just fine, but the prefill container could never communicate with the bundle for whatever reason. The only reason I spun up the separate containers was because prefill did work, but then had these issues instead. In addition, the lancache-bundle is also using an array share and hasn't crushed I/O at all.

[ich777]

7 minutes ago, evakq8r said:

Based on Spaceinvader's tutorial video I linked earlier, yes it does seem to be needed.

No, it isn't I only use a combination of Unbound, AdGuard and Lancache-Monolithic

I simply rewrite the URLs with the lancache Monolithic IP in my local network, in PiHole you would use something like DNSMASQ.

 

But my set up is a little different from most people ones because I use everything in a LXC container on Unraid where the applications talk to each other in the container and have only one exposed IP which is used then on my local network as DNS <- this is also doable with Docker containers on Unraid itself, I really like over complicating things for me...

 

14 minutes ago, evakq8r said:

Unfortunately for me it's quite often. Downloading direct from Steam CDNs outside of Lancache are always stable, so I find it odd for it to become unstable while using Lancache and stable outside of it.

Also when downloading from the lancache or also when filling the cache?

If this is happening also while downloading from the cache then there must be something going on on your network...

 

19 minutes ago, evakq8r said:

Interestingly enough, I was using Josh5's Lancache-Bundle just fine

Didn't he say that he want to deprecate it or am I wrong about that?

 

20 minutes ago, evakq8r said:

but the prefill container could never communicate with the bundle for whatever reason.

Which prefill? Did it work on your computer?

 

21 minutes ago, evakq8r said:

In addition, the lancache-bundle is also using an array share and hasn't crushed I/O at all.

Please keep in mind that this was only a guess and I really don't know what is going on over there.

I only can tell you that I run my lancache from a spinning HDD from UD and it doesn't slow down the server at all, of course I have heavy I/O wait because my HDD is simply not fast enough for my 10Gbit/s connection but that is a common issue.

Did @Josh.5 maybe use a different chunk size?

 

 

But keep in mind that this is only the support thread for the prefill container which seems like to me isn't the issue here at all and it rather seems that there is something wrong with lancache itself.

 

[evakq8r]

Welp, managed to get the prefill container to work with the lancache-bundle container (changed DNS back to what the bundle was and it just worked? (didn't before)).

 

Sounds like the standalone lancache containers are a bit broken (for my use case), and I do hope @Josh.5 doesn't depricate his container as it works wonders.

 

Either way, crisis averted. Thanks for responding.

[ich777]

9 minutes ago, evakq8r said:

Sounds like the standalone lancache containers are a bit broken (for my use case), and I do hope @Josh.5 doesn't depricate his container as it works wonders.

As far as I can tell something seems wrong with the configuration with you lancache containers, the official containers indeed work fine for me and also for a lot of other people that I know.

 

11 minutes ago, evakq8r said:

Either way, crisis averted. Thanks for responding.

Glad that it is working now for you.

[ss1gohan13]

On 12/20/2022 at 10:25 PM, ich777 said:

What do you mean exactly with the update command?

If I have cached all my games, but a game i no longer play comes out with an update, how do I update that game? Do I follow your notes in the template? Or is there a way I can automate game updates?

Edited December 23, 2022 by ss1gohan13

[ich777]

40 minutes ago, ss1gohan13 said:

Do I follow your notes in the template? Or is there a way I can automate game updates?

You can select particular games hut then you have to delete the parameter -recent from the template, by doing so it will only update the games that you‘ve selected.

 

There is also a parameter for pulling all games, but I would recommend that you read trough the official GitHub Readme from Prefill for more information and what parameters are available (linked in the template).

[slimshizn]

Ich77, have you had to do any performance changes on your setup for lancache? I guess I expected to see at least line speeds via ethernet, but not getting that. I do have the cache on a sas drive, moving it over to a NVMe ssd.

Edited December 23, 2022 by slimshizn
wat

[ich777]

1 hour ago, slimshizn said:

Ich77, have you had to do any performance changes on your setup for lancache? I guess I expected to see at least line speeds via ethernet, but not getting that. I do have the cache on a sas drive, moving it over to a NVMe ssd.

No, there is nothing I can do about the speeds since it is mostly limited to the Docker bridge network speed or other limiting factors like you've already mentioned drive speeds.

I have no issue in my setup reaching speeds about 800Mbit/s for a game that is already in the lancache in terms of prefill which is transferred from the cache (spinning HDD in my case).

In your case what is your line speed? In my case I got 250Mbit/s which is exactly my Fiber connections speed.

 

From my experience prefill is often not that fast as if you download from the cache to your physical computer.

[Enks]

I want to prefill my cache with two versions of the same game from Steam (Windows and Linux). Does the --os parameter accept more than one operating system? If yes, what is the correct way to set it: --os windows,linux or --os linux --os windows or even another way?

 

You have a tiny error in one of the descriptions in your template:

The SteamPrefill Schedule is by default set to: 0 2 * * *, which means 02:00. The description says:

 

"Set your cron schedule for SteamPrefill if enabled (by default it is set to 05:00 - head over to https://crontab.guru/ to create your own if you want to customize it)"

 

 

Edited December 23, 2022 by Enks

[ich777]

11 minutes ago, Enks said:

You have a tiny error in one of the descriptions in your template:

Thank you, fixed that.

 

13 minutes ago, Enks said:

I want to prefill my cache with two versions of the same game from Steam (Windows and Linux). Does the --os parameter accept more than one operating system? If yes, what is the correct way to set it: --os windows,linux or --os linux --os windows or even another way?

I'm assuming that it can be one or more like mentioned in the documentation, since it says "system(s)".

I think the right way would be like you've already pointed out:

--os windows,linux

 

BTW that's a specific thing from prefill itself and you can try it at least, I've already reached out to the developer about that.

[slimshizn]

5 hours ago, ich777 said:

No, there is nothing I can do about the speeds since it is mostly limited to the Docker bridge network speed or other limiting factors like you've already mentioned drive speeds.

I have no issue in my setup reaching speeds about 800Mbit/s for a game that is already in the lancache in terms of prefill which is transferred from the cache (spinning HDD in my case).

In your case what is your line speed? In my case I got 250Mbit/s which is exactly my Fiber connections speed.

 

From my experience prefill is often not that fast as if you download from the cache to your physical computer.

I must have something going wrong then, spinning HDD giving you 800Mbit/s, I should be able to get the same then. In terms of transfer speeds from server to PC and other devices, if it's ssd to ssd, I get expected speeds. For instance, I tested out a download from battle.net CoD Black Ops 4. It's about 160G download. Not on the cache, I get about 300Mbit/s, on the cache, 50Mbit/s. I did use prefill, and it verifies the download up to date for that game. I'll continue to test and move the cache to the NVMe.

[Kilrah]

Will probably depend on the game, some use few large packed files and some use lots of little ones - the latter will perform terribly on a spinning drive.

[tpill90]

On 12/23/2022 at 12:09 AM, ich777 said:

You can select particular games hut then you have to delete the parameter -recent from the template, by doing so it will only update the games that you‘ve selected.

 

There is also a parameter for pulling all games, but I would recommend that you read trough the official GitHub Readme from Prefill for more information and what parameters are available (linked in the template).

One thing to note here, you don't need to drop the --recent flag if you want to also use select-apps. You can do any combination of select-apps, --recent, --top, etc.  And it will prefill the combination of them all. 

[Enks]

21 hours ago, ich777 said:

BTW that's a specific thing from prefill itself and you can try it at least, I've already reached out to the developer about that.

i tried it with the game Aragami, which has a Windows and a Linux native version. After the completion of the prefill i booted Ubuntu and started downloading it. Download speed was about 5 MByte/s (speed of my internet connection) so prefill didn't work. After completing the download over the steam client I switched back to Windows and started the download of the game there. Speed was about 40 MByte/s (Lancache is on my array). Conclusion: Prefill only cached the Windows version of Aragami. Now after downloading it over the linux steam client both versions are cached (size of the cache-share increased with the download of the linux version).

 

@tpill90 Is it possible to use prefill to cache more versions of the same game, for example Windows and Linux? If yes, how?

Edited December 24, 2022 by Enks

[slimshizn]

Switched over the the NVMe SSD, and yeah, that was the issue on my end. Speeds are much better.

[ich777]

4 hours ago, Enks said:

Prefill only cached the Windows version of Aragami.

What do the logs say?

 

Try to separate it by a space instead of a comma.

[tpill90]

8 hours ago, Enks said:

i tried it with the game Aragami, which has a Windows and a Linux native version. After the completion of the prefill i booted Ubuntu and started downloading it. Download speed was about 5 MByte/s (speed of my internet connection) so prefill didn't work. After completing the download over the steam client I switched back to Windows and started the download of the game there. Speed was about 40 MByte/s (Lancache is on my array). Conclusion: Prefill only cached the Windows version of Aragami. Now after downloading it over the linux steam client both versions are cached (size of the cache-share increased with the download of the linux version).

 

@tpill90 Is it possible to use prefill to cache more versions of the same game, for example Windows and Linux? If yes, how?

Use `--os windows linux`

[Enks]

On 12/24/2022 at 7:26 PM, ich777 said:

Try to separate it by a space instead of a comma.

 

On 12/24/2022 at 11:41 PM, tpill90 said:

Use `--os windows linux`

Thanks, it works perfectly now

[BuTscH]

Greetings!
I have a question. I already use your firefox docker and am super happy with it.

Would it be possible for you to make such a docker with the Tor-Browser?

[ich777]

2 minutes ago, BuTscH said:

Would it be possible for you to make such a docker with the Tor-Browser?

Maybe but I‘m jot that damiliar with TOR.

I will look into that, can you point me to the exact browser (homepage) that you want to use?

 

I will look into that, but that could take some time.

[BuTscH]

Sure that would be this page here: https://www.torproject.org/de/download/
The Linux version.
No need to hurry it would just be nice not to always have to start a VM just to use the Tor-Browser 🙂

[aterfax]

On 11/26/2022 at 7:10 PM, aterfax said:

By default the apt mirroring seems to be missing the cnf directory. This caused my ubuntu machines to throw the 'Failed to fetch' error from my original post.

I added the cnf.sh linked from above to the config directory, edited it to support "jammy" then ran it from inside the running container to download the missing files.

Script as follows:
 

#!/bin/bash
cd /ubuntu-mirror/data/mirror/
for p in "${1:-jammy}"{,-{security,updates}}\
/{main,restricted,universe,multiverse};do >&2 echo "${p}"
wget -q -c -r -np -R "index.html*"\
 "http://archive.ubuntu.com/ubuntu/dists/${p}/cnf/Commands-amd64.xz"
wget -q -c -r -np -R "index.html*"\
 "http://archive.ubuntu.com/ubuntu/dists/${p}/cnf/Commands-i386.xz"
done

This resolved the 'Failed to fetch'  problem.

This could be added to the container in one way or another I suspect, but it does not appear to be something that needs to be ran regularly according to the link above.

Was this an issue for other users? Did this get resolved in the container?

[ich777]

1 hour ago, aterfax said:

Was this an issue for other users? Did this get resolved in the container?

One user reported it on GitHub but he stopped responding.

 

No, I haven't implemented this in the container since from my testing it wasn't necessary.

[Cicatrix]

Hey there, I have problems with Jdownloader2. It simply won't start. Can't see the log, the window will directly close again. Logs with not runing Jdownloader2:

WebSocket server settings:
  - Listen on :8080
  - Web server. Web root: /usr/share/novnc
  - No SSL/TLS support (no cert file)
  - Backgrounding (daemon)
/opt/scripts/start-server.sh: line 158: /jDownloader2/runtime/jre1.8.0_261: Is a directory
WebSocket server settings:
  - Listen on :8080
  - Web server. Web root: /usr/share/novnc
  - No SSL/TLS support (no cert file)
  - Backgrounding (daemon)
/opt/scripts/start-server.sh: line 158: /jDownloader2/runtime/jre1.8.0_261: Is a directory
WebSocket server settings:
  - Listen on :8080
  - Web server. Web root: /usr/share/novnc
  - No SSL/TLS support (no cert file)
  - Backgrounding (daemon)
/opt/scripts/start-server.sh: line 158: /jDownloader2/runtime/jre1.8.0_261: Is a directory
---Resolution check---
---Width to low must be a minimal of 1024 pixels, correcting to 1024...---
---Height to low must be a minimal of 768 pixels, correcting to 768...---
---Window resolution: 1024x768---
---Starting TurboVNC server---
---Starting Fluxbox---
---Starting noVNC server---
---Starting jDownloader2---
---Ensuring UID: 99 matches user---
usermod: no changes
---Ensuring GID: 100 matches user---
usermod: no changes
---Setting umask to 000---
---Checking for optional scripts---
---No optional script found, continuing---
---Checking configuration for noVNC---
Nothing to do, noVNC resizing set to default
Nothing to do, noVNC qaulity set to default
Nothing to do, noVNC compression set to default
---Taking ownership of data...---
---Starting...---
---Checking for 'runtime' folder---
---'runtime' folder found---
---Checking if Runtime is installed---
---Runtime found---
---Checking for 'jDownloader.jar'---
---'jDownloader.jar' folder found---
---Preparing Server---
---Checking libraries---
---Libraries found!---
---Checking for old logfiles---
---Checking for old display lock files---
---Resolution check---
---Width to low must be a minimal of 1024 pixels, correcting to 1024...---
---Height to low must be a minimal of 768 pixels, correcting to 768...---

 

But Unraid shows me the following:

 

Dec 29 01:16:34 Tower kernel: device vethe0b3834 entered promiscuous mode
Dec 29 01:16:34 Tower kernel: eth0: renamed from veth6558272
Dec 29 01:16:34 Tower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethe0b3834: link becomes ready
Dec 29 01:16:34 Tower kernel: docker0: port 2(vethe0b3834) entered blocking state
Dec 29 01:16:34 Tower kernel: docker0: port 2(vethe0b3834) entered forwarding state
Dec 29 01:16:36 Tower  avahi-daemon[4953]: Joining mDNS multicast group on interface vethe0b3834.IPv6 with address fe80::4418:bdff:fe4d:4494.
Dec 29 01:16:36 Tower  avahi-daemon[4953]: New relevant interface vethe0b3834.IPv6 for mDNS.
Dec 29 01:16:36 Tower  avahi-daemon[4953]: Registering new address record for fe80::XXXX:XXXX:XXXX:XXXX on vethe0b3834.*.
Dec 29 01:16:42 Tower kernel: docker0: port 2(vethe0b3834) entered disabled state
Dec 29 01:16:42 Tower kernel: veth6558272: renamed from eth0
Dec 29 01:16:42 Tower  avahi-daemon[4953]: Interface vethe0b3834.IPv6 no longer relevant for mDNS.
Dec 29 01:16:42 Tower  avahi-daemon[4953]: Leaving mDNS multicast group on interface vethe0b3834.IPv6 with address fe80::4418:bdff:fe4d:4494.
Dec 29 01:16:42 Tower kernel: docker0: port 2(vethe0b3834) entered disabled state
Dec 29 01:16:42 Tower kernel: device vethe0b3834 left promiscuous mode
Dec 29 01:16:42 Tower kernel: docker0: port 2(vethe0b3834) entered disabled state
Dec 29 01:16:42 Tower  avahi-daemon[4953]: Withdrawing address record for fe80::XXXX:XXXX:XXXX:XXXX on vethe0b3834.
Dec 29 01:17:42 Tower kernel: docker0: port 2(veth7b15146) entered blocking state
Dec 29 01:17:42 Tower kernel: docker0: port 2(veth7b15146) entered disabled state
Dec 29 01:17:42 Tower kernel: device veth7b15146 entered promiscuous mode
Dec 29 01:17:42 Tower kernel: docker0: port 2(veth7b15146) entered blocking state
Dec 29 01:17:42 Tower kernel: docker0: port 2(veth7b15146) entered forwarding state
Dec 29 01:17:42 Tower kernel: docker0: port 2(veth7b15146) entered disabled state
Dec 29 01:17:42 Tower kernel: eth0: renamed from vethe0b5b5d
Dec 29 01:17:42 Tower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth7b15146: link becomes ready
Dec 29 01:17:42 Tower kernel: docker0: port 2(veth7b15146) entered blocking state
Dec 29 01:17:42 Tower kernel: docker0: port 2(veth7b15146) entered forwarding state
Dec 29 01:17:43 Tower  avahi-daemon[4953]: Joining mDNS multicast group on interface veth7b15146.IPv6 with address fe80::acde:b9ff:feda:ede6.
Dec 29 01:17:43 Tower  avahi-daemon[4953]: New relevant interface veth7b15146.IPv6 for mDNS.
Dec 29 01:17:43 Tower  avahi-daemon[4953]: Registering new address record for fe80::XXXX:XXXX:XXXX:XXXX on veth7b15146.*.
Dec 29 01:17:49 Tower kernel: docker0: port 2(veth7b15146) entered disabled state
Dec 29 01:17:49 Tower kernel: vethe0b5b5d: renamed from eth0
Dec 29 01:17:49 Tower  avahi-daemon[4953]: Interface veth7b15146.IPv6 no longer relevant for mDNS.
Dec 29 01:17:49 Tower  avahi-daemon[4953]: Leaving mDNS multicast group on interface veth7b15146.IPv6 with address fe80::acde:b9ff:feda:ede6.
Dec 29 01:17:49 Tower kernel: docker0: port 2(veth7b15146) entered disabled state
Dec 29 01:17:49 Tower kernel: device veth7b15146 left promiscuous mode
Dec 29 01:17:49 Tower kernel: docker0: port 2(veth7b15146) entered disabled state
Dec 29 01:17:49 Tower  avahi-daemon[4953]: Withdrawing address record for fe80::XXXX:XXXX:XXXX:XXXX on veth7b15146.
Dec 29 01:18:49 Tower kernel: docker0: port 2(veth2eec858) entered blocking state
Dec 29 01:18:49 Tower kernel: docker0: port 2(veth2eec858) entered disabled state
Dec 29 01:18:49 Tower kernel: device veth2eec858 entered promiscuous mode
Dec 29 01:18:49 Tower kernel: eth0: renamed from veth30656df
Dec 29 01:18:49 Tower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth2eec858: link becomes ready
Dec 29 01:18:49 Tower kernel: docker0: port 2(veth2eec858) entered blocking state
Dec 29 01:18:49 Tower kernel: docker0: port 2(veth2eec858) entered forwarding state
Dec 29 01:18:50 Tower  avahi-daemon[4953]: Joining mDNS multicast group on interface veth2eec858.IPv6 with address fe80::b815:25ff:fe9e:1a09.
Dec 29 01:18:50 Tower  avahi-daemon[4953]: New relevant interface veth2eec858.IPv6 for mDNS.
Dec 29 01:18:50 Tower  avahi-daemon[4953]: Registering new address record for fe80::XXXX:XXXX:XXXX:XXXX on veth2eec858.*.

This behavior stops when closing Jdownloader2. Can someone help pls?

Edited December 29, 2022 by Cicatrix

[ich777]

8 hours ago, Cicatrix said:

Hey there, I have problems with Jdownloader2. It simply won't start. Can't see the log, the window will directly close again. Logs with not runing Jdownloader2:

Something went really wrong with your installation.

 

I would recommend that you remove the jDownloader2 container, then delete the jdownloader2 folder that lives in your appdata directory and pull a fresh copy from the CA App and wait for it to fully start.

 

The first start can take some time so please be patient and wait for it to start.