[Support] ich777 - Application Dockers

[ich777]

7 hours ago, Ricky4K said:

First of all thanks for the response and solutions. 

Before overthinking it, most Routers or even Access Points do support QoS and with that you can prioritize bandwidth, so to speak you could give the Prefill container (as long as you assign it it's own IP on br0) a really low priority and therefore other clients are prioritized higher.

 

BTW how often do you let prefill run? I don't recommend running it more than once a day since if one client pulls updates they will also end up in LANCache, maybe at best at night or in the early morning hours.

[Ricky4K]

4 minutes ago, ich777 said:

Before overthinking it, most Routers or even Access Points do support QoS and with that you can prioritize bandwidth, so to speak you could give the Prefill container (as long as you assign it it's own IP on br0) a really low priority and therefore other clients are prioritized higher.

 

BTW how often do you let prefill run? I don't recommend running it more than once a day since if one client pulls updates they will also end up in LANCache, maybe at best at night or in the early morning hours.

Well as unfortunate as it sounds, my Router does not have any config options what so ever that's why I'll need to get another Router which you can at least configure such things. I even tried to get some custom Firmware onto the Router but without any chance.

 

For your question about the daily usage of the prefill: Currently I am using it all the time to fill up the cache with all the games as fast as possible, but I am planning to set up a schedule to only do the downloads during the night once they have been initially downloaded. IMO it would still be better to be able to limit the traffic no matter how often you are using the prefill.

 

[ich777]

25 minutes ago, Ricky4K said:

IMO it would still be better to be able to limit the traffic no matter how often you are using the prefill.

Here is also an issue from Steam Prefill itself from another user who was requesting that feature and the simple answer is that Prefill is not designed to support that:

https://github.com/tpill90/steam-lancache-prefill/issues/104#issuecomment-1258340657

 

TBH this is the first request that I got about a bandwidth limit and that doesn't make any sense to me since your firewall/router should handle sharing bandwidth automatically, even if I saturate my Internet connection, all of my devices still have Internet and are able to stream from Youtube/Amazon/Netfilx/...

Maybe your issue is also caused because of the WLAN connection to your server because WLAN (at least no up until 6E MIMO) to handle big amounts of data through multiple devices and you run into bottlenecks.

 

Sorry but you have to request that from LANCache Prefill since the container is working perfectly fine, but as from the GitHub Issue from above that is not possible.

 

BTW Running prefill during the day and multiple times is not the intended use case anyways...

[Ricky4K]

15 minutes ago, ich777 said:

Here is also an issue from Steam Prefill itself from another user who was requesting that feature and the simple answer is that Prefill is not designed to support that:

https://github.com/tpill90/steam-lancache-prefill/issues/104#issuecomment-1258340657

 

TBH this is the first request that I got about a bandwidth limit and that doesn't make any sense to me since your firewall/router should handle sharing bandwidth automatically, even if I saturate my Internet connection, all of my devices still have Internet and are able to stream from Youtube/Amazon/Netfilx/...

Maybe your issue is also caused because of the WLAN connection to your server because WLAN (at least no up until 6E MIMO) to handle big amounts of data through multiple devices and you run into bottlenecks.

 

Sorry but you have to request that from LANCache Prefill since the container is working perfectly fine, but as from the GitHub Issue from above that is not possible.

 

BTW Running prefill during the day and multiple times is not the intended use case anyways...

Well the main reason why it does not work properly in terms of bandwidth distribution is that my Router does not support some features e.g. QoS. This is why I desperately need to get some kind of bandwidth limiter.

[ich777]

29 minutes ago, Ricky4K said:

Well the main reason why it does not work properly in terms of bandwidth distribution is that my Router does not support some features e.g. QoS. This is why I desperately need to get some kind of bandwidth limiter.

I think you missed my point here, there is more to it that simply that, even a good router without QoS should handle bandwidth distribution well enough that every client has access to the Internet (including streaming) <- my OPNsense box for example limits traffic to the Prefill automatically without having to set up QoS at all and ensures that all clients have enough bandwidth.
Also don't forget that WLAN is not capable for such huge loads (as said above until 6E with MIMO) since such huge traffic will affect other devices heavily because each transaction on WLAN has to actually go through for one client and then the next client is allowed to initiate a traffic request, so to speak only one WLAN client can speak and wait for the answer at a time <- this is a really oversimplified explanation.

 

However as said above, the container itself is working perfectly fine and you have to create an issue on the Prefill GitHub.

 

EDIT: I now read your other thread and `tc` is also not the solution to this, you would solve most of your issues by connecting the server directly to your router with a reliable connection and with Network equipment that is up to the task.

 

You could of course always use the official container if you are not satisfied with my container.

 

Hope that helps.

[ich777]

54 minutes ago, Ricky4K said:

QoS

Oh, just because I thought of it, you can install Prefill in a LXC container and set it up yourself and try out `tc` if it fit your needs but in most cases it won't fully solve the issue.

[DeadPixel]

On 7/2/2024 at 7:02 AM, ich777 said:

Is this a new installation or an already existing one?

 

The container is running but you have to exclude:

<module name="ssl_gnutls">

and

<module name="userip">

 

so that it properly starts but that is part of the configuration process, I will might release an update for new installation so that this is fixed by default. <- this is now fixed, if you remove the container and the directory for InspIRCd and reinstall it it will work right OOB.

Super, thank you for sorting that

[Merijeek]

@ich777 - Is it possible using your OpenVPN container to run a script after the connection is made? 

 

For the sake of one tracker, I need to run this after the tunnel is up:

(example, obviously)

 

/path/docker/persists/tun_up.sh

#!/bin/bash
curl -c /path/docker/persists/mam.cookies -b /path/docker/persists/mam.cookies https://t.myanonamouse.net/json/dynamicSeedbox.php

[ich777]

32 minutes ago, Merijeek said:

@ich777 - Is it possible using your OpenVPN container to run a script after the connection is made? 

Nope but I can think of a solution.

 

Since you can mount a script to /opt/scripts/user.sh which will run before the start of the VPN connection you could do something like that:

#!/bin/bash
cookies() {
  sleep 20
  curl -c /path/docker/persists/mam.cookies -b /path/docker/persists/mam.cookies https://t.myanonamouse.net/json/dynamicSeedbox.php
}

cookies &

 

What this change does is:

1. Put the curl command in a function
2. Add 20 seconds of sleep to the function before the curl command is executed
3. Start function "cookies" and run it in the background

 

With that the function will sleep 20 seconds and execute the curl command (hopefully) after the VPN connection is made.

 

I hope that makes sense.

[Merijeek]

It does. Thanks, I'll give it a shot. 

[Merijeek]

(actually I lied)

 

I understand what you're saying as a concept, but not how to implement it. 

 

Maybe it's my fundamental lack of Docker understanding, well, almost certainly, but let's break it down Barney Style

 

I should add a path in my container setup so that the path  /opt/scripts/user.sh maps to the actual script on my server.

 

So....like this?

 

 

Where mam.sh contains the block of scripting you sent me? 

 

 

[ich777]

Just now, Merijeek said:

So....like this?

Yes.

 

Just now, Merijeek said:

Where mam.sh contains the block of scripting you sent me? 

Yes.

[AlexON]

Hi there!

 

Small question regarding the Ubuntu APT Mirror docker. You say to configure something in

cd ubuntu-mirror/config/mirror.list

 

but I don't see what to change there. The only way I noticed to use it was the

apt-mirror

command. It seems to download all Ubuntu packages (around 390 GB). And the end of the download. On the container web page, I don't see any File tree. And no way to make it use my server instead of the internet for updates.

 

And I would really like to know if it is possible to put 2 Ubuntu sources? That is to say version 22.04 LTS and version 24.04 LTS. Thank you so much!

Edited July 5 by AlexON
add

[GTvert90]

I'm getting my butt kicked with openvpn-client I download and rename my .ovpn from my provider and place it in the appdata folder and restart the terminal / log for openvpn-client will no longer stay open to view of it is connected or if there is another issues 

It opens fine until my vpn.ovpn file is put in the appdata folder. Any ideas? 

[knowledge-raider-82]

I'm also trying to curl a specific tracker from within the container, but when I docker exec into the VPN container and run the curl it fails, citing either "Couldn't connect to server" or "Timeout was reached". Curling the same sites from outside container succeeds normally. Additionally, some other sites can be curled from within the VPN container with no issues at all (like example.com). I need the request to go through the VPN container so that the tracker can see what IP my requests will be coming from.

[ich777]

13 hours ago, AlexON said:

mirror.list

What is in this file on your instance?

 

13 hours ago, AlexON said:

apt-mirror

Don't do anything manual, you will most likely break how the container is working fundamentally and since you don't see anything in the web server directory you already did.

 

13 hours ago, AlexON said:

And I would really like to know if it is possible to put 2 Ubuntu sources?

Sure that's also possible you have to properly configure mirror.list from above, you even can configure it to pull down multiple architectures.

 

13 hours ago, AlexON said:

And no way to make it use my server instead of the internet for updates.

You have to point to your apt-mirror on your Ubuntu installations, even better would be to use a revers proxy to make it work properly.

[ich777]

9 hours ago, GTvert90 said:

It opens fine until my vpn.ovpn file is put in the appdata folder. Any ideas? 

Are you using NordVPN? If yes, it seems they changed something lately that prevents it from working properly.

I can confirm that PIA, Privado and hideme is working as it should.

 

9 hours ago, GTvert90 said:

terminal / log

Stop the container, after that open the logs and see what the actual issue is.

 

I think it would be also okay to upload your .ovpn file here since if the credentials are missing nobody will be able to connect.

[ich777]

9 hours ago, knowledge-raider-82 said:

I'm also trying to curl a specific tracker from within the container, but when I docker exec into the VPN container and run the curl it fails, citing either "Couldn't connect to server" or "Timeout was reached". Curling the same sites from outside container succeeds normally. Additionally, some other sites can be curled from within the VPN container with no issues at all (like example.com). I need the request to go through the VPN container so that the tracker can see what IP my requests will be coming from.

Also trying? What user tried that?

 

If that fails from within the container it has most likely to do with your VPN provider since the container on it's own blocks nothing as long as it goes through the VPN connections.

[AlexON]

9 hours ago, ich777 said:

What is in this file on your instance?

############# config ##################
#
set base_path         /ubuntu-mirror/data
#
#set mirror_path       $base_path/mirror
#set skel_path         $base_path/skel
#set var_path          $base_path/var
#set postmirror_script $var_path/postmirror.sh
#set defaultarch       i386
#set run_postmirror    0
set nthreads          20
#set limit_rate        100m
set _tilde            0
# Use --unlink with wget (for use with hardlinked directories)
#set unlink            1
#set use_proxy         off
#set http_proxy        127.0.0.1:3128
#set proxy_user        user
#set proxy_password    password
#
############# end config ##############

deb http://archive.ubuntu.com/ubuntu jammy main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu jammy-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu jammy-updates main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse

#deb-src http://archive.ubuntu.com/ubuntu jammy main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu jammy-security main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu jammy-updates main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse

clean http://archive.ubuntu.com/ubuntu

 

I'm not sure I understand but as you see, my file is original. But even if I make a modification, I still get this error message.

 

Session terminated, killing shell... ...killed.

Session terminated, killing shell... ...killed.

Session terminated, killing shell... ...killed.
---Ensuring UID: 99 matches user---
---Ensuring GID: 100 matches user---
---Setting umask to 0000---
---Checking for optional scripts---
---No optional script found, continuing---
---Starting cron---
---Starting apache2---
---Taking ownership of data...---
---Starting...---
---Please edit your 'mirror.list' file in your conig directory and restart the container when done!---
---Ensuring UID: 99 matches user---
usermod: no changes
---Ensuring GID: 100 matches user---
usermod: no changes
---Setting umask to 0000---
---Checking for optional scripts---
---No optional script found, continuing---
---Starting cron---
---Starting apache2---
---Taking ownership of data...---
---Starting...---
---Starting first mirror---
Downloading 152 index files using 20 threads...
Begin time: Fri Jul  5 09:51:49 2024
[20]... [19]... [18]... [17]... [16]... [15]... [14]... [13]... [12]... [11]... [10]... [9]... [8]... [7]... [6]... [5]... [4]... [3]... [2]... ---Ensuring UID: 99 matches user---
usermod: no changes
---Ensuring GID: 100 matches user---
usermod: no changes
---Setting umask to 0000---
---Checking for optional scripts---
---No optional script found, continuing---
---Starting cron---
---Starting apache2---
---Taking ownership of data...---
---Starting...---
---Something went horribly wrong, can't find the mirror directory!---
---Ensuring UID: 99 matches user---
usermod: no changes
---Ensuring GID: 100 matches user---
usermod: no changes
---Setting umask to 0000---
---Checking for optional scripts---
---No optional script found, continuing---
---Starting cron---
---Starting apache2---
---Taking ownership of data...---
---Starting...---

 

Edited July 6 by AlexON

[ich777]

27 minutes ago, AlexON said:

But even if I make a modification, I still get this error message.

How does the folder structure look like?

[AlexON]

Same, I'm not sure what you want to see. But everything is original.

[ich777]

30 minutes ago, AlexON said:

Same, I'm not sure what you want to see. But everything is original.

I mean the folder structure from the container and what folders that it created.

 

I have to say when you don‘t get it to work I will deprecate the container, I have no use case for it because I prefer Debian over Ubuntu

I did this container on user request and it was working for a few users from what I heard back from them.

[Merijeek]

Alright, so I'm back into the connected containers rodeo. 

 

I've been having various (other) issues so haven't been working on this one. This morning I find my OpenVPN up 7 hours, Sonarr up 12, and Sonarr is inaccessible. 

 

From the OPVN setup:

 

 

...and from the Sonarr setup:

 

 

Any thoughts on what I could have wrong here?

[AlexON]

@ich777

I'm quite the opposite, I prefer Ubuntu for its ease of use. Debian frustrates me despite sharing the same source code.

Edited July 6 by AlexON

[ich777]

1 hour ago, AlexON said:

And what is in data?

 

Again, if you can't get it to run I will deprecate the container since I have no use for it and I'll take that as a not working report.