[Support] ich777 - Application Dockers

MiniMoe · November 30, 2022

1 minute ago, ich777 said:

I don't usually force a sync or do a manual sync because it just works in my case and let cron do it's job.

The logs are only written if a cron job is run or you force the sync on container start.

I've set a schedule for 1 minutes time from now, so I'll see what happens soon..

MiniMoe · November 30, 2022

8 minutes ago, ich777 said:

I don't usually force a sync or do a manual sync because it just works in my case and let cron do it's job.

The logs are only written if a cron job is run or you force the sync on container start.

Set the job, here's the output.

---Checking if UID: 99 matches user---
---Checking if GID: 100 matches user---
---Setting umask to 000---
---Checking for optional scripts---
---No optional script found, continuing---
---Starting cron---
---Taking ownership of data...---
---Starting...---
---SteamPrefill enabled!---
---Version Check!---
---SteamPrefill v1.8.1 up-to-date---
---Prepare Server---

Your cron schedule for SteamPrefill is: 50 19 * * *

---Container fully started, waiting for next cron job to start!---

[2022-11-30] Starting SteamPrefill
[7:50:02 PM] Starting login!
Connecting to Steam...
Logging into Steam...
[7:50:04 PM] Logged into Steam
Saving Steam session...
Retrieving owned apps...
[7:50:04 PM] Steam session initialization complete!
Retrieving latest App metadata...

[7:50:06 PM] Starting Call of Duty: Black Ops III
 Getting available CDN Servers... 
Fetching depot manifests...
   An unexpected error (System.Threading.Tasks.TaskCanceledException) occurred 
while downloading manifests.  Retrying...
[7:50:52 PM] Downloading 140.52 GiB
Detecting Lancache server...
[7:50:52 PM] Detected Lancache server at lancache.steamcontent.com [
192.168.0.249]
Downloading..: 0%
Downloading..: 0%
Downloading..: 0%
Downloading..: 1%
Downloading..: 1%
Downloading..: 2%
Downloading..: 2%
Downloading..: 3%

The cache directory it still empty in my lancache share though, so it looks like the same issue is occurring when scheduled.

ich777 · November 30, 2022

4 minutes ago, dGG2uA said:

The cache directory it still empty in my lancache share though, so it looks like the same issue is occurring when scheduled.

Wait, what cache directory, from Prefill?

The prefill container fills your LANCache Monolithic container over night or whatever cron schedule you have set...

MiniMoe · November 30, 2022

3 minutes ago, ich777 said:

Wait, what cache directory from Prefill?

The prefill container fills your LANCache Monolithic container over night or whatever cron schedule you set...

The share I set the data to cache to, when running lancache manually (without pre-fill, caching from manually downloading a steam game on my PC) - it does not populate with anything when running the pre-fill.

Edited November 30, 2022 by dGG2uA

ich777 · November 30, 2022

Just now, dGG2uA said:

The share I set the data to cache to, when running lancache manually (without pre-fill, caching from manually downloading a steam game on my PC) it does not populate with anything when running the pre-fill.

What does the access.log say from your LANCache Monolithic container when the prefill is running and displaying that it is downloading?

MiniMoe · November 30, 2022

14 minutes ago, ich777 said:

What does the access.log say from your LANCache Monolithic container when the prefill is running and displaying that it is downloading?

Trying to check, having a weird problem now - I've cleared out the lancache directory to start fresh, I started the Monolothic container to find the empty log files:

root@Rocinante:/mnt/user/lancache/logs# ls -la
total 0
drwxr-xr-x 1 nobody users 140 Nov 30 20:10 ./
drwxrwx--- 1 nobody users 115 Nov 30 20:10 ../
-rw-r--r-- 1 nobody users   0 Nov 30 20:10 access.log
-rw-r--r-- 1 nobody users   0 Nov 30 20:10 error.log
-rw-r--r-- 1 nobody users   0 Nov 30 20:10 stream-access.log
-rw-r--r-- 1 nobody users   0 Nov 30 20:10 stream-error.log
-rw-r--r-- 1 nobody users   0 Nov 30 20:10 upstream-error.log

But AFTER starting the pre-fill container, the directory is full of fuse_hidden files:

I think it's a non issue relating to this.

root@Rocinante:/mnt/user/lancache/logs# ls -la
total 0
drwxrwx--- 1 nobody users 210 Nov 30 20:10 ./
drwxrwx--- 1 nobody users 142 Nov 30 20:10 ../
-rwxrwx--- 0 nobody users   0 Nov 30 20:10 .fuse_hidden0085c2bf0000de5a*
-rwxrwx--- 0 nobody users   0 Nov 30 20:10 .fuse_hidden0085c2c00000de5b*
-rwxrwx--- 0 nobody users   0 Nov 30 20:10 .fuse_hidden0085c2c10000de5e*
-rwxrwx--- 0 nobody users   0 Nov 30 20:10 .fuse_hidden0085c2c20000de5c*
-rwxrwx--- 0 nobody users   0 Nov 30 20:10 .fuse_hidden0085c2c30000de5d*

I can cat the .fuse_hidden file to get this, which I think is from the access log;

2022/11/30 20:14:16 [crit] 1812#1812: *7354 open() "/data/cache/cache/10/21/20b396b0e7a89c7b3b3978049b8f2110.0000001836" failed (13: Permission denied) while reading upstream, client: 192.168.0.246, server: , request: "GET /depot/1/chunk/2799e572ee6e8c0bd2c35f3a01678b71ec33cd52 HTTP/1.1", upstream: "http://127.0.0.1:3128/depot/1/chunk/2799e572ee6e8c0bd2c35f3a01678b71ec33cd52", host: "cache9-iad1.steamcontent.com"
2022/11/30 20:14:16 [crit] 1830#1830: *7366 open() "/data/cache/cache/29/aa/30dcaf42f506b11a1bf853a0b2d1aa29.0000001837" failed (13: Permission denied) while reading upstream, client: 192.168.0.246, server: , request: "GET /depot/1/chunk/14366cf2e0aace27ca10a6fdc71a42c18c83c7cb HTTP/1.1", upstream: "http://127.0.0.1:3128/depot/1/chunk/14366cf2e0aace27ca10a6fdc71a42c18c83c7cb", host: "cache9-iad1.steamcontent.com"
2022/11/30 20:14:16 [crit] 1813#1813: *7358 open() "/data/cache/cache/04/5f/58ec91e19b4ecb1611396315bbbc5f04.0000001838" failed (13: Permission denied) while reading upstream, client: 192.168.0.246, server: , request: "GET /depot/1/chunk/c4d06de785ad7e026da3688189d4e5cfcc18c618 HTTP/1.1", upstream: "http://127.0.0.1:3128/depot/1/chunk/c4d06de785ad7e026da3688189d4e5cfcc18c618", host: "cache9-iad1.steamcontent.com"
2022/11/30 20:14:16 [crit] 1927#1927: *7370 open() "/data/cache/cache/d2/c8/de1fe6cc7b747c13378bf4ada218c8d2.0000001839" failed (13: Permission denied) while reading upstream, client: 192.168.0.246, server: , request: "GET /depot/1/chunk/e4acddff51c74e5eb142ad462755c80396294a44 HTTP/1.1", upstream: "http://127.0.0.1:3128/depot/1/chunk/e4acddff51c74e5eb142ad462755c80396294a44", host: "cache9-iad1.steamcontent.com"
2022/11/30 20:14:16 [crit] 1811#1811: *7362 open() "/data/cache/cache/c3/7c/4977b242bf0b680af4445c7dcee57cc3.0000001840" failed (13: Permission denied) while reading upstream, client: 192.168.0.246, server: , request: "GET /depot/1/chunk/0ea5b21250f3d04fe124dd5b6cd585c6db6b2b2a HTTP/1.1", upstream: "http://127.0.0.1:3128/depot/1/chunk/0ea5b21250f3d04fe124dd5b6cd585c6db6b2b2a", host: "cache9-iad1.steamcontent.com"
2022/11/30 20:14:16 [crit] 1927#1927: *7374 open() "/data/cache/cache/52/6a/70879240dbee5f893745a3305fff6a52.0000001841" failed (13: Permission denied) while reading upstream, client: 192.168.0.246, server: , request: "GET /depot/1/chunk/39365c55d7817a06d8551f16e0aabe84dfa1c9f4 HTTP/1.1", upstream: "http://127.0.0.1:3128/depot/1/chunk/39365c55d7817a06d8551f16e0aabe84dfa1c9f4", host: "cache9-iad1.steamcontent.com"
2022/11/30 20:14:16 [crit] 1830#1830: *7382 open() "/data/cache/cache/ec/73/e2814063368afa8009eabb41a08673ec.0000001842" failed (13: Permission denied) while reading upstream, client: 192.168.0.246, server: , request: "GET /depot/1/chunk/0bcb4eb1508c349b710dcaac9e7ab1ffca8cadff HTTP/1.1", upstream: "http://127.0.0.1:3128/depot/1/chunk/0bcb4eb1508c349b710dcaac9e7ab1ffca8cadff", host: "cache9-iad1.steamcontent.com"

There are permission errors in here though.

EDIT: I think the below is from the access.log

[steam] 192.168.0.246 / - - - [30/Nov/2022:20:17:04 +0000] "GET /depot/1/chunk/a0470dbc0b988f75d2ae75e4021f86a84a1df127 HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache3-par1.steamcontent.com" "-"

Edited November 30, 2022 by dGG2uA

ich777 · November 30, 2022

Just now, dGG2uA said:

Trying to check, having a weird problem now - I've cleared out the lancache directory to start fresh, I started the Monolothic container to find the empty log files:

Please stop your prefill container before doing anything, if you are deleting stuff while a container is running is always bad...

1 minute ago, dGG2uA said:

But AFTER starting the pre-fill container, the directory is full of fuse_hidden files:

Those files are created when files are in use by another application and you delete them...

I would recommend that you stop your LANCache Monolithic and LANCache DNS container, after that delete the contents from the cache and the logs folder including hidden files, then start both containers back again and after that start a scheduled prefill and attach to the access.log with tail like mentioned before.

Please also post screenshots from your LANCache templates, to what is your appdata share set in terms of Use Cache in the Share settings?

MiniMoe · November 30, 2022

2 minutes ago, ich777 said:

I would recommend that you stop your LANCache Monolithic and LANCache DNS container, after that delete the contents from the cache and the logs folder including hidden files, then start both containers back again and after that start a scheduled prefill and attach to the access.log with tail like mentioned before.

Doing this now.. will update soon.

3 minutes ago, ich777 said:

Please also post screenshots from your LANCache templates, to what is your appdata share set in terms of Use Cache in the Share settings?

Attached images, the lancache share is not set to use my cache drive, it writes straight to the array.

ich777 · November 30, 2022

2 minutes ago, dGG2uA said:

Attached images, the lancache share is not set to use my cache drive, it writes straight to the array.

I would recommend that you set the cache to /mnt/user/lancache/cache since this is the default folder structure for lancache but it should not harm anything in your case.

MiniMoe · November 30, 2022

1 minute ago, ich777 said:

I would recommend that you set the cache to /mnt/user/lancache/cache since this is the default folder structure for lancache but it should not harm anything in your case.

I've set the pre-fill container to /mnt/user/lancache/cache/ which essentially does the same thing, I think! Docker directory mappings always trip me up.

MiniMoe · November 30, 2022

16 minutes ago, ich777 said:

I would recommend that you stop your LANCache Monolithic and LANCache DNS container, after that delete the contents from the cache and the logs folder including hidden files, then start both containers back again and after that start a scheduled prefill and attach to the access.log with tail like mentioned before.

Access log output:

[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:43 +0000] "GET /depot/311211/chunk/038620b80a02a28ba63425a5c469126bebc84cd0 HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"
[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:43 +0000] "GET /depot/311211/chunk/0c1ed2844985fc71eb927f403bb81faeed4d42b2 HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"
[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:44 +0000] "GET /depot/311211/chunk/1438c6231105f91a298096ed6235c69daeadd38a HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"
[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:44 +0000] "GET /depot/311211/chunk/149de2481725418654f3b9c537958fee32032cfd HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"
[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:44 +0000] "GET /depot/311211/chunk/1253ceaa87363418793f5b9953e6974086c6e411 HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"
[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:44 +0000] "GET /depot/311211/chunk/1772ad762d704e6bfc14dc913130365ffe595d59 HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"
[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:44 +0000] "GET /depot/311211/chunk/1908a6776dff4c04a8e1fe7bd1be8c1af7eb15a0 HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"
[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:44 +0000] "GET /depot/311211/chunk/1e49257947b8771ad1bc8da6db952cc1dbecee81 HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"
[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:44 +0000] "GET /depot/311211/chunk/06d8d4fb4f5c8fc9d0c306fbcf6d7dcebe75f2b2 HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"
[steam] 192.168.0.246 / - - - [30/Nov/2022:20:39:44 +0000] "GET /depot/311211/chunk/14f6f8f70aa18691619af3c57f70f18ed8a115b3 HTTP/1.1" 200 0 "-" "Valve/Steam HTTP Client 1.0" "MISS" "cache2-sto2.steamcontent.com" "-"

ich777 · November 30, 2022

1 minute ago, dGG2uA said:

Access log output:

This is looking good and should fill you lancache.

If nothing is on the array, check your cache too, maybe the share is set to use cache Yes.

MiniMoe · November 30, 2022

4 minutes ago, ich777 said:

This is looking good and should fill you lancache.

I'm going to leave it overnight and see if it works. Problem with running scheduled is that it downloads Black Ops 3, it'll take hours to see if it actually works or not since my DL speed isn't great.

I'll report back tomorrow, maybe it's just a problem with running it manually..

Edit: there are files populating in a ".cache" directory, so maybe it is working as expected, just doesnt work manual.

Edited November 30, 2022 by dGG2uA

ich777 · November 30, 2022

3 minutes ago, dGG2uA said:

I'm going to leave it overnight and see if it works. Problem with running scheduled is that it downloads Black Ops 3, it'll take hours to see if it actually works or not since my DL speed isn't great.

Prefill says that it detected LANCache and shows that it is downloading?

You should also see some activity on the share where you actually downloading the game to from time to time (this is caused by Linux caching), also check your Cache, maybe the share is set to Yes and that's why you don't see anything on the Array.

I would also recommend that you maybe put the cache folder on a device outside of your Array, maybe through UD since I don't think that you need a Parity protected game mirror.

(This will maybe also speed up writes and reads)

MiniMoe · November 30, 2022

3 minutes ago, ich777 said:

Prefill says that it detected LANCache and shows that it is downloading?

You should also see some activity on the share where you actually downloading the game to from time to time (this is caused by Linux caching), also check your Cache, maybe the share is set to Yes and that's why you don't see anything on the Array.

I would also recommend that you maybe put the cache folder on a device outside of your Array, maybe through UD since I don't think that you need a Parity protected game mirror.

(This will maybe also speed up writes and reads)

The logs show downloading with a % number, I can see data writing to the array aswell (but plex is also in use right now). I'm hopeful its working, but I'll not know until a check again tomorrow and try to install black ops 3.

The share is definitely set to write to only the array!

I wish I could add a UD but I'm all out of drive trays! Its on my list of things to get, but for now it'll need to go on the array.

MiniMoe · November 30, 2022

1 hour ago, ich777 said:

Prefill says that it detected LANCache and shows that it is downloading?

You should also see some activity on the share where you actually downloading the game to from time to time (this is caused by Linux caching), also check your Cache, maybe the share is set to Yes and that's why you don't see anything on the Array.

I would also recommend that you maybe put the cache folder on a device outside of your Array, maybe through UD since I don't think that you need a Parity protected game mirror.

(This will maybe also speed up writes and reads)

It did not work, once it got to 100% it began retrying.

I'll try again tomorrow, for now just using lancache without prefill.

ich777 · December 1, 2022

6 hours ago, dGG2uA said:

It did not work, once it got to 100% it began retrying.

Can you please post the log from prefill?

It did not work is really nothing I can work with.

You can try to set up prefill manually on your server and see if you can get it to work.

kri kri · December 4, 2022

Krusader feature request. Can you include more themes / colormaps? They appear very small in size so shouldn't have a large impact on docker size I wouldn't think.

https://store.kde.org/browse?cat=371&ord=latest

ich777 · December 4, 2022

21 minutes ago, kri kri said:

Can you include more themes / colormaps?

I will look into that and report back.

Give me a few days please.

danimal86 · December 4, 2022

Is there a timeout for Luckybackups? I'm using it to copy my main server over to my backup server and when copying large amounts of data (probably around 10tb) i'll set it to run and come back the next day and it seems like it has stopped transfering. I'll go through and start the transfer again and it picks right back where it left off, but its strange.

Not a huge deal, but just curious.

Luckybackup has been great solution for my non-command-line-brain backups, thanks for creating it!

ich777 · December 4, 2022

17 minutes ago, danimal86 said:

Is there a timeout for Luckybackups? I'm using it to copy my main server over to my backup server and when copying large amounts of data (probably around 10tb) i'll set it to run and come back the next day and it seems like it has stopped transfering. I'll go through and start the transfer again and it picks right back where it left off, but its strange.

In general, no.

Do you maybe have CA Backup running? CA Backup by default stops the container when taking a backup.

danimal86 · December 4, 2022

3 minutes ago, ich777 said:

In general, no.

Do you maybe have CA Backup running? CA Backup by default stops the container when taking a backup.

I do....dang, i knew it was something simple!

I have it setup to do a backup every night....I kinda like having it that way, maybe it wont really matter once i do the bulk transfer and then i'll setup the schedule to start once the CA Backup/Restore has done its thing.

Unless there's a better way to do it.

ich777 · December 4, 2022

Just now, danimal86 said:

Unless there's a better way to do it.

Go into the settings from CA Backup, at the bottom click on Show Advanced Settings and simply click the toggle switch to Don't Stop the container.

danimal86 · December 4, 2022

1 hour ago, ich777 said:

Go into the settings from CA Backup, at the bottom click on Show Advanced Settings and simply click the toggle switch to Don't Stop the container.

Awesome. Is there any downside/risk to this?

Edit: ohh cool, you can toggle which one to not stop....so just set LuckyBackup to not stop and good to go?

Edited December 4, 2022 by danimal86

ich777 · December 4, 2022

3 minutes ago, danimal86 said:

Awesome. Is there any downside/risk to this?

Not in the case of luckyBackup.

I have set all my containers not to stop, but that isn‘t really common practice, that‘s just how things I do. 😅

[Support] ich777 - Application Dockers

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

ich777

ich777

ich777

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation