[Support] ich777 - Application Dockers

[ryans100]

Getting an odd error trying to update Luckybackup to the most recent version. I did an uninstall on it and reinstall, and I'm still getting the error.

 

Pulling image: ich777/luckybackup:latest
IMAGE ID [995095759]: Pulling from ich777/luckybackup.
IMAGE ID [e9995326b091]: Already exists.
IMAGE ID [7ffd6c2f1d35]: Already exists.
IMAGE ID [d6276708addb]: Already exists.
IMAGE ID [55497894789e]: Already exists.
IMAGE ID [b9b53c6477e6]: Already exists.
IMAGE ID [723f47b26624]: Already exists.
IMAGE ID [2fed92c09f0f]: Already exists.
IMAGE ID [a1063aaa0fc0]: Already exists.
IMAGE ID [bf6130e4dcab]: Pulling fs layer. Downloading 100% of 2 MB. Verifying Checksum. Download complete. Extracting.
IMAGE ID [97c82dbb912f]: Pulling fs layer. Downloading 100% of 177 MB. Verifying Checksum. Download complete.
IMAGE ID [a2c08ebca08f]: Pulling fs layer. Downloading 100% of 2 KB. Download complete.
IMAGE ID [1ea2d71905e2]: Pulling fs layer. Downloading 100% of 2 KB. Download complete.
IMAGE ID [e48f1fd1e16c]: Pulling fs layer. Downloading 100% of 2 KB. Download complete.
IMAGE ID [cb789845174f]: Pulling fs layer. Downloading 100% of 359 KB. Verifying Checksum. Download complete.
IMAGE ID [9169d6962369]: Pulling fs layer. Downloading 100% of 450 B. Verifying Checksum. Download complete.
IMAGE ID [e5226bdcf121]: Pulling fs layer. Downloading 100% of 151 B. Verifying Checksum. Download complete.
IMAGE ID [39fc8179513d]: Pulling fs layer. Downloading 100% of 133 MB. Verifying Checksum. Download complete.

TOTAL DATA PULLED: 312 MB

Error: failed to register layer: stat /var/lib/docker/btrfs/subvolumes/3f9b25f0a5e5fd41668734cffe3951f784f2d7a965eaa411c3ec43794e0eb9bc: no such file or directory

 

The /var/lib/docker/btrfs/subvolumes/3f9b25f0a5e5fd41668734cffe3951f784f2d7a965eaa411c3ec43794e0eb9bc folder itself doesn't exist inside of my /var/lib/docker/btrfs/subvolumes folder.

 

Any ideas? I'm not familiar with that folder, so I'm hesitant to just create a folder named 3f9b25f0a5e5fd41668734cffe3951f784f2d7a965eaa411c3ec43794e0eb9bc in there.

 

Other containers are working fine, I can install new stuff, uninstall, etc just fine as well. 

[ich777]

12 minutes ago, ryans100 said:

Getting an odd error trying to update Luckybackup to the most recent version. I did an uninstall on it and reinstall, and I'm still getting the error.

I'm not sure but I don't think it's directly related to my container, I think this has more to do with the underlying filesystem I think.

Do you maybe have Diagnostics? Do you use a image or directory for Docker?

 

@JorgeB can you please help here?

[Kilrah]

27 minutes ago, ryans100 said:

 

Error: failed to register layer: stat /var/lib/docker/btrfs/subvolumes/3f9b25f0a5e5fd41668734cffe3951f784f2d7a965eaa411c3ec43794e0eb9bc: no such file or directory

 

That's generic Docker filesystem corruption. Did your Docker image get filled up? Might want to recreate it.

https://docs.unraid.net/unraid-os/manual/docker-management/#re-create-the-docker-image-file

Edited July 12 by Kilrah

[te5s3rakt]

On 5/29/2022 at 4:38 PM, ich777 said:

If anything breaks again take a look at this: Click

 

Or what I would recommend:

1. Stop the container
2. Delete the files "Core.jar" & "JDownloader.jar" and the folders "tmp" & "update" from your jdownloader2 directory
3. Start the container again

 

Since this happens frequently, I wonder if it's worth adding something to the container on start. Something like "delete files if greater than 30 days old"?

 

Because it's literally not a matter of if, but when, this error will return. And it seems to be quite regular.

[ich777]

4 minutes ago, te5s3rakt said:

Because it's literally not a matter of if, but when, this error will return. And it seems to be quite regular.

I'm not sure if everyone has this issue, I only had that happen once to me.

May I ask what kind of storage do you have and where the container data is located on?

 

Everything is working over here and as said this only happened once to me.

 

You can try to switch the container to the other one that's available in the CA App, I think this container won't suffer from this issue.

[alturismo]

2 hours ago, ich777 said:

You can try to switch the container to the other one that's available in the CA App, I think this container won't suffer from this issue.

wont help, im using "the other" container  and it also happened twice here in the last couple years ...

 

its a matter of "unclean" shutdown of the docker, or the FS (btrfs image prolly) is likely broken

[Merijeek]

@ich777 -The good news, it seems like all the restarts are working. The bad news, "all" seems to be an overstatement. 

 

I've got a pile of stuff working just fine. However, one of your containers (readarr) isn't restarting with the others. 

 

What I've noticed (again, still nowhere near an expert on Docker stuff) is that this particular container, once it's dropped off can't even run ifconfig:

 

 

Oops. Windows habit:

 

Edited July 15 by Merijeek

[ich777]

3 hours ago, Merijeek said:

What I've noticed (again, still nowhere near an expert on Docker stuff) is that this particular container, once it's dropped off can't even run ifconfig:

 

Oops. Windows habit:

Yes, because you usually don't need ifconfig or any network tools in such a container, that's why it's not installed and to safe space on users machines.

 

The logs are what real important <- since it will display everything there too and the above tools aren't needed anyways.

Does the Readarr container say that the Connected containers service is started? Does it also display a message that the connection is lost and the container does restart?

 

But just to confirm, everything else is working correct?

[Merijeek]

Everything else is working. Readarr is showing this: 

 

The config in the container for that is identical to that in all the other containers: 

 

[ich777]

6 minutes ago, Merijeek said:

Everything else is working. Readarr is showing this: 

Sorry for the maybe dumb question but are you sure that it is using the OpenVPN network?

If yes, the only thing that I could recommend, since I see something with IPv6 mentioned in the logs is to disable IPv6 on your server entirely (of course if you don't need it).

[Merijeek]

1 hour ago, ich777 said:

Sorry for the maybe dumb question but are you sure that it is using the OpenVPN network?

If yes, the only thing that I could recommend, since I see something with IPv6 mentioned in the logs is to disable IPv6 on your server entirely (of course if you don't need it).

Yup, thought of that: 

 

 

As far as IPv6, I don't need it. But literally the only place I can find IPv6 anywhere in my network settings of my server is a single Tailscale route. Otherwise, all my interfaces are IPv4 only. 

 

It seems unlikely something is already bound to that port, but either way I can't find any commands that could tell me that which actually work on the docker. 

Edited July 15 by Merijeek

[ich777]

2 minutes ago, Merijeek said:

As far as IPv6, I don't need it. But literally the only place I can find IPv6 anywhere in my network settings of my server is a single Tailscale route. Otherwise, all my interfaces are IPv4 only. 

Please try to disable IPv6 on your server entirely in the network settings.

 

Maybe that will help. In the meantime I'll take a quick look in the source from my Readarr container if I made a mistake there.

[Merijeek]

I think the point is I don't know HOW to disable it entirely. The only way I can see to add or remove IPv6 is if I have an interface that is set for both. 

 

Looking at my ifconfig, I've got a bunch of virtual addresses like this: 

 

veth0c92aa4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::24f4:b3ff:fe7b:596e  prefixlen 64  scopeid 0x20<link>
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 308 (308.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

 

All the IPv6 veths go away if I stop the Docker service. But I don't know how to actually disable IPv6 everywhere. 

[ich777]

15 minutes ago, Merijeek said:

I think the point is I don't know HOW to disable it entirely. The only way I can see to add or remove IPv6 is if I have an interface that is set for both. 

Go to Settings -> Network Settings -> Network protocol -> IPv4 only:

 

 

 

My bad in terms of Readarr, I was missing a dependency, now it should work fine, just force a container update.

[Merijeek]

Yeah, that's how I've always been set on my NICs. 

 

Isn't there an advanced command switch in Docker that can be added to a container that says no ipv6? 

 

Right now I've got them all disabled except OVPN. And OVPN upon start is giving me a veth interface. 

[ich777]

43 minutes ago, Merijeek said:

Isn't there an advanced command switch in Docker that can be added to a container that says no ipv6? 

Yes but it's working sub par AFAIK.

 

43 minutes ago, Merijeek said:

Right now I've got them all disabled except OVPN. And OVPN upon start is giving me a veth interface. 

Did you already update the container? It should now work.

[Merijeek]

I did, and the restart piece seems to be working since it's got an uptime of about 10 seconds, along with my other connected containers. Figuring out why the OVPN container seems to crap out so often would be nice, but isn't all that important as long as containers are rebooting correctly. 

 

The IPv6 thing was just trying to understand things more than anything else. 

 

[ich777]

7 hours ago, Merijeek said:

Figuring out why the OVPN container seems to crap out so often would be nice

Usually the container should reconnect anyways but I'm not sure what's going on on your system and why it#s even restarting or better speaking why you are getting a connection timeout.

 

However, as long as the restart is working everything should be good for now.

[ryans100]

On 7/12/2024 at 10:14 PM, ich777 said:

I'm not sure but I don't think it's directly related to my container, I think this has more to do with the underlying filesystem I think.

Do you maybe have Diagnostics? Do you use a image or directory for Docker?

 

@JorgeB can you please help here?

 

On 7/12/2024 at 10:27 PM, Kilrah said:

That's generic Docker filesystem corruption. Did your Docker image get filled up? Might want to recreate it.

https://docs.unraid.net/unraid-os/manual/docker-management/#re-create-the-docker-image-file

I don't use the docker image file, I'm using the directory, so I'm not sure how the image get be corrupted or getting filled up. Like I said, other containers work fine and update fine.

 

Diagnostics attached.

 

Edited July 16 by ich777
removed diagnostics because of credentials

[ich777]

17 minutes ago, ryans100 said:

Like I said, other containers work fine and update fine.

You have a lot of these in your syslog which need immediate attention:

May 27 02:42:42 Tower kernel: BTRFS warning (device sdd1): csum failed root 5 ino 230608208 off 4624384 csum 0x8941f998 expected csum 0x842c504f mirror 1
May 27 02:42:42 Tower kernel: BTRFS error (device sdd1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 71, gen 0
May 27 02:42:42 Tower kernel: BTRFS warning (device sdd1): csum failed root 5 ino 230608208 off 4628480 csum 0x8941f998 expected csum 0x9d2dc113 mirror 1
May 27 02:42:42 Tower kernel: BTRFS error (device sdd1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 72, gen 0
May 27 02:42:42 Tower kernel: BTRFS warning (device sdd1): csum failed root 5 ino 230608208 off 4632576 csum 0x8941f998 expected csum 0x65345017 mirror 1
May 27 02:42:42 Tower kernel: BTRFS error (device sdd1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 73, gen 0
May 27 02:42:42 Tower kernel: BTRFS warning (device sdd1): csum failed root 5 ino 230608208 off 4624384 csum 0x8941f998 expected csum 0x842c504f mirror 1
May 27 02:42:42 Tower kernel: BTRFS error (device sdd1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 74, gen 0
May 27 02:42:42 Tower kernel: BTRFS warning (device sdd1): csum failed root 5 ino 230608208 off 4624384 csum 0x8941f998 expected csum 0x842c504f mirror 1
May 27 02:42:42 Tower kernel: BTRFS error (device sdd1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 75, gen 0

 

This is a bit out of scope for this thread, please create a Post in the General subforums with your Diagnostics and a appropriate title like "BTRFS errors in syslog" or similar.

[Mainfrezzer]

@ryans100delete that attachment from your diagnostics and be absolutely careful for the next upload. There are login credentials visible

[ryans100]

5 hours ago, Mainfrezzer said:

@ryans100delete that attachment from your diagnostics and be absolutely careful for the next upload. There are login credentials visible

Thanks. I had the Anonymize diagnostics checked, but I guess that didn't matter.  

[Mainfrezzer]

39 minutes ago, ryans100 said:

Thanks. I had the Anonymize diagnostics checked, but I guess that didn't matter.  

Gotta check the system/ps file for running programs with variables in the parameter. That issue should be fixed in the future as far as I know.

[ryans100]

14 minutes ago, Mainfrezzer said:

Gotta check the system/ps file for running programs with variables in the parameter. That issue should be fixed in the future as far as I know.

Ah, I see what you're talking about. Yeah you'd think that would be cleared out. Appreciate that!

[AwesomeAustn]

On 5/21/2024 at 10:58 PM, ich777 said:

So to speak, XFS is working, BTRFS not.

 

Have you yet reported that to MEGA support?

 

I have been in contact with MEGA for months on and off. Since I disabled the cache drive for my MEGA syncs, I haven't had the FSID errors, but I have been having another issue with certain files of a certain folder that I update monthly not overwriting/updating by themselves.

 

It makes me choose or rename them one at a time. It wouldn't choose the newest date automatically even when the mode is set to Smart. All syncs are wait until these are resolved.

 

They sent me this new version tonight. Could you please update the docker? I'm not sure what this version is suppose to fix since I had two tickets open. One for the FSID errors, and one for my pick a file to keep annoyance.

https://mega.nz/file/hyITiCBD#vUekp0gldmS3qRVb9QodpYtc7ianGkb3Wsq2mB8A51s