Gday all,

 

It's been bugging me for a while, but more often than not, i'm getting a message 'no such container' after i restart (yes gracefully) my unraid server. I had this on my Ryzen platform and now on my Epyc platform.

 

I have 'rebuild' my libvirt file a couple of times, i have done parity checks. Heck i even moved my docker image from the array to cache to a mounted volume, but the problem persists. Any pointers where it could be?

 

Is it maybe that my array isn't fully 'booted' when docker starts? if so, can i delay starting the docker service somehow?

 

Turning docker service off and then on again fixes the issue

Edited August 18, 20214 yr by Caennanu

9 minutes ago, Caennanu said:

I have 'rebuild' my libvirt

This is used for VMs, not docker, posting diagnostics might show something. 

dohh my bad, obviously i ment my docker.img file.

(p.s. i'm aware of the one memory module acting up)

 

Caennanu-diagnostics-20210818-0901.zip

Edited August 18, 20214 yr by Caennanu

Aug 18 08:34:36 Bigboii rc.docker: binhex-couchpotato-git: Error response from daemon: network 255d84a6579f8c87400791f1109ebd9bb1f2a41a504ab97fb3a4f3e9d722e26c not found
Aug 18 08:34:36 Bigboii rc.docker: Error: failed to start containers: binhex-couchpotato-git
Aug 18 08:34:36 Bigboii rc.docker: binhex-sickchill: Error response from daemon: network 255d84a6579f8c87400791f1109ebd9bb1f2a41a504ab97fb3a4f3e9d722e26c not found
Aug 18 08:34:36 Bigboii rc.docker: Error: failed to start containers: binhex-sickchill
Aug 18 08:34:36 Bigboii rc.docker: binhex-syncthing: Error response from daemon: network 255d84a6579f8c87400791f1109ebd9bb1f2a41a504ab97fb3a4f3e9d722e26c not found
Aug 18 08:34:36 Bigboii rc.docker: Error: failed to start containers: binhex-syncthing
Aug 18 08:34:36 Bigboii rc.docker: Unifi: Error response from daemon: network 255d84a6579f8c87400791f1109ebd9bb1f2a41a504ab97fb3a4f3e9d722e26c not found
Aug 18 08:34:36 Bigboii rc.docker: Error: failed to start containers: Unifi

 

Some containers are failing to start, not sure what these errors mean though, maybe something is not ready yet, someone else might know what they are about.

Looks like the dockers start quicker than the vlan's are set.

Not obviously related, but why do you have 50G for docker.img. And even 50G for libvirt.img (not sure I've ever seen that).

 

20G is usually more than enough for docker.img, and if you had problems filling it, making it larger won't fix that.

 

I'm pretty sure nobody has ever needed more than the default 1G for libvirt.img.

No particular reason i can think off currently. . . 

Have you ever had problems filling docker.img?

its been like this for over a year, i know i had issues with dockers taking rather large amounts of space in the past, maybe that's why i still have it set to 50G. Currently there's 11GB in use if i'm to believe the scrubber.

You can recover that space by recreating docker.img.

 

https://wiki.unraid.net/Manual/Troubleshooting#How_do_I_recreate_docker.img.3F

 

Not sure about libvirt.img since I have never needed to recreate that. Do you have it backed up?

Don't have a current backup but easy to make. But do you really think that because they're large they are causing these issues? (i can always try ofcourse)

8 hours ago, JorgeB said:

binhex-syncthing: Error response from daemon: network 255d84a6579f8c87400791f1109ebd9bb1f2a41a504ab97fb3a4f3e9d722e26c not found

Most likely the containers are running through another container for their internet access (VPN) and that container isn't started yet 

I have lancache that is the only thing they could run thru and starts first.

 

Other than that all network related things are handled by my router. The only thing i can think off is that the bridges for the Vlan's are started at a later time than the dockers. Ill increase the wait time for lancache.

Edited August 18, 20214 yr by Caennanu

12 hours ago, Caennanu said:

do you really think that because they're large they are causing these issues?

 

13 hours ago, trurl said:

Not obviously related

But it does make me wonder if you don't have other things misconfigured.

4 hours ago, trurl said:

 

But it does make me wonder if you don't have other things misconfigured.

That makes sence.

I've reduced libvirt to 2gb and docker.img to 20gb

 

Is there any way i can put a pause on the entire docker service before starting up? So unraid has time to start the bridges?

(as i don't seem to have the issue with bridges on VM's)

 

 

Each container is started in the order it appears on the Dockers page. You can reorder these by dragging them.

 

If you turn on Advanced View (slider at upper right), you can set a delay for each container, which will make it wait before it tries to start, each one delaying after the previous one has started.

45 minutes ago, trurl said:

Each container is started in the order it appears on the Dockers page. You can reorder these by dragging them.

 

If you turn on Advanced View (slider at upper right), you can set a delay for each container, which will make it wait before it tries to start, each one delaying after the previous one has started.

Oh i thought it worked the other way around. The next container has to wait x amount before starting. This makes much more sence 😛

2 hours ago, Caennanu said:

Oh i thought it worked the other way around. The next container has to wait x amount before starting. This makes much more sence 😛

Just to make it perfectly clear, if you only have 1 container, the delay doesn't do anything. Container starts, waits X seconds as specified, moves to next container. I really wish it was the other way around, but I get around it by putting a container that doesn't need network in the first slot and giving it a long (300 seconds) wait to ensure the rest of the box is settled and ready to work.

1 hour ago, JonathanM said:

Just to make it perfectly clear, if you only have 1 container, the delay doesn't do anything. Container starts, waits X seconds as specified, moves to next container. I really wish it was the other way around, but I get around it by putting a container that doesn't need network in the first slot and giving it a long (300 seconds) wait to ensure the rest of the box is settled and ready to work.

Yeah that is what i originally thought. Unfortunately i have more than one container 😛

 

Thanks for the clarification

so . . . so far, i've decreased the size of libvirt and docker.img.

I have added a docker infront of the rest with 120 seconds wait time to start the next docker.

 

So far issue persists. (some dockers start, but most don't and when manually starting them you get the message, no such container, restarting docker from settings and everything works as it should)

 

updated diagnostics attached.

 

Caennanu2-diagnostics-20210820-1240.zip

Edited August 20, 20214 yr by Caennanu

Do you have any orphan containers?

I currently have one orphaned, but that is because i recently updated sickchill to latest. Had the issue before the orphan tho.

To test i have removed the orphan image. Issue persists 

Could it be caused by the bond i have made with the 2 nics?

 

 

 

 

Edited August 21, 20214 yr by Caennanu

Old topic, I know, but I came across this (and other topics) while researching my very similar issue.

 

The error diagnostics / error messages from @Caennanu posted by @JorgeB caught my eye:

 

 

Do these containers have additional networks, as in 'docker network connect...'?

 

If so I think I found the issue, I posted about it here:

 

 

Hopefully this helps (if you are still even having the issue...)

Hello. I had the same issue and tried a very quick fix that seeme to do the work :

TLDR;

Disable > Enable docker service in settings > Docker

 

I had the same issue after trying some change in my config and ended up on this thread that seemed interesting.

After a restart, none of my container did start. Error message : "Container does not exist" (or something of the sort). 
no message in containers logs since there was nothing to start at all.

But after some tries, this quick fix is what I tried first and it worked

 

Edited February 25, 20233 yr by DrKamp