Hello all,

 

Recently starting having an issue, unsure when exactly it started as I don't often have a reason to stop a docker container.  Issue exists on 6.6.6 as well as 6.6.5.  Unknown if it existed in prior versions or if it is even unRAID OS version related.

 

I run the following containers.  All containers are the linuxserver.io version with the exception of UniFi Video which is pducharme:

Plex

UniFi

UniFi Video

Radarr

Sonarr

Sabnzbd

Tautulli

Transmission

 

Problem is as follows.  If you attempt to stop a running container, it does not stop.  The GUI shows the spinning arrows forever and once you finally refresh it still shows it running.  The container is in fact dead at that point and the container WebGUI does not respond, but it does not finish exiting correctly.  Issuing a docker stop containername or a docker kill containername from cmd line does nothing and hangs until you ctrl-c out.  I have not found a way to kill and/or restart the container successfully.  Some of the container logs appear as if the container exited successfully, while others the last line in the logs is "[s6-finish] syncing disks".  At this point the only way to get the container running again is to restart unRAID.  Problem is unRAID is now unable to stop the docker service, and therefore unable to stop the array.  The only way to restart the server is a hard power cycle, and hence an unclean shutdown.

 

In limited testing I have found that if only Plex and UniFi Video are running you can stop the array and the containers will successfully stop and the array successfully stops.  I have yet to start the containers one by one and find which ones are causing the issue.  I am currently in the process of migrating all drives to xfs and so have not yet had the time to test further.  All that said, it appears when the containers are automatically stopped/restarted weekly to update via CA Auto Update settings, they do stop and start correctly.

 

I have searched the forums and have not found a similar issue with resolution.  Attached are diags from when the containers were hung.  Any assistance would be appreciated.

 

 

 

landfill-diagnostics-20190102-1851.zip

Edited January 7, 20197 yr by dirtysanchez
Marked solved

Unifi Video crashed when a backup was run.  Plex crashed @ 6am on Jan 2

 

And right now some container is refusing to stop (not the docker system per se). 

 

It really wouldn't hurt to nuke the docker.img and reinstall the apps via Apps, Previous Apps.

Thanks for taking a look Squid.  Yes, plex crashed a few days ago (first time in years if I recall correctly) and that is what lead to the discovery of the issue as I was unable to get it running again without a hard reset that caused an unclean shutdown.  The following day I discovered it went beyond the plex container when I was attempting to stop some containers to prepare for migrating all my drives to xfs.

 

As for the current container refusing to stop (not the docker system per se), even if you turn Docker off it doesn't kill the running containers.  You can change Settings > Docker > Enable Docker to no and the containers don't die nor does Docker itself stop.  docker ps still shows them running.

 

Once the migrations are complete (tomorrow morning) I'll nuke the docker.img and reinstall all dockers per your suggestion and report back.  Thank you for the assistance.

Nuked the docker.img and recreated containers from my templates.  So far so good.  I am able to stop containers successfully.  I'll marked this solved in another 24 hours if all is still working.

 

Thanks again for the help.

Edited January 5, 20197 yr by dirtysanchez

Everything working as it should.  Marking this as solved.

For us beginners, it would help to describe how exactly you 'nuke' a docker image.   I'm trying to stop a docker that won't stop and still don't know how.  Do you mean I just putty in and remove them?

22 minutes ago, [email protected] said:

For us beginners, it would help to describe how exactly you 'nuke' a docker image.   I'm trying to stop a docker that won't stop and still don't know how.  Do you mean I just putty in and remove them?

https://forums.unraid.net/topic/57181-docker-faq/#comment-564309

 

my two cents (but I'm a noob). I had to do this a few times as the docker stop commands didn't result in an actual container stop.

 

Get you docker container id using

docker container list

then 

ps auxw | grep yourcontainerid

to get the pid

then 

kill -9 yourpid

If that doesn't work, you've got a zombie process and I'm afraid you'll need a reboot to unlock it

Edited August 30, 20214 yr by Osiris

On 8/31/2021 at 12:13 AM, Osiris said:

my two cents (but I'm a noob). I had to do this a few times as the docker stop commands didn't result in an actual container stop.

 

Get you docker container id using

docker container list

then 

ps auxw | grep yourcontainerid

to get the pid

then 

kill -9 yourpid

If that doesn't work, you've got a zombie process and I'm afraid you'll need a reboot to unlock it

 

Thanks! did a force update afterwards and its back up and responding.

On 8/30/2021 at 6:13 PM, Osiris said:

my two cents (but I'm a noob). I had to do this a few times as the docker stop commands didn't result in an actual container stop.

 

Get you docker container id using

docker container list

then 

ps auxw | grep yourcontainerid

to get the pid

then 

kill -9 yourpid

If that doesn't work, you've got a zombie process and I'm afraid you'll need a reboot to unlock it

Worked perfectly

I know it's been a while but just wanted to chime in that this worked for me as well.  Got the container ID, killed both instances that were running, went back to the GUI, stopped the container (deluge), and then did a force update.  Worked great.  Thank you!

On 8/31/2021 at 1:13 AM, Osiris said:

my two cents (but I'm a noob). I had to do this a few times as the docker stop commands didn't result in an actual container stop.

 

Get you docker container id using

docker container list

then 

ps auxw | grep yourcontainerid

to get the pid

then 

kill -9 yourpid

If that doesn't work, you've got a zombie process and I'm afraid you'll need a reboot to unlock it

I came to say, THANK YOU! I try with docker kill xxxx, wait 10 min and nothing. Then this solution did the thing!

Edited August 7, 20232 yr by b0n3v

On 8/30/2021 at 5:13 PM, Osiris said:

my two cents (but I'm a noob). I had to do this a few times as the docker stop commands didn't result in an actual container stop.

 

Get you docker container id using

docker container list

then 

ps auxw | grep yourcontainerid

to get the pid

then 

kill -9 yourpid

If that doesn't work, you've got a zombie process and I'm afraid you'll need a reboot to unlock it

just wanna laugh and say i opened up terminal to use this force stop method, and i guess my server got scared and suddenly let the docker stop, without typing a dam thing

...this experience continues to feed my theory that computers are just demons in physical form!

Edited August 24, 20232 yr by miicar

Worked.
Just wanted to add what worked for me:

I followed the instructions here, so:

 

On 8/31/2021 at 12:13 AM, Osiris said:

my two cents (but I'm a noob). I had to do this a few times as the docker stop commands didn't result in an actual container stop.

 

Get you docker container id using

docker container list

then 

ps auxw | grep yourcontainerid

to get the pid

then 

kill -9 yourpid

If that doesn't work, you've got a zombie process and I'm afraid you'll need a reboot to unlock it

 

After that I was able to stop the affected docker. Then I went to the docker (in the web interface) and went to edit. Here I didn't change anything, just a space, which I removed afterwards so I could click Apply. After that the Docker started as if nothing had happened and was directly accessible again via my domain.
Maybe this works for others as well...

On 8/30/2021 at 6:13 PM, Osiris said:

my two cents (but I'm a noob). I had to do this a few times as the docker stop commands didn't result in an actual container stop.

 

Get you docker container id using

docker container list

then 

ps auxw | grep yourcontainerid

to get the pid

then 

kill -9 yourpid

If that doesn't work, you've got a zombie process and I'm afraid you'll need a reboot to unlock it

I encountered the issue too with one docker in a started state not being accessible by its webUI. I tried killing it in this manner by PID, but it was not clear what integer was the PID in the grep output. However, I was not able to cleanly stop or restart the docker.

I ultimately had to stop all other containers, stop the array (which hung), and eventually had to reboot unraid softly to resolve the issue with one docker. All my other dockers were healthy.

I'm on unraid 6.1.2.6

On 8/30/2021 at 6:13 PM, Osiris said:

my two cents (but I'm a noob). I had to do this a few times as the docker stop commands didn't result in an actual container stop.

 

Get you docker container id using

docker container list

then 

ps auxw | grep yourcontainerid

to get the pid

then 

kill -9 yourpid

If that doesn't work, you've got a zombie process and I'm afraid you'll need a reboot to unlock it

I swapped out the final command with this one...

 

kill -9 `pgrep -f yourcontainerid` 

 

When I clicked on the docker it finally stopped!! I was trying to figure out the pid and someone suggested this on a stackoverflow...

Happy working!