I noticed the past couple days after leaving the server running all my docker containers would stop responding and the only way to fix it is to restart the server. 

I thought it was a one time thing but it just happened again and I'm not sure what the cause is. 

 

Going on the docker tab takes a longer time to load than usual (around 1min) and then pressing stopping all Dockers just ends up with a looping animation. I can't restart any docker containers and the only option is to restart the server.

unnas-diagnostics-20190313-1221.zip

I noticed the same thing off and on for a few weeks.  In my case, I found that switching sonarr from the lsio build to instead be binhex fixed everything up for me.  YMMV

Thanks, I'll move my sonarr docker to binhex. Did you do it for just sonarr or for also radarr,jackett,lidarr,plex?

Only for sonarr.  I suspect that the "update done every friday" wound up pulling bad apps.

Unfortunately changing to binhex-sonarr didn't fix the issue. I noticed that all my cpu and ram are being used as well when the docker containers stop responding but htop shows otherwise . Attached are some new diagonostics.

 

Edit: weird thing happened but once docker containers automatically updated the CPU all went back to normal. The docker containers that got updated were :

 

Community Applications
Docker Auto Update
normal: airsonic (was stopped), bazarr binhex-sonarr nextcloud radarr rutorrent sonarr (was stopped), tautulli unifi-controller jackett letsencrypt Automatically Updated

 

How can I figure out which docker containers is causing the hang ups?

 

 

unnas-diagnostics-20190325-0529.zip

Edited March 25, 20197 yr by bobokun

Just go to docker and click on advanced (right top)

 

I also suggest to add the ca backup, to restart your apps every day,  helps with memory leaks and all sort of problems.

The hard drives start maknig a lot of noise when this happens. I think there is an issue with I/O and it gets stuck? Is there any way I can figure out if it's a hardware issue or software? My drives are all relatively new (1-2years) and have gone through several runs of preclear before using them so I don't think it should be the hard drives. Any way I can check?

If you run out of RAM, your disks get more stress, and since tehy muich much much much slower then RAM...

 

Did you find out which docker it is? Did a (docker) restart fix it?

No, I need to wait until the next  time it happens again for me to find out which one is causing the issue. The problem is when I tried to stop a docker container it just hangs (I think because the cpu and RAM are all 100%), so I can't even kill the docker container or shut it down from the GUI. Is there a way to stop all docker containers using ssh commands? Or if that doesn't work can I kill the process running the docker container? 

We cant help you if you dont answer... why u cant find out which docker it is? Did u install ca backup for auto restart every day?

 

you can try docker stop name

Edited March 25, 20197 yr by nuhll

Seems to be plex being the culprit, it has a memory leak. I'm using the linuxserverio docker container. Should I be switching to binhex for plex?

U should be restarting your dockers every day. Im using the same docker since months without problems.

Edited March 26, 20197 yr by nuhll

I thought invoking the mover every night will automatically stop the docker containers and start them again once mover is completed. Do you run a custom script to stop and start all docker containers?

17 minutes ago, bobokun said:

I thought invoking the mover every night will automatically stop the docker containers and start them again once mover is completed. Do you run a custom script to stop and start all docker containers?

No, mover does not stop any dockers.    In fact if a docker has any files open it will stop mover from moving that file.

In that case should i be using a script to stop all docker containers before invoking mover?

2 minutes ago, bobokun said:

In that case should i be using a script to stop all docker containers before invoking mover?

It is really up to you!  Any recommendation has to be based on exactly which dockers you run and what they are doing.    I know in my case I have no dockers that will permanently have file open that I want to be moved using mover so I do not bother.   It is worth remembering that mover never touches any files that are in a Cache-only Share (e.g. appdata) so they can be ignored in any such consideration.

My first post told you how to archive that the dockers restart every day?

 

Install the plugin "CA Backup" it 1. create a backup and 2. restart dockers for that

you create backups of all your docker containers everyday?

6 minutes ago, bobokun said:

you create backups of all your docker containers everyday?

Yes. But you can change the interval like you want. I do backups every day and save them for a month. So in incase some update goes wrong, i can just try the last versions of it.

Edited March 26, 20197 yr by nuhll