[6.8.0] Unresponsive

[christofferm]

Gets gradually more and more unresponsive. Please see attached picture.

After approx one our the server is not responding at all.

 

Intel based platform.

[JorgeB]

This looks more like a general support issue, but in any case we need the diagnostics (tools -> diagnostics), if you can't get the when the server becomes unresponsive see here to enable syslog mirroring to flash.

[christofferm]

Changed Priority to Minor

[christofferm]

Ok I change it to minor...

 

When I say its not responding at all, thats including pressing the chassis power button to turn off the server resulting in that I need make a force shutdown.

 

Rolling back to 6.7.3-rc4 all work ok.

 

I get back with the diagnostic soon

[bonienl]

Please open a topic under general support. Something "else" is interfering with your set up.

Did you try "safe mode"?

[christofferm]

Here comes a fresh set of diagnostic data.

 

No I have not try "safe mode"

 

Isn't this the place to report bugs related to upgrading to 6.8.0? I can reproduce that this issue does not exist in 6.7.3-rc4.

 

Is there an easy way to move this issue to the general support section?

srv01-diagnostics-20191214-1429.zip

[trurl]

Copied from the bug report to this thread in General Support. Please continue this discussion here until such time as it is determined this is an actual bug.

 

[trurl]

For future reference see this post about how to report bugs:

 

https://forums.unraid.net/bug-reports/stable-releases/report-guidelines-r68/

 

So in addition to diagnostics, you should try to reproduce the issue in SAFE mode to eliminate any plugins.

[bonienl]

First thing to do is start your system in "safe mode" without plugins and retest.

[christofferm]

5 minutes ago, trurl said:

For future reference see this post about how to report bugs:

 

https://forums.unraid.net/bug-reports/stable-releases/report-guidelines-r68/

 

So in addition to diagnostics, you should try to reproduce the issue in SAFE mode to eliminate any plugins.

Ok got it!.. sorry for that.

 

4 minutes ago, bonienl said:

First thing to do is start your system in "safe mode" without plugins and retest.

Rebooting in safe mode now...

[christofferm]

Ok your right, seems to be an plugin issue (working ok in safe mode)

 

Trying to hunt down what plugin causes this...

[bonienl]

18 minutes ago, christofferm said:

Trying to hunt down what plugin causes this...

Start with

libvirt.hotplug.usb
NerdPack

 

[christofferm]

Wrong conclusion from my part. I'm having this issue in safe mode.

 

Checking docker containers now...

srv01-diagnostics-20191215-0123.zip

[bonienl]

Something is eating your memory, causing resource issues.

 

Most common reason is a misconfigured docker container, which results in storing everything in memory rather than disk.

 

[trurl]

And the fact you have allocated 40G to docker image also makes me suspect you have misconfigured docker applications. I usually recommend 20G. Have you had problems with docker image filling? Making it larger will not fix it, it will just make it take longer to fill.

 

There are 2 different ways you can get problems with misconfigured dockers:

1. If you have an application that writes to any path that isn't mapped, it will be writing into the docker image, filling it up. Common mistakes are not using the same upper/lower case as in the mappings, or not using an absolute path (beginning with /).
2. If you have a host path that is not actual storage, that path is in RAM. This includes any path to an Unassigned Device that isn't actually mounted.

[christofferm]

2 hours ago, trurl said:

And the fact you have allocated 40G to docker image also makes me suspect you have misconfigured docker applications. I usually recommend 20G. Have you had problems with docker image filling? Making it larger will not fix it, it will just make it take longer to fill.

 

There are 2 different ways you can get problems with misconfigured dockers:

1. If you have an application that writes to any path that isn't mapped, it will be writing into the docker image, filling it up. Common mistakes are not using the same upper/lower case as in the mappings, or not using an absolute path (beginning with /).
2. If you have a host path that is not actual storage, that path is in RAM. This includes any path to an Unassigned Device that isn't actually mounted.

Yes I increased the docker image to troubleshooting "image filling" in the past and I'm just lazy not recreating a smaller image

 

It seems to be my unifi-video container thats gets out of hand. Just trying to understand what changed in the docker engine between 18.09.2 and 19.03.5 that I need adapt to.

 

Thanks for the support!  I guess we can consider this issue closed for now as it is a docker issue and not an unraid issue.