How to debug out of memory kill child issue


Recommended Posts

I keep having my 1 and only VM get shutdown due to an out of memory issue. I don’t know how to debug this to find the root cause. Can anyone point me into the direction of this? Here is a segment of my log that shows where the error occurred. 192...88 is the IP of my VM, before that you se the “out of memory”. I don’t know how to find out which docker container, if it is a docker container that is hogging all the RAM enough to shut down my VM.

 

On another note, this is why i was wanting to setup some notifications for unRAID to be able to send me information on when RAM or CPU is maxing out so I can go log in immediately and see what is going on. If anyone has pointers on this it would help too. 

edith-syslog-20191108-1928.zip

Link to comment
  • 4 weeks later...
On 11/11/2019 at 12:26 AM, BRiT said:

Might start looking at Crashplan, as the virtual memory (Virt) is several times more than what your Qemu VM is using and it always was listed high OOM score when it was being triggered. There was also another process "java" that was listed high during OOM but I couldn't tell you what it was running.

 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
10128 root      20   0 2691908   2.1g  21920 S  18.8  17.6   1252:56 qemu-syst+
 9002 nobody    20   0 6970012 818916   9104 S   6.2   6.7 957:22.30 CrashPlan+

Thank you! This keeps happening. Do you think adding more RAM would solve my issue? Not sure how to debug crashplan here

Link to comment
7 hours ago, Ustrombase said:

Thank you! This keeps happening. Do you think adding more RAM would solve my issue? Not sure how to debug crashplan here

More RAM alone won't necessarily resolve your problem.

Crashplan is known to eat memory for breakfast lunch and dinner (btw, I believe Crashplan uses java so that's what Brit saw).

 

The short term way around it is to limit Crashplan docker memory usage so it doesn't end up eating other stuff.

In Docker template, click Advanced View then in the Extra Parameters box add

--memory=8G

(in the above example, it will limit the memory available to Crashplan to 8G - I actually had to use 8G because 4G crashes my Crashplan - just to demo how much memory Crashplan needs).

 

In the medium term, add more RAM so you have more available for other stuff.

 

In the long term, I would recommend moving away from Crashplan all together. They used to be great but no more.

  • Thanks 1
Link to comment
19 hours ago, testdasi said:

More RAM alone won't necessarily resolve your problem.

Crashplan is known to eat memory for breakfast lunch and dinner (btw, I believe Crashplan uses java so that's what Brit saw).

 

The short term way around it is to limit Crashplan docker memory usage so it doesn't end up eating other stuff.

In Docker template, click Advanced View then in the Extra Parameters box add


--memory=8G

(in the above example, it will limit the memory available to Crashplan to 8G - I actually had to use 8G because 4G crashes my Crashplan - just to demo how much memory Crashplan needs).

 

In the medium term, add more RAM so you have more available for other stuff.

 

In the long term, I would recommend moving away from Crashplan all together. They used to be great but no more.

Thank you for your response. I have done what you suggested. The weird thing is that the docker template for CrashPlan claims that it will only use a max of 1G if the memory section is left blank so this is weird.image.thumb.jpeg.24c55fe00a70e25b844299c4a0aa8a69.jpeg

image.png

Link to comment
5 hours ago, Ustrombase said:

Thank you for your response. I have done what you suggested. The weird thing is that the docker template for CrashPlan claims that it will only use a max of 1G if the memory section is left blank so this is weird.

It's not that box. It's the extra parameters box - shown when Advanced View is toggled.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.