Out Of Memory errors detected on your server

[enmesh-parisian-latest]

Hey, fix common probs dumped me with the oft posted problem: 

Out Of Memory errors detected on your server

Any unraid masters have any thoughts on my diagnostics? I've had a number of issues recently so I suspect there may be something awry under the bonnet.

 

Looking through the forums as similar issues, I've followed the advice in the below post and installed the Tips and Tweaks plugin and made settings as follows:

Disk Cache 'vm.dirty_background_ratio' (%):    to 1%

Disk Cache 'vm.dirty_ratio' (%):                           to 2%

 

I have 32GB of RAM so the lower settings seemed more appropriate.

 

 

tobor-server-diagnostics-20180820-1553.zip

[trurl]

Lots of segfaults from deluge. Not sure OOM would be the cause though. Have you done a memtest lately?

 

What dockers are you running? Another thing people sometimes do is misconfigure a docker in such a way that it is trying to write directly to the /mnt folder or some other path that is not actually storage. All of the usual OS folders in unRAID are actually in RAM. The only places that have actual storage are:

 

/mnt/disk# and /mnt/cache (the actual disks)

/mnt/user and /mnt/user0 (the user shares)
/boot (the flash drive).

/mnt/disks (unassigned devices)

 

Writing into any other path is going to be filling up RAM. Also note that trying to write to an unassigned device path that is not actually mounted is also writing into RAM.

[enmesh-parisian-latest]

Thanks for the reply. Yes Deluge has been causing a lot of problems, typically when loading more than a few torrents at a time. It has been behaving particularly badly lately, I'd hoped that solving this OOM problem would help.

 

I ran a memtest for 24h when I last had an OOM error, no faults.

 

I've attached a screenshot of all dockers and their mappings, all settings are pretty standard as far as I can tell.

 

Any suggestions would be seriously appreciated, there are quite a few issues popping up recently and I suspect they're connected somehow...

 

EDIT: Thinking back, the only time I mapped a container to a path other than the /mnt/user/ path, was a few weeks ago when I ran a local musicbrainz server, the settings required the database be loaded directly to a disk rather than a user share. In that case I mapped it to /mnt/disk12/musicbrainz/ I've since deleted it though.

Edited August 21, 2018 by enmesh-parisian-latest

[BRiT]

You should check each application configuration to make sure they match the docker mappings you have setup.

[enmesh-parisian-latest]

3 minutes ago, BRiT said:

You should check each application configuration to make sure they match the docker mappings you have setup.

 

Is there a simple way to go about this? I assume what you mean is to make sure no settings inside the container refer to any path not mapped by docker? eg. looking in deluge settings I can see that the "download to:" path is fine (/media/ is mapped), however the "GeoIP Database" setting is mapped to /usr/share/GeoIP/GeoIP.dat; /usr/ is not mapped.

 

Is this an example which trurl mentioned which would fill up ram?

[Frank1940]

I believe your download path in the last screen shot should be      /mnt/user0/media/downloads/deluge   The location where you are putting these downloads is on a RAM disk that unRAID creates to store the OS and all of its support files.  

[enmesh-parisian-latest]

21 minutes ago, Frank1940 said:

I believe your download path in the last screen shot should be      /mnt/user0/media/downloads/deluge   The location where you are putting these downloads is on a RAM disk that unRAID creates to store the OS and all of its support files.  

 

Deluge container mappings for the media share are (screenshot below):

/media/

/mnt/user/media/

Cache is set to "no" for the media share. I'm not sure why I'd need the user0 path if cache is set to "no"

 

My understanding is that /media/downloads/deluge is a correct path inside the deluge container.

 

Edited August 21, 2018 by enmesh-parisian-latest

[BRiT]

Your Plex docker doesnt show a /transcode mapping but the process list shows pkex transcoding. But its odd that docker doesnt show as being full.

 

 

 

nobody   31297  190  2.1 1789004 698368 ?      Rl   15:04  93:12  |   |               \_ /usr/lib/plexmediaserver/Plex Transcoder -codec:0 h264 -codec:1 dca -i /movies/Monsters, Inc. (2001)/Monsters, Inc. (2001) Bluray-1080p.mkv -filter_complex [0:0]scale=w=1920:h=1040[0];[0]format=pix_fmts=yuv420p|nv12[1] -filter_complex [0:1] aresample=async=1:ocl='stereo':osr=48000[2] -map [1] -metadata:s:0 language=eng -codec:0 libx264 -crf:0 23 -maxrate:0 7402k -bufsize:0 14804k -r:0 23.975999999999999 -preset:0 veryfast -x264opts:0 subme=2:me_range=4:rc_lookahead=10:me=hex:8x8dct=1 -force_key_frames:0 expr:gte(t,0+n_forced*1) -map [2] -metadata:s:1 language=eng -codec:1 aac -b:1 112k -f dash -min_seg_duration 1000000 -skip_to_segment 1 -time_delta 0.0625 -manifest_name http://127.0.0.1:32400/video/:/transcode/session/jcgk80m4mwzo5fbf7u6rmszl/4031ba94-2c7e-4642-9f87-c1370b8dc30e/manifest -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 dash -start_at_zero -copyts -vsync cfr -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/jcgk80m4mwzo5fbf7u6rmszl/4031ba94-2c7e-4642-9f87-c1370b8dc30e/progress

[enmesh-parisian-latest]

10 minutes ago, BRiT said:

Your Plex docker doesnt show a /transcode mapping but the process list shows pkex transcoding. But its odd that docker doesnt show as being full.

 

 

 

nobody   31297  190  2.1 1789004 698368 ?      Rl   15:04  93:12  |   |               \_ /usr/lib/plexmediaserver/Plex Transcoder -codec:0 h264 -codec:1 dca -i /movies/Monsters, Inc. (2001)/Monsters, Inc. (2001) Bluray-1080p.mkv -filter_complex [0:0]scale=w=1920:h=1040[0];[0]format=pix_fmts=yuv420p|nv12[1] -filter_complex [0:1] aresample=async=1:ocl='stereo':osr=48000[2] -map [1] -metadata:s:0 language=eng -codec:0 libx264 -crf:0 23 -maxrate:0 7402k -bufsize:0 14804k -r:0 23.975999999999999 -preset:0 veryfast -x264opts:0 subme=2:me_range=4:rc_lookahead=10:me=hex:8x8dct=1 -force_key_frames:0 expr:gte(t,0+n_forced*1) -map [2] -metadata:s:1 language=eng -codec:1 aac -b:1 112k -f dash -min_seg_duration 1000000 -skip_to_segment 1 -time_delta 0.0625 -manifest_name http://127.0.0.1:32400/video/:/transcode/session/jcgk80m4mwzo5fbf7u6rmszl/4031ba94-2c7e-4642-9f87-c1370b8dc30e/manifest -avoid_negative_ts disabled -map_metadata -1 -map_chapters -1 dash -start_at_zero -copyts -vsync cfr -y -nostats -loglevel quiet -loglevel_plex error -progressurl http://127.0.0.1:32400/video/:/transcode/session/jcgk80m4mwzo5fbf7u6rmszl/4031ba94-2c7e-4642-9f87-c1370b8dc30e/progress

 

Good spotting, although I think that's standard practice for the plex container. I found this post which relates to your observation; Apparently writing to RAM is the desired function, although no the default. If I read that link correctly the (unmapped) standard setup transcodes into the docker.img file, which resides on my cache drive.

 

[enmesh-parisian-latest]

Not sure if this helps, but docker just locked up, pretty much unresponsive until I stopped all containers, even the docker UI page took about 30 secs to load. I attached the diagnostics.

tobor-server-diagnostics-20180821-1820.zip

[trurl]

9 hours ago, enmesh-parisian-latest said:

however the "GeoIP Database" setting is mapped to /usr/share/GeoIP/GeoIP.dat; /usr/ is not mapped.

 

Is this an example which trurl mentioned which would fill up ram?

 

Any location inside the application that is not mapped would be a location inside docker image which is on unRAID storage. I don't use deluge but transmission has something similar. It shouldn't take much space.

 

3 hours ago, enmesh-parisian-latest said:

Not sure if this helps, but docker just locked up, pretty much unresponsive until I stopped all containers, even the docker UI page took about 30 secs to load.

 

Nothing obvious. Maybe you could try recreating docker image and use Previous Apps feature of Community Applications.