Out of Memory errors - Plex transcoder culprit

[diabillic]

Hi everyone,

 

Hope someone can track down this memory leak issue in my linuxserver/plex container that just started happened a few weeks ago where I need to restart the container every few hours to get it back up due to it chewing up almost all 32GB of memory on the machine.  The CPU will also get completed pegged and pretty much lock up the entire system.

 

Running 6.8.3 with 32GB RAM on Unraid-Nvidia to support transcoding with a P2000.  I previously had 2xSSD in cache BTFRS that I discovered the horrendous excessive writing bug and broke them up into 1 cache drive and 1 drive in the array, both formatted as XFS.

 

Ive already tried the following (ill add more if there is something I may have left out):

- Transcode mapped to /tmp to RAMDisk

- Transcode mapped to a share on the array /mnt/user/local/transcode

- Transcode mapped to /dev/shm

- Without any transcode folder mapped
- Confirmed DLNA is not enabled in Plex

- Set memory limit on container to 6 or 8GB, eventually hits the cap and plex container becomes unresponsive until restarted

- Set vm.dirty_background_ratio to 1 and vm.dirty_ratio to 2

 

Since I have an rclone mount, I thought perhaps that was the case but the problem only starts to lurk once users start to transcode so I don't believe the issue is with rclone but its possible for sure.  Also I noticed that when transcode is mapped to either the cache or the array either disk once transcoding starts hits almost 150F and eventually will balloon in memory.  When idle the drives will idle around 90F.  I also noticed that the swap hits around 6-8GB but not sure if that's normal behavior inside Unraid or not.

I am also attaching a diag report since fix common problems of course reports out of memory errors as well.  

I've read quite a few threads on these forums and Plex forums and haven't been able to nail down the root cause yet so I'm hoping someone can point me in the right direction.  I'm pretty proficient on the linux side so I can execute/retrieve any info needed.  Thanks!!

colorcube-diagnostics-20200614-2228.zip

Edited June 15, 2020 by diabillic

[Pducharme]

did you try mounting your transcoding to /dev/shm ?  It is supposed to use a maximum of 50% of the RAM instead of all of the RAM.  I never tried it myself, but a friend is doing it on his Unraid without issue.

[diabillic]

1 hour ago, Pducharme said:

did you try mounting your transcoding to /dev/shm ?  It is supposed to use a maximum of 50% of the RAM instead of all of the RAM.  I never tried it myself, but a friend is doing it on his Unraid without issue.

Yes actually I have tried that, thanks for the reminder.  I'll add that bullet point in there...I've been poking around with this for a week or so and I knew there was going to be something I forgot to list in there that I've tried.

[mfwade]

One of my previous posts may be related... I am not using any GPU's nor am I using RAMDISK however, it sounds like the same issue you are having with the tmp dir. I created a share like the following

 

plex

=> transcode

=> tmp

 

Post:

I too have been having issues as of late. Plex sits arounf 423mb used then spikes to 7.4G then 14G, etc. I found that the /tmp dir was the culprit.

 

I found an article that states the developers are aware of it and is related to the recent  "Skip Intro” feature. That being said, it mentioned setting the following setting to never: (Settings - Server - Library - ‘Generate intro video markers’ set to “never” until this issue is fixed. I didn't set mine to never, rather i added an additional Path (below).

 

I also created an additional Path to /tmp that resides outside of the docker image, see below for my settings. I believe this will only need to be done temporarily until the real issue is resolved.

 

-MW

[diabillic]

very interesting...I have tried to mount the transcode folder to the array but the problem still occurs and the drive heats up to almost 150F.  I do in fact have generate intro video markers turned on and I do know that Plex recently rolled that feature out so that could very well be the case.  I'm going to disable that and see how it behaves.

Another thing I did after my post was created is switch the volume that mounts to my NAS from RW/Slave to RO/Slave.  After the OOM error i see some CIFS VFS messages so I figured I would mess with that but I have that same mapping to radarr, sonarr, lidarr and jellyfin with no issues on those containers...may not make a difference but it is something I just changed as well.

[diabillic]

UPDATE:  Another crash early AM but seems to have recovered itself:

 

Jun 16 06:31:05 ColorCube kernel: Killed process 18276 (Plex Media Serv) total-vm:39987080kB, anon-rss:28243632kB, file-rss:67744kB, shmem-rss:5420kB