16GB of RAM not enough?

[Januszmirek]

Not sure I picked the right section of the forum for this, if not please move it to the correct one.

So I run a rather standard set of docker containers and 2 vms that i very rarely turn on.

The issue I have is that even with 16g of ram which I thought would be plenty for my use, I do experience very high memory and cpu cores use and subsequently unraid and all containers not responding. This usually happens with Plex when multiple streams (3-4) are being played with  transcoding.

I do realize that linux is using all the available Ram and this is fine, I dont need to see free ram;) The issue is that with all these containers runnning, and Plex streaming 3-4 streams my ram is touching 70-80%. If I turn on any vm, the system practically freezes with RAM utilization around 100% and same for all CPU cores.

Now, I dont have any issue with upgrading my 16gb to 32gb. But I wonder if this would actually make any difference since Linux would use all available ram anyway, and even with ram upgrade I can get back to square one;(.

As far as plex transcoding set up, i guess its also pretty standard - transcode to ram with 60seconds buffer. I would like to keep transcode to ram to save up on disks wear.

Turning off transcode is not an option as many clients do not support all formats natively and hence need for transcode.

 

I observed that when I stop the array and start it again the 'idle' ram usage drops to 30% and stays around 50% for a couple of days, then gradually builds up to reach around 70-80% after a week or so when I start to experience issues described above. I also don't get why ram fill up immediately kills cpu utilization which in turn practically freezes the whole unraid server;( Well, not freezes but makes it barely responsive.

 

Ultimately my questions are:

1. Would an upgrade to 32gb from 16gb make a meaningful difference in my case or its a waste of money as its rather configuration issue, not hardware?

2. If not, what easy solutions I can utilise to manage ram more efficiently? Maybe a cron job to release ram every now and then? Or maybe there's a plugin that can handle this job?

Thank for help.

 

 

 

 

 

[Squid]

Based on the screenshot sans any further information, it would appear that Plex is transcoding via the CPU, and maxing out the cores is what you should expect to see.

3 hours ago, Januszmirek said:

gradually builds up to reach around 70-80%

You should post the diagnostics when it's in this situation

 

[Januszmirek]

Diagnostics attached.

27 minutes ago, Squid said:

Based on the screenshot sans any further information, it would appear that Plex is transcoding via the CPU, and maxing out the cores is what you should expect to see.

You should post the diagnostics when it's in this situation

It is hw transcode through internal igpu. It never maxes out cpu without maxing out ram first though, so i thought it is ram not cpu issue.

tower-diagnostics-20211231-1536.zip

[Squid]

The CPU isn't running anywhere near what the dashboard says, but rather it appears that you continually are having processes killed off due to out of memory issues and the dashboard can't make head or tails of what the current situation is.  

 

Looks like you've allocated ~3Gig to the Windows VM, but try and look at all of the docker apps.  Are they storing stuff in memory that they shouldn't be?  If you stop Plex does the memory issue immediately disappear? 

 

 

[Januszmirek]

3 minutes ago, Squid said:

The CPU isn't running anywhere near what the dashboard says, but rather it appears that you continually are having processes killed off due to out of memory issues and the dashboard can't make head or tails of what the current situation is. 

I don't understand what make out of it and where to start.

3 minutes ago, Squid said:

Looks like you've allocated ~3Gig to the Windows VM, but try and look at all of the docker apps.  Are they storing stuff in memory that they shouldn't be?  If you stop Plex does the memory issue immediately disappear?

 

Forget about the vm for a minute. I barely use vms. the issue described in original post happen regardless vm use.

Yes, if I stop Plex, no memory issues accur.

[Squid]

If there's no path mapping for /tmp in the Plex template, then it's transcoding directly to RAM within the container which has no upper limit on it's size.

 

If you've got it mapped to say /tmp/plex on the host then it has a 50% upper limit of memory.  Either way it would appear that Plex isn't clearing out the files and you should probably post in the applicable Plex thread to get more suggestions from other users who are heavy Plex users.

[Januszmirek]

11 minutes ago, Squid said:

If there's no path mapping for /tmp in the Plex template, then it's transcoding directly to RAM within the container which has no upper limit on it's size.

 

If you've got it mapped to say /tmp/plex on the host then it has a 50% upper limit of memory.  Either way it would appear that Plex isn't clearing out the files and you should probably post in the applicable Plex thread to get more suggestions from other users who are heavy Plex users.

Got it, thank you for pointing me in the right direction. My plex mapping for tmp pasted below:

 

I followed this post when set up transcoding location, maybe ive messed up sth;)

 

 

[itimpi]

Looking at your screenshots you have mapped /transcode to /tmp (which is in RAM rather than /mnt/user/tmp which the guide suggests which is a User share called ‘tmp’ on physical media.  If you have sufficient RAM then this might be OK to get best performance, but even then it should be a sub-folder of /tmp (e.g. /tmp/plex) to avoid issues.