Plex: Guide to Moving Transcoding to RAM

[mgutt]

4 hours ago, Bolagnaise said:

SWAP file plugin

A swap file is only used if Linux thinks, it's a good idea. And of course it would use the SSD (location of the swap file).

[Bolagnaise]

On 7/26/2022 at 1:42 AM, mgutt said:

A swap file is only used if Linux thinks, it's a good idea. And of course it would use the SSD (location of the swap file).

yep so not really viable,

 

Anyway, i submitted a feature request to try and resolve this. See you in 3 years when it gets implemented. 

 

https://forums.plex.tv/t/feature-request-ability-to-map-new-downloads-feature-transcodes-to-seperate-path/803203

[thedudeabides79]

Just want to share another use for this approach.

 

Found this post and thought it might solve issues I am having with plex unable to transcode certain types of audio codecs. I am reading it is likely a folder permissions issue or a linux limit on number of directories plex can monitor; all issues that involve FOLDER permissions or settings.

 

Moving the transcoding to the RAM appears to have solved this problem for me. Thanks so much.

[tmchow]

I think I'm having problems related to this as I have upgraded lots of movies to 4K HEVC HDR, and been consistently blowing my ram out I think.  In reading the last few pages of this thread, I'm guessing that's the issue here.

 

My system has 64GB of memory, but 16-20GB of it is dedicated to various VMs.  I assumed that the remaining was sufficient but i have lots of people streaming remotely from plex (so transcoding) and possibly downloading.

 

So I'm wondering what hte right thing to do here is as the last few pages left me confused.  Currently, my plex container has /transcode mapped to /tmp.

[Hoopster]

1 hour ago, tmchow said:

Currently, my plex container has /transcode mapped to /tmp.

This could be a problem.  I used to have mine set up this way but without limiting transcoding RAM in some way, tmp will eventually fill up all available RAM completely especially if there are multiple transcoding sessions going on simultaneously.

 

I also have 64GB RAM on my server but have limited transcoding to using 16GB RAM. 

 

I have this in my go file to limit transcoding to 16GB and to recreate the folder in RAM on reboot:

mkdir /tmp/PlexRamScratch
chmod -R 777 /tmp/PlexRamScratch
mount -t tmpfs -o size=16g tmpfs /tmp/PlexRamScratch

 

With this, the mapping in the Plex docker container is /transcode to /tmp/PlexRamScratch

 

This forces Plex to reclaim space by deleting older transcoded bits that have already played rather than just waiting to delete them all when the entire transcode finishes.

[tmchow]

16 minutes ago, Hoopster said:

This could be a problem.  I used to have mine set up this way but without limiting transcoding RAM in some way, tmp will eventually fill up all available RAM completely especially if there are multiple transcoding sessions going on simultaneously.

 

I also have 64GB RAM on my server but have limited transcoding to using 16GB RAM. 

 

I have this in my go file to limit transcoding to 16GB and to recreate the folder in RAM on reboot:

mkdir /tmp/PlexRamScratch
chmod -R 777 /tmp/PlexRamScratch
mount -t tmpfs -o size=16g tmpfs /tmp/PlexRamScratch

 

With this, the mapping in the Plex docker container is /transcode to /tmp/PlexRamScratch

 

This forces Plex to reclaim space by deleting older transcoded bits that have already played rather than just waiting to delete them all when the entire transcode finishes.

 

Thanks!  Out of curiosity what does this last line do?  Is it just putting a max size on the folder? I don't quite understsand all the components of the command.

[Hoopster]

1 hour ago, tmchow said:

 

Thanks!  Out of curiosity what does this last line do?  Is it just putting a max size on the folder? I don't quite understsand all the components of the command.

mount -t tmpfs -o size=16g tmpfs /tmp/PlexRamScratch

 

Mounts a RAM disk with a size of 16GB.  You can use less.  Some use as little as 4GB.  I am not dedicating much RAM to VMs so I have 16GB to spare for transcoding.

[DanielPT]

I did just try this but the CPU stills go way up?

 

 

 

 

Am i doing something wroung?

[Hoopster]

37 minutes ago, DanielPT said:

Am i doing something wroung?

Assuming you are trying to do Plex transcdoing,

 

What is your CPU? Does it have an iGPU or are you trying to use a discrete graphics card for transcoding?

If using an iGPU, do you have the iGPU specified as the primary graphics adapter in your BIOS?

What Plex docker container are you using?

For iGPU, Do you have a /dev/dri folder on you server with the proper contents for transcoding?

For iGPU, Have you specified /dev/dri as a device for the Plex container to use?

For discrete graphics card, have you setup NVIDIA-VISIBLE_DEVICES variable?

 

In other words, if you are using an iGPU have you done everything specified in this tutorial?

If using a discrete Nvidia graphics card, have you done everything in this tutorial?

Edited November 7, 2022 by Hoopster

[mgutt]

31 minutes ago, DanielPT said:

I did just try this but the CPU stills go way up?

Did you read the initial post?! This thread has absolutely nothing to do with CPU load.

[DanielPT]

Ahh okay  My CPU dosent have a iGPU. So i just want to move the transcode to RAM to save I/Os on the ssd

 

[kernelpanic]

On 10/13/2022 at 7:37 PM, Hoopster said:

This could be a problem.  I used to have mine set up this way but without limiting transcoding RAM in some way, tmp will eventually fill up all available RAM completely especially if there are multiple transcoding sessions going on simultaneously.

 

I also have 64GB RAM on my server but have limited transcoding to using 16GB RAM. 

 

I have this in my go file to limit transcoding to 16GB and to recreate the folder in RAM on reboot:

mkdir /tmp/PlexRamScratch
chmod -R 777 /tmp/PlexRamScratch
mount -t tmpfs -o size=16g tmpfs /tmp/PlexRamScratch

 

With this, the mapping in the Plex docker container is /transcode to /tmp/PlexRamScratch

 

This forces Plex to reclaim space by deleting older transcoded bits that have already played rather than just waiting to delete them all when the entire transcode finishes.

 

Thank you for this @Hoopster! Transcoding with Plex has been a nightmare for me. I've tried ten ways to Sunday to get transcoding to RAM to work. Then I tried using an unassigned device which was a pain in its own right. This has fixed it for me! Thanks again.

[Hoopster]

21 minutes ago, kernelpanic said:

 

Thank you for this @Hoopster! Transcoding with Plex has been a nightmare for me. I've tried ten ways to Sunday to get transcoding to RAM to work. Then I tried using an unassigned device which was a pain in its own right. This has fixed it for me! Thanks again.

Actually, credit for the idea goes to @mgutt.  I implemented it based on a suggestion in one of his posts. 

 

I was really running up against the /tmp problem because I also have Live TV & DVR implemented in Plex with HDHomeRun tuners.  If two simultaneous recordings of 1-hour+ OTA programs were happening, my server ran out of RAM due to /tmp filling up.  Plex DVR with HDHR can use up to 16GB per hour.

 

Irrespective of credit attribution, I am glad to hear this worked for you. 

Edited November 16, 2022 by Hoopster

[DanielPT]

Can i somehow "confirm" that im running the transcode from ram in Unraid?

 

Thanks!

[Hoopster]

On 11/24/2022 at 1:06 AM, DanielPT said:

Can i somehow "confirm" that im running the transcode from ram in Unraid?
 
Thanks!

Search in your designated transcode location when you think transcoding should be taking place. Files will be created under the Transcode/Sessions folder.

Edited November 25, 2022 by Hoopster

[SirCadian]

On 10/14/2022 at 12:37 AM, Hoopster said:

This could be a problem.  I used to have mine set up this way but without limiting transcoding RAM in some way, tmp will eventually fill up all available RAM completely especially if there are multiple transcoding sessions going on simultaneously.

 

I also have 64GB RAM on my server but have limited transcoding to using 16GB RAM. 

 

I have this in my go file to limit transcoding to 16GB and to recreate the folder in RAM on reboot:

mkdir /tmp/PlexRamScratch
chmod -R 777 /tmp/PlexRamScratch
mount -t tmpfs -o size=16g tmpfs /tmp/PlexRamScratch

 

With this, the mapping in the Plex docker container is /transcode to /tmp/PlexRamScratch

 

This forces Plex to reclaim space by deleting older transcoded bits that have already played rather than just waiting to delete them all when the entire transcode finishes.

I'm thinking about implementing transcoding in RAM for plex but also run HD Homerun and suspect i'll run into RAM issues.  I'm a relative linux newbie but have been running with Unraid for a year and a half at this point so my knowledge is improving.  I'm not a fan of implementing stuff without understanding what it's doing.  I get what all the commands above do and how to map the temp ram drive in Plex. What I don't understand is, what is a "go" file?  A quick google tells me it sits in /boot/config for Unraid.  There's not a right lot in mine at the moment.  Is it essentially just a shell script that runs on boot?

Also, once I've edited the go file, I assume I either need to reboot my unraid server or manually type in those commands before I'll be able to map the drive in Plex?

Thanks for the help.

[alturismo]

11 minutes ago, SirCadian said:

I'm thinking about implementing transcoding in RAM for plex but also run HD Homerun and suspect i'll run into RAM issues.

how you come to this conclusion ?

 

Plex will free up its drive as soon space gets low, sadly emby or jellyfin doesnt work like this, they will run and then just stop when the transcode location is full, a known issue there ...

 

11 minutes ago, SirCadian said:

Also, once I've edited the go file, I assume I either need to reboot my unraid server or manually type in those commands before I'll be able to map the drive in Plex?

exactly, you could also just use a simple userscript and run at 1st startup on array only as sample

[SirCadian]

2 hours ago, alturismo said:

how you come to this conclusion ?

 

Plex will free up its drive as soon space gets low, sadly emby or jellyfin doesnt work like this, they will run and then just stop when the transcode location is full, a known issue there ...

 

exactly, you could also just use a simple userscript and run at 1st startup on array only as sample

I mean if I don't use the tmpfs suggestion above.  I've got 32Gb RAM which is usually around 30-40% utilised.  My understanding is Plex won't clean up until the transcode location is full.  If the transcode location is /tmp that's effectively my RAM without limitation?  Seems much more sensible to me to create a ram drive of fixed size.

 

I like the idea of using a user-script.  I think I'll do it that way as it's visible in the UI.  Thanks for the suggestion.

Edited January 14, 2023 by SirCadian

[alturismo]

Just now, SirCadian said:

My understanding is Plex won't clean up until the transcode location is full.  If the transcode location is /tmp that's effectively my RAM without limitation?  Seems much more sensible to me to create a ram drive of fixed size.

your understanding is then wrong  Plex does clean up, but actually only on the edge then, so yes, your RAM would be pretty much filled up indeed ... just make 1 3 GB temp disk and you should be good, may 3.5, that was my limit until Plex may moan on True HD audio streams (whyever ...)

[SirCadian]

9 minutes ago, alturismo said:

your understanding is then wrong  Plex does clean up, but actually only on the edge then, so yes, your RAM would be pretty much filled up indeed ... just make 1 3 GB temp disk and you should be good, may 3.5, that was my limit until Plex may moan on True HD audio streams (whyever ...)

Cool.  I'll give it a punt.  Thanks.

[mgutt]

But note: If someone uses the download feature, 3 to 4GB won't be enough as a download creates one huge transcoded file. That's why my RAM disk is now 60GB (4GB left for unRAID). 

 

[SirCadian]

20 minutes ago, mgutt said:

But note: If someone uses the download feature, 3 to 4GB won't be enough as a download creates one huge transcoded file. That's why my RAM disk is now 60GB (4GB left for unRAID). 

 

Good to know.  That might be an issue for me.  I tend to use download so that I can watch films on flights etc.  Think I'll need to do some testing with that as I "only" have 32Gb of RAM with about 10Gb of that in use.  I was planning on a 10 - 16Gb RAM drive.

[gustomucho]

On 1/14/2023 at 9:51 AM, mgutt said:

But note: If someone uses the download feature, 3 to 4GB won't be enough as a download creates one huge transcoded file. That's why my RAM disk is now 60GB (4GB left for unRAID). 

 

Interesting!

What happens if you limit yourself to 4GB and for example you try to locally download a 8GB transcoded movie to a device?

[jvorn]

On 10/13/2022 at 6:37 PM, Hoopster said:

This could be a problem.  I used to have mine set up this way but without limiting transcoding RAM in some way, tmp will eventually fill up all available RAM completely especially if there are multiple transcoding sessions going on simultaneously.

 

I also have 64GB RAM on my server but have limited transcoding to using 16GB RAM. 

 

I have this in my go file to limit transcoding to 16GB and to recreate the folder in RAM on reboot:

mkdir /tmp/PlexRamScratch
chmod -R 777 /tmp/PlexRamScratch
mount -t tmpfs -o size=16g tmpfs /tmp/PlexRamScratch

 

With this, the mapping in the Plex docker container is /transcode to /tmp/PlexRamScratch

 

This forces Plex to reclaim space by deleting older transcoded bits that have already played rather than just waiting to delete them all when the entire transcode finishes.

 

This is great but I have a question, is this a UserScript that runs for you on startup or something? Thanks!!!

[tmchow]

4 minutes ago, jvorn said:

 

This is great but I have a question, is this a UserScript that runs for you on startup or something? Thanks!!!

 

put it in your go file.