Plex: Guide to Moving Transcoding to RAM

[alturismo]

after some tests also consider to test the apps how they behave when tmpfs gets full

 

sample, plex has a management and is working in a 80 - 90 % windows when filling up

 

emby on livetv filling to 100 % and stops/crashing the stream, only exit, restart works, wont continue even it tries to free up ...

emby on local media also filled up to 100 % ... playback runs a little longer (due not a live stream) but also stalls after a while ...

 

both emby tests with active throttle on transcode

 

so, plex, good solution, emby, not (yet) prolly by emby design

[alturismo]

2 minutes ago, mgutt said:

 

Is it needed? As /transcode does not exists, it should fallback to /tmp automatically.

 

emby falls back to /config/transcoding-temp ..

 

plex yes, it will use /tmp

[alturismo]

here a sample when emby stalls ... after 100 % fillrate in /tmp

 

stream stops, cant continue, only stop and restart stream.

 

[ich777]

8 minutes ago, mgutt said:

Is it needed? As /transcode does not exists, it should fallback to /tmp automatically.

Good question, you have to test it if the path is not defined in the template but I would recommend to set it to '/tmp' just in case someone let's the mount for it in the template...

[ich777]

43 minutes ago, alturismo said:

here a sample when emby stalls ... after 100 % fillrate in /tmp

 

stream stops, cant continue, only stop and restart stream.

This is eventually something for the Emby forums...

Wonder if Jellyfin does the same he thought and fired up his dev machine...

 

EDIT: Jellyfin does the same like Emby... Just ends if the transcoding cache is full, if you end the movie and go back to the main menu the transcoding cache is cleaned but that's also something for the Jellyfin Github issue tracker.

[mgutt]

On 2/1/2021 at 4:13 PM, mgutt said:

Is it needed? As /transcode does not exists, it should fallback to /tmp automatically.

 

Yes it is needed as /transcode will be created, even it is not mapped through the containers settings.

 

So I will hopefully update my guide for the last time ^^ EDIT: Done.

[DiscDuck]

Just had a movie not wanting to play on an iPad. Error not enough space to convert. Upped the ramsdisk from 4 to 6 GB and everything worked fine. Maybe this will help the next person running into this.

[Arby]

How would I check I have got this working correctly? I have entered /transcode within Plex. The last poster mentions upping the ramdisk from 4 to 6GB but how do I do that exactly?

 

[arturovf]

On 9/19/2020 at 6:22 AM, mgutt said:

By this guide Plex uses your RAM while transcoding which prevents wearing out your SSD.

 

Edit the Plex Container and enable the "Advanced View":

 

Add this to "Extra Parameters" and hit "Apply":

--mount type=tmpfs,destination=/tmp,tmpfs-size=4000000000

Result:

 

Side note: If you dislike permanent writes to your SSD add " --no-healthcheck ", too.

 

Now open Plex -> Settings -> Transcoder and change the path to "/tmp":

 

If you like to verify it's working, you can open the Plex containers Console:

 

Now enter this command while a transcoding is running:

df -h

 

Transcoding to RAM-Disk works if "Use%" of /tmp is not "0%":

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.8G  193M  3.7G   5% /tmp

 

After some time it fills up to nearly 100%:

tmpfs           3.8G  3.7G  164M  97% /tmp

 

And then Plex purges the folder automatically:

tmpfs           3.8G  1.3G  3.5G  33% /tmp

 

If you stop the movie Plex will delete everything:

tmpfs           3.8G  3.8G  0  0% /tmp

 

By this method Plex never uses more than 4GB RAM, which is important, as fully utilizing your RAM can cause an unexpected server behaviour.

 

Thank you for this, it seems to work in a quick test with 6 transcodes (3 4k and 3 1080p), will leave it this way for a while and see if everything keeps running smoothly

[Goldmaster]

hi there @binhex Im trying to do what others have done with transcoding to ram (which is located at /tmp ) so for the Container Variable: TRANS_DIR do i just type /tmp and make sure there is nothing in the transcode directory under plex advanced settings? I still seam to get quite a few reads on the cache drive and there is no plex transcode folder in the tmp folder. any ideas on how i can verify things are working correctly?

[mgutt]

27 minutes ago, Goldmaster said:

how i can verify things are working correctly?

Bottom of my post:

https://forums.unraid.net/topic/35878-plex-guide-to-moving-transcoding-to-ram/page/12/?tab=comments#comment-894460

[Goldmaster]

18 minutes ago, mgutt said:

Bottom of my post:

https://forums.unraid.net/topic/35878-plex-guide-to-moving-transcoding-to-ram/page/12/?tab=comments#comment-894460

thank you it does seam to be using the ram

 

Filesystem      Size  Used Avail Use% Mounted on
/dev/loop2       20G   17G  3.4G  83% /
tmpfs            64M     0   64M   0% /dev
tmpfs            63G     0   63G   0% /sys/fs/cgroup
shm              64M  4.0K   64M   1% /dev/shm
rootfs           63G   21G   43G  33% /transcode
shfs             17T   36G   17T   1% /media
tmpfs           3.8G  182M  3.6G   5% /tmp
/dev/loop2       20G   17G  3.4G  83% /etc/hosts
tmpfs            63G     0   63G   0% /proc/acpi
tmpfs            63G     0   63G   0% /sys/firmware

 

However, the tmpfs is only filled up by 5%. does that matter or quite what? Also the cache folder has appered in the plex appdata. when it should appear in ram. Or is that quite normal. any way to make the cache folder appear in tmp as well?

 

the part about --no-healthcheck does that mean to block using the ssd as transcoding or quite what? can you elaberate by your point? "If you dislike permanent writes to your SSD".

 

 

thank you for your help.

Edited October 10, 2021 by Goldmaster

[mgutt]

11 hours ago, Goldmaster said:

However, the tmpfs is only filled up by 5%

Look at the entry /transcode. You obviously missed to change the transcode dir in the Plex settings to /tmp.

 

11 hours ago, Goldmaster said:

the part about --no-healthcheck does that mean to block using the ssd as transcoding or quite what?

This has nothing to do with Transcoding:

https://forums.unraid.net/bug-reports/stable-releases/683-unnecessary-overwriting-of-json-files-in-dockerimg-every-5-seconds-r1079/

[IMTheNachoMan]

On 9/19/2020 at 7:22 AM, mgutt said:

By this guide Plex uses your RAM while transcoding which prevents wearing out your SSD.

 

 

I think I am missing something.

 

Isn't /tmp of unraid already in RAM? So wouldn't setting /transcode in container to /tmp in unraid already have Plex use RAM for transcoding? 

[mgutt]

7 hours ago, IMTheNachoMan said:

Isn't /tmp of unraid already in RAM? So wouldn't setting /transcode in container to /tmp in unraid already have Plex use RAM for transcoding? 

Yes, but it will use 100% of your RAM which could cause crashing your server. But /dev/shm would be an option. It uses "only" 50%.

[IMTheNachoMan]

4 hours ago, mgutt said:

Yes, but it will use 100% of your RAM which could cause crashing your server. But /dev/shm would be an option. It uses "only" 50%.

 

Got it. Thank you!

[Goldmaster]

is there also a way to get /plexmediaserver/cache folder to go also into the ram? please?

 

[IMTheNachoMan]

1 hour ago, Goldmaster said:

is there also a way to get /plexmediaserver/cache folder to go also into the ram? please?

 

Can't you just create a volume mapping in your Docker config? But won't you lose a lot of data if you reboot?

[awediohead]

On 9/19/2020 at 12:22 PM, mgutt said:

Edit the Plex Container and enable the "Advanced View":

 

Add this to "Extra Parameters" and hit "Apply":

--mount type=tmpfs,destination=/tmp,tmpfs-size=4000000000

Result:

 

Side note: If you dislike permanent writes to your SSD add " --no-healthcheck ", too.

Probably a dumb question but in my Plex Extra Parameters I have  --runtime=nvidia

Does that mean the transcoding 'cache' is happening in the GPU's RAM or that it's using the SSD?

I changed my Plex docker and Plex settings according to the OP and it seems to be working fine, and the dashboard shows it's using the GPU - but no idea how or even if I should be adding anything to Extra Parameters?

i.e. should I have:

 

--runtime=nvidia --mount type=tmpfs,destination=/tmp,tmpfs-size=4000000000

 

?

 

 

[mgutt]

55 minutes ago, awediohead said:

Does that mean the transcoding 'cache' is happening in the GPU's RAM

No

 

56 minutes ago, awediohead said:

should I have

If you change the transcoding dir in the Plex settings to /tmp, then yes.

[casperse]

On 9/19/2020 at 1:22 PM, mgutt said:

By this guide Plex uses your RAM while transcoding which prevents wearing out your SSD.

 

Edit the Plex Container and enable the "Advanced View":

 

Add this to "Extra Parameters" and hit "Apply":

--mount type=tmpfs,destination=/tmp,tmpfs-size=4000000000

Result:

 

Side note: If you dislike permanent writes to your SSD add " --no-healthcheck ", too.

 

Now open Plex -> Settings -> Transcoder and change the path to "/tmp":

 

If you like to verify it's working, you can open the Plex containers Console:

 

Now enter this command while a transcoding is running:

df -h

 

Transcoding to RAM-Disk works if "Use%" of /tmp is not "0%":

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.8G  193M  3.7G   5% /tmp

 

After some time it fills up to nearly 100%:

tmpfs           3.8G  3.7G  164M  97% /tmp

 

And then Plex purges the folder automatically:

tmpfs           3.8G  1.3G  3.5G  33% /tmp

 

If you stop the movie Plex will delete everything:

tmpfs           3.8G  3.8G  0  0% /tmp

 

By this method Plex never uses more than 4GB RAM, which is important, as fully utilizing your RAM can cause an unexpected server behaviour.

 

Hi @mgutt

 

I think you made some changes to your guide since last time. 😄

 

And I now have some issues after I enabled "download" option from Plex locally to a few family members iPad's they now get a error message stating:

 

Playback error - Not enough diskspace to convert this file/subject (Translating)

 

I then checked my RAM usage and it was +99% and in the "PlexRamScratch" folder  (I use your script to create two dirs on tmp one for Plex & Emby

 

PlexRamScratch Script runs at boot:
#!/bin/bash
mkdir /tmp/PlexRamScratch
chmod -R 777 /tmp/PlexRamScratch
mount -t tmpfs -o size=8g tmpfs /tmp/PlexRamScratch

 

My existing extra parameters PLEX:
--runtime=nvidia --no-healthcheck --log-opt max-size=50m --log-opt max-file=1 --restart unless-stopped

 

So I guess this would be my new extra parameters of PLEX?:
--runtime=nvidia --no-healthcheck --log-opt max-size=50m --log-opt max-file=1 --mount type=tmpfs,destination=/tmp/PlexRamScratch,tmpfs-size=8000000000 --restart unless-stopped

 

PLEX docker mount as before:
Container Path: /transcode --> /tmp/PlexRamScratch

 

Would this be enough to fix these new Download/transcode media files for local devices?

Thanks for spending the time to share all this info with everyone!

 

[mgutt]

31 minutes ago, casperse said:

So I guess this would be my new extra parameters of PLEX?:
--runtime=nvidia --no-healthcheck --log-opt max-size=50m --log-opt max-file=1 --mount type=tmpfs,destination=/tmp/PlexRamScratch,tmpfs-size=8000000000 --restart unless-stopped

No. By that you create the dir /tmp/PlexRamScratch inside of the container, which will be a ram disk inside of your container and as your plex server writes to /transcode, it won't use it. It would only use it, if you change /transcode to /tmp/PlexRamScratch in the transcode settings. By that you could delete all your other changes like the script and the mounted container path as everything happens inside the container then. Thats why I changed the manual as it became easier.

 

But finally this won't solve your problem. Your problem is that the download feature fills up the /transcode dir and as you limited it to 8GB, you can't solve this without making it much bigger.

 

But why does the download feature fill up the /transcode dir? Is this a new behavior? Is it possible to change the dir for download transcodes?

[casperse]

So basically I should just keep the existing solution? - no benefits but simplicity

(I like to see how much Plex/Emby uses of my memory)

 

I haven't used this sync/download feature for a very very long time (years) long before setting RAM up for transcoding

But I know Plex just released a major update on how it works now:

https://www.plex.tv/blog/going-off-grid-just-got-great/

 

I cant see a way to define a separate download path in Plex

So my only solution would be to upgrade my memory to 128GB 🤑 or move transcoding to my NVMe cache drive😖

[mgutt]

10 hours ago, casperse said:

So basically I should just keep the existing solution?

Yes. Finally you have the same result.

 

10 hours ago, casperse said:

So my only solution would be to upgrade my memory to 128GB 🤑 or move transcoding to my NVMe cache drive😖

It seems so. I tested it and Plex creates the complete movie file, before it is downloaded through the client. My RAM Disk is limited to 6GB and as you can see it is automatically removed after the file hits this limit:

 

This means we need 10 to 50GB per movie, depending on the size of the movie.

 

But there is another problem. Multiple downloads per user, creates at least two parallel transcodes:

 

I did not tested multiple users, but I think this will be even more. So we really need to be able to change the download transcoding path somehow or we are forced to use an SSD for all transcodes or install much more RAM 🙈

[casperse]

On 11/14/2021 at 9:43 AM, mgutt said:

Yes. Finally you have the same result.

 

It seems so. I tested it and Plex creates the complete movie file, before it is downloaded through the client. My RAM Disk is limited to 6GB and as you can see it is automatically removed after the file hits this limit:

 

I did not tested multiple users, but I think this will be even more. So we really need to be able to change the download transcoding path somehow or we are forced to use an SSD for all transcodes or install much more RAM 🙈

 

I came to the same conclusion I did a test of 3 streams and one download, and since I have installed and are running allot more dockers and never had a issue utilizing the max 8G for /tmp/PlexRamScratch it "almost" utilized the server 98%

 

I decided to disable all synch/download on the Plex server for now (I don't want to loose the gain I have using RAM)

Lets hope Plex creates a new path in the future for these synch/download conversions