Jump to content
jonp

Plex: Guide to Moving Transcoding to RAM

271 posts in this topic Last Reply

Recommended Posts

On 11/29/2017 at 2:56 AM, david279 said:

I had my UNRAID server freeze up a bit while using this. Was watching a football game for about 2 hours and the transcoding must have filled the ram up till there was no memory left. I have 32 GB of RAM but I had a VM running at the same time. unRAID started killing off Dockers till it finally killed Plex. Works great for movies, TV shows etc but be careful with the DVR.

Sent from my SM-G955U using Tapatalk
 

I'm experiencing the same thing. I have 32GB of RAM total, using 16GB for my primary VM. It took me a while to figure out what was causing this because it had been working flawlessly for so long. Then I started realizing that it only happened when I was watching a channel on DVR for a long period of time.  When it filled up the RAM it would lock the entire WebUI up (not that I would expect any less). I would then have to shut down the VM  in order to have enough RAM to restart some of the services.

 

Thanks for confirming others are having the same issue. I've moved my transcode directory back to SSD for now.

Share this post


Link to post

Under "Container Path" I added "/transcode" and under "Host Path", I added: "/tmp". Was that the right way to do it? 

Share this post


Link to post
21 hours ago, BendikHa said:

Under "Container Path" I added "/transcode" and under "Host Path", I added: "/tmp". Was that the right way to do it? 

 

That should be correct, also make sure that the "Transcoder temporary directory" in Plex is set to /transcode

Transcoding can require a lot of space and there might be risk of running out of RAM.

 

Share this post


Link to post
28 minutes ago, Leifgg said:

Transcoding can require a lot of space and there might be risk of running out of RAM.

Roughly how much ram is needed for this? I’m wondering if 8GB is enough?

Share this post


Link to post
47 minutes ago, wgstarks said:

Roughly how much ram is needed for this? I’m wondering if 8GB is enough?

 

Was just waiting for that one coming…. There are so many parameters involved so it’s hard to predict.

 

There are typically three reasons for transcoding, the video stream, audio stream and subtitles. In addition, transcoding might be needed due to the bit rate (to high). All above need to be within what the client player can handle so obviously it will be different for different clients.

 

The number of simultaneous clients playing can increase the required space needed as well.

 

It should also be noted that Plex does the transcoding “on the fly” in smaller data block and they are kept for a while (probably the whole session) so that you can stop playing and revers back in the movie and start playing again.

 

I haven’t checked actual file sizes for a transcoded video, but I believe that a “worst case scenario” would be the total size of the actual movie, but once again I never checked it myself.

 

My suggestion is that you start with transcoding to a folder first and play a full video and look at the files in the transcoder folder to get a feel for what it actually does when using different formats and clients. Let that decide the way to go and keep in mind that running out of memory eventually could crash Plex but potentially the server as well.

Edited by Leifgg

Share this post


Link to post
36 minutes ago, Leifgg said:

My suggestion is that you start with transcoding to a folder first and play a full video and look at the files in the transcoder folder to get a feel for what it actually does when using different formats and clients. Let that decide the way to go and keep in mind that running out of memory eventually could crash Plex but potentially the server as well.

Thanks. I’ll give this a shot since my server  maxs at 32GB of ram then if your worst case is true I wouldn’t be able to transcode many of my movies. I’m guessing almost all of the BD rips are over 32GB.

Share this post


Link to post
5 minutes ago, wgstarks said:

Thanks. I’ll give this a shot since my server  maxs at 32GB of ram then if your worst case is true I wouldn’t be able to transcode many of my movies. I’m guessing almost all of the BD rips are over 32GB.

 

I have 32 GB as well and a Windows 10 VM running (4 GB) as well as a few dockers and I am able to do transcoding to RAM but my Samsung TV plays most formats natively.

Also have remote friends but they have bandwidth limitations so files are likely to decrees in size after transcoding.

 

Share this post


Link to post

I have 8GB of RAM but only use plex.  I did this trick however it's useless for my needs.  

  

Even if I had 16GB RAM it wouldn't be enough for a single stream.  20mbps transcode is 18GB of data per 2 hours.  

  

I wish it would only store the latest 1GB and continually delete it.  When i'm halfway through a film, why do I need the first half stored in memory :(.  

Share this post


Link to post

There doesn't seem to be a need to worry about memory consumption. The Plex/Unraid combo seems to handle itself just fine.

 

This is my box configured with RAM transcoding working on a 13 Mbps video over 2 hours with only 8 GB memory.

MemUsage.PNG.cb46604dfbe8a8318ef30a9a818c02b5.PNG

Share this post


Link to post

Since doing this i keep getting: errors about low disk space unable to convert. I have 32 gb of ram and htop reports 22 gb in use. However when i look at my logs i see:  

 

Mar 17, 2018 16:50:33.422 [0x14c7793fd700] WARN - Low disk space: 176.4MB source file, 15.64GB capacity, 252.42MB available on "/transcode/Transcode/Sessions"

 

Which makes ZERO sense. anyone have any tips or ideas?

 

 

 

EDIT: I guess this no longer works without doing manual changes to the docker which will be  wiped with each update.

 

 

Capture.PNG

Capture2PNG.PNG

Edited by pyrater

Share this post


Link to post
On 5/20/2017 at 3:08 AM, mbicca said:


This is really odd, I have been using this for a long time, did you try running the commands manually? Any errors?

Like:
mkdir /ramdisk
mount -t tmpfs -o size=2048m tmpfs /ramdisk

 

This is my go file and the output of df -h which shows the /ramdisk of 2GB:

# cat /boot/config/go
#!/bin/bash
# Start the Management Utility
#ethtool -K eth0 tx off sg off tso off
/usr/local/sbin/emhttp &
mkdir /ramdisk
mount -t tmpfs -o size=2048m tmpfs /ramdisk

# df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs           12G  396M   12G   4% /
tmpfs            12G  248K   12G   1% /run
devtmpfs         12G   16K   12G   1% /dev
cgroup_root      12G     0   12G   0% /sys/fs/cgroup
tmpfs           128M  2.4M  126M   2% /var/log
/dev/sdd1       7.6G  587M  7.1G   8% /boot
tmpfs           2.0G     0  2.0G   0% /ramdisk
/dev/md1        2.8T  1.4T  1.4T  51% /mnt/disk1
/dev/md2        2.8T  279G  2.5T  10% /mnt/disk2
/dev/md3        2.8T   17M  2.8T   1% /mnt/disk3
/dev/md4        2.8T  748G  2.0T  27% /mnt/disk4
/dev/md5        2.8T  218G  2.6T   8% /mnt/disk5
/dev/sdc1       233G  121G  111G  53% /mnt/cache
shfs             14T  2.6T   12T  19% /mnt/user
/dev/loop0       30G 1018M   28G   4% /var/lib/docker
shm              64M     0   64M   0% /var/lib/docker/containers/dfc268df8c41735b6868324017ee4856f1dd9f1d6d78521101b81b126411eb03/shm
shm              64M  8.0K   64M   1% /var/lib/docker/containers/508dfe59ead3611622afde93e85ad50f77f796511ff80c2bc4e07a814c6ea626/shm
/dev/loop1      5.0G   18M  4.9G   1% /etc/libvirt

 

 

I was able to get it to work using this method. 

 

Share this post


Link to post

Odd. When I Transcode to Ram ITs only using up a few 1.8Mb at a time of my Ram until its viewed and then Plex drops in another couple of Meg. 

I'm viewing a 1080p File with AC3 Audio at 5Mbs. 

 

I'm only using 16Meg of Ram on my Setup and viewing it via the Plex App on my Windows machine. I haven't tried tracking the Ram usage on my AppleTV. I'll try it sometime. 

 

I also just tried a Full BluRay rip of Transformers at 30Mbs. I viewed it the same way as mentioned above. I didn't see more than 5megs of Space taken up in my Ram. 

Share this post


Link to post

Is anyone using the Plex docker from Plex themselves and able to use this?  I think I have it set up correctly but it appears to be just writing to the docker.img instead of /tmp.  

 

I have /transcode<-->/tmp set up in the docker and /transcode as the temp directory in the Plex application.  However, no files appear to ever be written to this directory but the docker image seems to be "modified" every minute a movie is playing.  Additionally if a movie plays long enough I get warnings that the docker image is taking up more and more space.

 

Does the "official" Plex docker not obey the settings or have I done something silly?

Share this post


Link to post

I ran this for a long time using the official Plex docker. It worked and never had a problem with the docker image filling. Now I had my ram fill up a couple times watching live TV. If you cd into the /tmp/ directory using the terminal run ls -lh to see if anything Plex appears. For some reason all plex's transcoding folders are hidden when in /tmp/.

Sent from my SM-G955U using Tapatalk

Share this post


Link to post

Hey all,

 

Please can you tell me how to see if the change is active in plex or unraid ?
Indeed I have the impression that it does not work for me.

 

Thank You.

Share this post


Link to post
On 4/7/2018 at 10:29 AM, rxnelson said:

Is anyone using the Plex docker from Plex themselves and able to use this?  I think I have it set up correctly but it appears to be just writing to the docker.img instead of /tmp.  

 

I have /transcode<-->/tmp set up in the docker and /transcode as the temp directory in the Plex application.  However, no files appear to ever be written to this directory but the docker image seems to be "modified" every minute a movie is playing.  Additionally if a movie plays long enough I get warnings that the docker image is taking up more and more space.

 

Does the "official" Plex docker not obey the settings or have I done something silly?

 

I"m running the Offical Plex Docker. 

I just changed the /transcode location in my Docker Template

from /mnt/cache/system/appdata/transcode

to /tmp 

Share this post


Link to post

Does transcoding to ram still work?

I am using the Official Plex Docker.

I'm asking because the Real Docker FAQ states "Note: Since about Plex version 0.9.14, transcoding to RAM does not work.  Recommend avoiding this option."

 

Edited by prtsmgr
Included I am using Official Plex Docker

Share this post


Link to post
On 3/8/2017 at 7:21 AM, uaeproz said:

 

 

I tried it but it didn't work:

 

root@Tower:/ramdisk# df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs           16G  389M   16G   3% /
tmpfs            16G  316K   16G   1% /run
devtmpfs         16G     0   16G   0% /dev
cgroup_root      16G     0   16G   0% /sys/fs/cgroup
tmpfs           128M  2.6M  126M   3% /var/log
/dev/sda1       7.5G  421M  7.1G   6% /boot
/dev/md1        7.3T  4.2T  3.1T  58% /mnt/disk1
/dev/md2        9.1T  4.9T  4.3T  53% /mnt/disk2
/dev/md3        7.3T  4.2T  3.2T  58% /mnt/disk3
/dev/md4        3.7T  2.7T  992G  74% /mnt/disk4
/dev/md5        3.7T  3.5T  211G  95% /mnt/disk5
/dev/md6        3.7T  2.6T  1.1T  70% /mnt/disk6
/dev/md7        3.7T  3.1T  646G  83% /mnt/disk7
/dev/md8        7.3T  5.2T  2.2T  71% /mnt/disk8
/dev/md9        3.7T  3.1T  565G  85% /mnt/disk9
/dev/md10       7.3T  5.0T  2.4T  69% /mnt/disk10
/dev/md11       7.3T  6.7T  634G  92% /mnt/disk11
/dev/md12       7.3T  4.8T  2.6T  66% /mnt/disk12
/dev/md13       3.7T   33M  3.7T   1% /mnt/disk13
/dev/md14       7.3T  2.8T  4.6T  38% /mnt/disk14
/dev/md16       3.7T   33M  3.7T   1% /mnt/disk16
/dev/md18       3.7T   33M  3.7T   1% /mnt/disk18
/dev/sdf1       943G   34G  896G   4% /mnt/cache
shfs             90T   53T   38T  59% /mnt/user0
shfs             91T   53T   38T  58% /mnt/user
/dev/loop0       10G  3.0G  6.1G  33% /var/lib/docker
tmpfs            16G     0   16G   0% /ramdisk
shm              64M  4.0K   64M   1% /var/lib/docker/containers/805bc16f4bbb9e448c9f29a3949db212b0575f66ec224946c32eaa08feca5a48/shm
 


Did you change your plex settings to use that path? It does look like your tmpfs mounted correctly under /ramdisk. Remember you need to change your Plex Settings to actually use it. Attached you can find how I setup up mine and then inside the Plex setup you set it to use the /transcode path.

Screen Shot 2018-04-27 at 5.42.54 PM.png

Share this post


Link to post

Does anyone know how to get Plex to only use the /tmp directory for on the fly transcodes (live streaming) and not for mobile sync data?  I have a lot of users who do mobile syncing and my RAM will max out fast if those converted files are being stored there until being downloaded by mobile clients.

 

EDIT:  Nvmd, I figured it out.  Just used a symlink and set /config/Library/Application Support/Plex Media Server/Cache/Transcode/Sessions to /mnt/ramdisk in my Plex container.  All set.

Edited by IamSpartacus

Share this post


Link to post

Transcoding to RAM is severely broken I feel. I think it works fine as long as you’re the only user but If someone else starts watching too, needing a transcode, they’ll get an error. Plex could fix this if they removed old chunks from the other stream  when another one starts watching. 

Share this post


Link to post
7 hours ago, nadbmal said:

Transcoding to RAM is severely broken I feel. I think it works fine as long as you’re the only user but If someone else starts watching too, needing a transcode, they’ll get an error. Plex could fix this if they removed old chunks from the other stream  when another one starts watching. 

 

I've not noticed this issue and I have 8-10 people on my server every night.  Are you sure you have adequate free RAM to use for transcoding?

Edited by IamSpartacus

Share this post


Link to post

I don't have quite as many people as @IamSpartacus, usually a few at a time is my max, but haven't had any major issues with RAM transcoding (I have /transcode in the docker mapped to /tmp and set in Plex server settings). It might matter how much actual memory you have in your system. Of course that is only going to work if you have enough to spare. I have 64 GB in my server.

 

What I have noticed that started over the past few weeks is I get tons of "buffering" messages coming from Tautilli, but nobody remote is really complaining to be about any problems so I have mostly been ignoring them for now.

Share this post


Link to post
15 hours ago, nadbmal said:

Transcoding to RAM is severely broken I feel.

 

As others have mentioned, I also have noticed no issues with RAM transcoding.  I only have, at most, 3-4 users accessing my server simultaneously and it does have 32GB RAM with a good amount free, so, perhaps I am not bumping up against any limits.

Edited by Hoopster

Share this post


Link to post
On 11/30/2018 at 2:56 PM, IamSpartacus said:

 

I've not noticed this issue and I have 8-10 people on my server every night.  Are you sure you have adequate free RAM to use for transcoding?

 

18 hours ago, Hoopster said:

 

As others have mentioned, I also have noticed no issues with RAM transcoding.  I only have, at most, 3-4 users accessing my server simultaneously and it does have 32GB RAM with a good amount free, so, perhaps I am not bumping up against any limits.

 

I have 16 GB, with around 30% in use typically (looking at the dashboard page in Unraid).

And all I know is that whenever I set /transcode/ <--> /tmp/ alot of stuff just breaks. I see users restarting movies constantly (I'm guessing they're getting a transcoder error). If I reboot then Plex  wouldn't start transcoding anything immediately because it wanted to remove some left over Session files but since they were in /tmp/ they weren't there so it complained about that (not really sure how it gets solved, it just starts working eventually):

 

Dec 01, 2018 16:55:25.314 [0x153c166d5800] ERROR - Transcoder: Error cleaning old transcode sessions: boost::filesystem::directory_iterator::construct: No such file or directory: "/transcode/Transcode/Sessions"

 

One of the most absurd things I saw was someone direct streaming a movie, and it looked like it cached the whole video in the RAM and then quit the stream (I saw the Tautulli buffer bar fill up really fast and then disappear, and then re-appear when he restarted the movie...)

 

Then whenever I just point it back at the SSD everything works wonderfully...

 

Share this post


Link to post
2 hours ago, nadbmal said:

Then whenever I just point it back at the SSD everything works wonderfully...

 

Any chance you have some bad RAM?  Have you noticed any other potential RAM-related issues or is it just Plex transcoding in RAM?

 

Even if it is the latter, a 24-hour memtest should eliminate or confirm RAM as the problem.

 

Perhaps you are seeing the issues because Plex is trying to move things around in RAM if some is bad.

 

This is a long shot, but, there does not appear to be an issue with transcoding in RAM in general.

Edited by Hoopster

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.