Video Preloader (avoids HDD spinup latency when starting a Movie or Episode through Plex, Jellyfin or Emby)

[mathiasdoe]

Hey, I cant get anything to acutally preload to the ram. If i run the script with the standard values it tells me that no file has been preloaded, "as loading only needed 0.001 seconds". If i lower the preload threshold to 0.0001 seconds it tells me that it successfully preloaded the files in 0,001 seconds which seems quite fast to me. Trying to play one of the preloaded files in jellyfin results in buffering and the disks spinning up.
My video path is "/mnt/disk1/media", I have tried different paths aswell with the same results. Any idea on how to fix this?

[mgutt]

16 minutes ago, mathiasdoe said:

"/mnt/disk1/media

This is the path you used in the container settings? It's important to use /mnt/user or /mnt/diskX or /mnt/poolname exactly as you use it in the container.

[VideoVibrations]

Anyone noticed it doesn’t work as well with Apple TV?  

[zoro25]

On 1/22/2023 at 6:18 AM, mgutt said:

This is the path you used in the container settings? It's important to use /mnt/user or /mnt/diskX or /mnt/poolname exactly as you use it in the container.

I have the same issue as the user above.  It seems to preload only my subs and not my media. 

2023-01-25 04:40:14 Skipped /mnt/user/Media/Movies/Alien Covenant (2017)/Alien Covenant (2017).mkv as loading needed only 0.000
2023-01-25 04:40:14 Skipped /mnt/user/Media/TV Shows/Star Trek- Discovery/Season 4/Star Trek - Discovery - S04E10 - The Galactic Barrier.mkv as loading needed only 0.000
2023-01-25 04:40:14 Preloaded /mnt/user/Media/TV Shows/Star Trek- Discovery/Season 4/Plex Versions/Optimized for TV/Star Trek_ Discovery/S04E01.219441.eng.srt
2023-01-25 04:40:14 Preloaded /mnt/user/Media/TV Shows/Star Trek- Discovery/Season 4/Plex Versions/Optimized for TV/Star Trek_ Discovery/S04E02.227138.eng.srt

When you say use the same as the container settings are you talking about Plex container?.

If so my plex container media path is /mnt/user and I also set that as my media path in the script.

Edited January 26, 2023 by zoro25

[mgutt]

Execute the following to empty the cache:

 

free -h && sync && echo 1 > /proc/sys/vm/drop_caches && free -h

After that execute the script again

 

[Archangel]

I'm also experienceing this.  I've ran the empty cache code and still I'm getting everything skipped as it was loaded within 0.0001 seconds even though it's content that has been added since the script last ran.

[zoro25]

Same as Archangel . 

I run the empty cache code and I can see my cache is now empty. 
 

root@NewNas:~# free -m
               total        used        free      shared  buff/cache   available
Mem:          128592        5731       58698        1087       64161      120550
Swap:              0           0           0
root@NewNas:~# free -h && sync && echo 1 > /proc/sys/vm/drop_caches && free -h
               total        used        free      shared  buff/cache   available
Mem:           125Gi       5.6Gi        57Gi       1.1Gi        62Gi       117Gi
Swap:             0B          0B          0B
               total        used        free      shared  buff/cache   available
Mem:           125Gi       5.5Gi       118Gi       1.1Gi       2.0Gi       117Gi
Swap:             0B          0B          0B

 

But after running that I still get the message about skipping all video files and the script only preloading subs. 


They are not in ram as I just cleaned it and also when I start the show my disk spins up and nothing happens for about 6 seconds. 

Edited January 26, 2023 by zoro25

[mgutt]

Yes the script is broken. Will fix it....

[mgutt]

On 1/26/2023 at 7:38 PM, mgutt said:

Yes the script is broken. Will fix it....

Script is fixed. Released version 1.4 in the first post.

[zoro25]

On 2/12/2023 at 2:44 AM, mgutt said:

Script is fixed. Released version 1.4 in the first post.

Works perfectly for me. 
Thank you ever so much 🙂

[mathiasdoe]

Thank you very much!

[grants169]

Would it be possible to have a user variable that would load to RAM files OLDER than X days?

I have cache drive setup to only move files if they are older than 20 days and if cache is more than 70% full.  Doing this on a 1TB cache drive essentially re-creates the idea behind this script except pulling recent video files off nvme instead of RAM, eliminating spin ups for recent files already.   Without an "older than" definition, the script is loading all the files off my nvme instead of HDD.  Thanks for the consideration.

[mgutt]

2 hours ago, grants169 said:

Would it be possible to have a user variable that would load to RAM files OLDER than X days?

Search for "-size" and replace it against "-mtime +20 -size"

[grants169]

16 hours ago, mgutt said:

Search for "-size" and replace it against "-mtime +20 -size"

Perfect - Thank You!

[Mx986]

@mgutt Just wanted to say thanks for this wonderful script!

[orlando500]

Hi, the script has worked great for a long time here. But now i get errors when i run it:

 

available RAM in Bytes: 5537747353
Amount of Videos that can be preloaded: 60
/tmp/user.scripts/tmpScripts/preload plex/script: line 55: syntax error near unexpected token `head'
/tmp/user.scripts/tmpScripts/preload plex/script: line 55: ` benchmark=$(time ( head -c "$preload_head_size" "$file" ) 2>&1 1>/dev/null )'

[mgutt]

2 hours ago, orlando500 said:

But now i get errors when i run it:

It seems you are using an old version.

[orlando500]

On 3/31/2023 at 10:56 AM, mgutt said:

It seems you are using an old version.

yep was using 1.0. 

I notice 1.4 version checks if my disks are to fast and therefore do not load files into memory. Would you say there is no point in using preloading when the cache for the media library is on SSD?

 

[mgutt]

22 minutes ago, orlando500 said:

and therefore do not load files into memory.

It does load the files into the memory, but the obtained status is wrong. You could compare the loading time between ram and disk and change the preload_threshold="0.150" value.

 

24 minutes ago, orlando500 said:

Would you say there is no point in using preloading when the cache for the media library is on SSD?

Yes. Doesn't really make sense to. SSDs wake up in milliseconds. An HDD needs up to 10 seconds.

[Jurak]

Ok noticed that I have not been getting any notifications for a while. Updated to the latest 1.4 and still not getting anything. The logs are just this below.

Script Starting Apr 11, 2023 07:20.08

Full logs for this script are available at /tmp/user.scripts/tmpScripts/Plex Precache/log.txt

Script Finished Apr 11, 2023 07:20.08

Full logs for this script are available at /tmp/user.scripts/tmpScripts/Plex Precache/log.txt

 

Its this over and over the only difference is the date and time.

 

Any help would be great.

[mgutt]

22 hours ago, Jurak said:

Updated to the latest 1.4 and still not getting anything.

I assume your script died somehow and left a temp file in /tmp which avoids its execution. Add this line to the settings section of your script and remove it after you executed the script once:

 

rmdir "/tmp/${0///}"

 

This will remove the temp file. Usually the script does this automatically on exit. The file avoids parallel execution of the script 

[Jurak]

2 hours ago, mgutt said:

rmdir "/tmp/${0///}"

 

This will remove the temp file. Usually the script does this automatically on exit. The file avoids parallel execution of the script 

That fixed it. Thanks I appreciate it.

[DJ-BrianC]

Just installed version 1.4 and as you can see from the attachment it still doesn't work. It says they are loaded but then on the next run it loads them all over again with zero skipped. If they're already loaded they should have been skipped right? Also assuming this ever works correctly how does it decide which video to preload? Is it random newly added? What is the thought process here.

[mgutt]

1 hour ago, DJ-BrianC said:

Also assuming this ever works correctly how does it decide which video to preload?

It measures the loading time and if it is higher than 150ms it assumes it was not loaded from RAM:

# if a preload is faster than 0.150 seconds than the video is already preloaded to the RAM
preload_threshold="0.150"

 

 

It would be strange if your RAM would be slower than that. You could manually check this by spinning down the disk and execute the script. As long the data is in the RAM it won't spin up the disk.

 

[DJ-BrianC]

My drives are rarely spun down there's usually activity going on. Do you recommend I change this value?