20250217 update (spam logging and some cleaner file handling with http chars)

 

so, i made another test now and looking all ok herem, so please update and then ma report issues with specials ...

 

here my test scene (dont wonder, i abused the /boot here now as sample test, from cache > boot )

 

Feb 17 08:51:55 AlsServerII root: cache-mover: msm: INFO: /mnt/user/Media/RECORD/TVRIPS/F1 ᴸᶦᵛᵉ/Season 0 +- [äöü] - {12345}/F1 ᴸᶦᵛᵉ - E1382 - GP Katar_ Sprint +- [äöü] - {12345}.mkv
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Number of files: 2 (reg: 1, dir: 1)
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Number of created files: 1 (reg: 1)
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Number of deleted files: 0
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Number of regular files transferred: 1
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Total file size: 856.59M bytes
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Total transferred file size: 856.59M bytes
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Literal data: 856.59M bytes
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Matched data: 0 bytes
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: File list size: 0
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: File list generation time: 0.001 seconds
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: File list transfer time: 0.000 seconds
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Total bytes sent: 856.67M
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: Total bytes received: 38
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: 
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: sent 856.67M bytes  received 38 bytes  190.37M bytes/sec
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: total size is 856.59M  speedup is 1.00
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: INFO: /mnt/cache/Media/RECORD/TVRIPS/F1 ᴸᶦᵛᵉ/Season 0 +- [äöü] - {12345}/F1 ᴸᶦᵛᵉ - E1382 - GP Katar_ Sprint +- [äöü] - {12345}.mkv
Feb 17 08:51:59 AlsServerII root: cache-mover: msm: open file F1 ᴸᶦᵛᵉ - E1382 - GP Katar_ Sprint +- [äöü] - {12345}.mkv will be switched boot/Download now
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: INFO: /boot/Download/Media/RECORD/TVRIPS/F1 ᴸᶦᵛᵉ/Season 0 +- [äöü] - {12345}/F1 ᴸᶦᵛᵉ - E1382 - GP Katar_ Sprint +- [äöü] - {12345}.mkv
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: ready to switch live
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: -: No such file or directory.
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: (gdb) Attaching to process 14518
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 3440938]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 2684245]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 1356143]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 273928]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 271749]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 25247]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 17943]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 17777]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 14521]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [New LWP 14519]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [Thread debugging using libthread_db enabled]
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: Using host libthread_db library "/lib64/libthread_db.so.1".
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: 0x0000148c3f1e1c4f in __futex_abstimed_wait_cancelable64 ()
Feb 17 08:52:00 AlsServerII root: cache-mover: msm:    from /lib64/libc.so.6
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: (gdb) $1 = 27
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: (gdb) $2 = 26
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: (gdb) $3 = 0
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: (gdb) Detaching from program: /usr/libexec/unraid/shfs, process 14518
Feb 17 08:52:00 AlsServerII root: cache-mover: msm: [Inferior 1 (process 14518) detached]

 

looking all fine, and checking real file access before / after, also all as it should be ...

 

root@AlsServerII:/usr/local/emhttp/plugins/cache-mover# ls -alrt /proc/[0-9]*/fd/* 2> /dev/null | grep -i ".mkv"
lr-x------ 1 root   root    64 Feb 17 08:41 /proc/14518/fd/26 -> /mnt/cache/Media/RECORD/TVRIPS/F1\ ᴸᶦᵛᵉ/Season\ 0\ +-\ [äöü]\ -\ {12345}/F1\ ᴸᶦᵛᵉ\ -\ E1382\ -\ GP\ Katar_\ Sprint\ +-\ [äöü]\ -\ {12345}.mkv*
lr-x------ 1 nobody users   64 Feb 17 08:51 /proc/1521549/fd/4 -> /mnt/user/Media/RECORD/TVRIPS/F1\ ᴸᶦᵛᵉ/Season\ 0\ +-\ [äöü]\ -\ {12345}/F1\ ᴸᶦᵛᵉ\ -\ E1382\ -\ GP\ Katar_\ Sprint\ +-\ [äöü]\ -\ {12345}.mkv*
root@AlsServerII:/usr/local/emhttp/plugins/cache-mover# ./scripts/cache_mover1
root@AlsServerII:/usr/local/emhttp/plugins/cache-mover# ls -alrt /proc/[0-9]*/fd/* 2> /dev/null | grep -i ".mkv"
lr-x------ 1 root   root    64 Feb 17 08:51 /proc/14518/fd/26 -> /boot/Download/Media/RECORD/TVRIPS/F1\ ᴸᶦᵛᵉ/Season\ 0\ +-\ [äöü]\ -\ {12345}/F1\ ᴸᶦᵛᵉ\ -\ E1382\ -\ GP\ Katar_\ Sprint\ +-\ [äöü]\ -\ {12345}.mkv*
lr-x------ 1 nobody users   64 Feb 17 08:52 /proc/1521549/fd/4 -> /mnt/user/Media/RECORD/TVRIPS/F1\ ᴸᶦᵛᵉ/Season\ 0\ +-\ [äöü]\ -\ {12345}/F1\ ᴸᶦᵛᵉ\ -\ E1382\ -\ GP\ Katar_\ Sprint\ +-\ [äöü]\ -\ {12345}.mkv*
root@AlsServerII:/usr/local/emhttp/plugins/cache-mover#

 

so i tested with special characters here now on Mediaserver Mode, all ok ... please report back if there are any further issues when using 20250217 version.

@alturismo updated to the latest version and tested without any other changes, it worked perfectly this time. Thanks again for all your work on this plugin, as someone who (unfortunately for now) has their server in the same room as the tv, this plugin makes the viewing experience so much better.  

additionally, as i cross checked emby / jelly, there was a small culprit in mediaserver mode with special chars too, solved now too with 20250218 versioning ... 

 

so i hope now all "special char" users are happy too

Hello mate, great work, thank you so much.

The new version works like a charm in only mode, I tested multiple movies and shows with secial caracters either in file or folder name.

Edited February 18, 20251 yr by simatra

Is there a quick guide for setting up Simple Mover? I'd love to populate a new pool for Plex with some files to improve performance in Plex.

2 hours ago, tehg said:

Is there a quick guide for setting up Simple Mover? I'd love to populate a new pool for Plex with some files to improve performance in Plex.

actually i dont see the connection in your question, may as note

 

simple mover, what is it about ... 

 

its more or less my "mover tuning" replacement ... made more or less for Media (well, it can handle any files ...)

 

sample, my Shares layout

 

 

as, as i want to keep new Media files"longer" on my cache drive, there is Option 1, sample

 

 

simple explained

 

1/ from cache pool

2/ watch given Shares (Media/VCD Media/TVRIPS Media/.... ....)

3/ move to array user0

4/ but only once we have a fillrate > 85 %

5/ free down to < 60 % (so only move ~ 25 % and then stop)

6/ sorted by aged folders (last accessed)

7/ but must be min 20 days old

8/ and exclude given rules if wanted (sample, ARR apps sub folders if existent)

 

so, its running like setted N and checking how the fillrate is and once it hits the 85 % it ll move oldest files first until we are free again down to 60 %

 

from the logs as sample

 

Feb 20 03:10:01 AlsServerII root: simple-mover: sorting order is age, oldest 1st
Feb 20 03:10:02 AlsServerII root: simple-mover: 261G in queue to move when limits are reached, min 85% fillrate on cache
Feb 20 03:10:02 AlsServerII root: simple-mover: min. 20 days old to reach 60% treehold on cache
Feb 20 03:10:02 AlsServerII root: simple-mover: /mnt/cache used (72%), min limit is (85%), nothing todo

 

so im still under the 85 and nothing to move, IF it would move now, there are 261 GB in queue ... (~ 12 %)

 

and also a overview what would be moved ...

 

 

so the queue list will raise  and extend as higher the fillrate is or as higher the distance is between fillrate and free down rate and so on ... so its a "filtered moving then"

 

---

 

now, there is a Option 2 (basic), sample

 

 

as we see here, its way more simple, here those subfolders are only matched by age, difference to general setup

 

here, anything older then 2 days in /Media/GAMES as sample will be moved to array anyhow

 

so just a basic age filter NOT to move directly ...

 

---

 

as i dont want (well, i probably will include it as option in next release) to replace the Unraid mover or mover tuning, this runs "side by side" with mover tuning plugin, simple as, this plugin will exclude anything in mover tuning plugin which is handled by simple mover plugin

 

 

 

here i just add "my" simple mover folders to mover tuning exclusions so they dont "collide"

 

next release, it ll be optional and simple mover can run without mover tuning as i optionally disable the onboard mover ...

 

so simple mover will then be the only one needed (if wanted)

 

about running cycle, i have 2 different schedules, for 1/ and 2/

 

 

so i check every 3 hours the fillrate for extended filtered moving and only once a day for basic moving ...

 

---

 

now backk to your question  this plugin is to move FROM pool to some HDD array ... not vice vers, or are you looking for cache mover which will "cache" Media from HDD to a lets say SSD pool disk ... thats a different story

Hi guys,

 

Im running into a problem with cache mover.

Copying from array to cache works perfectly however the files are not being cleaned. Files that are set to keep 24 hours are not removed. In cache logger it says "3 still in range" in screenshot the movie "crank" has been watched 3 days ago.

 

What am i doing wrong?

 

Edited February 22, 20251 yr by ChrisNij

8 minutes ago, ChrisNij said:

What am i doing wrong?

basically its looking ok

 

may show me the report of the film folder (i check for last access)

 

sample here "Access: 2025-02-21 22:45:03.683420505 +0100"

 

root@AlsServerII:~# stat /mnt/cache/Media/TVRIPS/Supernatural/
  File: /mnt/cache/Media/TVRIPS/Supernatural/
  Size: 40              Blocks: 0          IO Block: 4096   directory
Device: 259,1   Inode: 1169372     Links: 4
Access: (0777/drwxrwxrwx)  Uid: (   99/  nobody)   Gid: (  100/   users)
Access: 2025-02-21 22:45:03.683420505 +0100
Modify: 2025-02-09 07:05:01.395430447 +0100
Change: 2025-02-21 22:45:03.683420505 +0100
 Birth: 2024-07-06 19:05:55.701512016 +0200
root@AlsServerII:~#

 

33 minutes ago, alturismo said:

basically its looking ok

 

may show me the report of the film folder (i check for last access)

 

sample here "Access: 2025-02-21 22:45:03.683420505 +0100"

 

root@AlsServerII:~# stat /mnt/cache/Media/TVRIPS/Supernatural/
  File: /mnt/cache/Media/TVRIPS/Supernatural/
  Size: 40              Blocks: 0          IO Block: 4096   directory
Device: 259,1   Inode: 1169372     Links: 4
Access: (0777/drwxrwxrwx)  Uid: (   99/  nobody)   Gid: (  100/   users)
Access: 2025-02-21 22:45:03.683420505 +0100
Modify: 2025-02-09 07:05:01.395430447 +0100
Change: 2025-02-21 22:45:03.683420505 +0100
 Birth: 2024-07-06 19:05:55.701512016 +0200
root@AlsServerII:~#

 

Thanks for your quick response!

 

root@NAS:~# stat /mnt/ssd_cache/films/"Crank (2006)"
  File: /mnt/ssd_cache/films/Crank (2006)
  Size: 3               Blocks: 79         IO Block: 16384  directory
Device: 0,37    Inode: 8           Links: 2
Access: (0755/drwxr-xr-x)  Uid: (   99/  nobody)   Gid: (  100/   users)
Access: 2025-02-22 03:27:19.088967354 +0100
Modify: 2025-02-04 22:11:30.607928231 +0100
Change: 2025-02-19 17:22:04.394548804 +0100
 Birth: 2025-02-19 16:42:03.582171137 +0100

 

 

Looks like for some reason the file has been accessed at night, now the great search to what accesses it begins...

18 minutes ago, ChrisNij said:

Looks like for some reason the file has been accessed at night, now the great search to what accesses it begins...

exactly

 

you see in my logs while playback when cache mover is updating access times while playing

 

sorry, but i have no control on other apps accessing ... and thats the most reliable time i can fetch in my point of view.

On 2/19/2025 at 8:06 PM, alturismo said:

actually i dont see the connection in your question, may as note

 

simple mover, what is it about ... 

 

its more or less my "mover tuning" replacement ... made more or less for Media (well, it can handle any files ...)

 

sample, my Shares layout

 

---

 

now backk to your question  this plugin is to move FROM pool to some HDD array ... not vice vers, or are you looking for cache mover which will "cache" Media from HDD to a lets say SSD pool disk ... thats a different story

 

This was perfect. The piece I wasn't sure on was the syntax for folders and what I needed to fill in manually. Thanks a ton!

On 2/22/2025 at 4:27 PM, ChrisNij said:

Looks like for some reason the file has been accessed at night, now the great search to what accesses it begins...

may i ask if you figured what was the cause ? in terms which app did change the access time.

16 hours ago, alturismo said:

may i ask if you figured what was the cause ? in terms which app did change the access time.

Not yet, it only happens around 3.30. To figure out which app is causing the access time to change, i have to disable each individual app at night and check access time in the morning :').

Let you know as soon as i find out which one is causing this.

Hello, I discovered that there's a new option to use HTTPS with the Cache Mover plugin. Besides setting the Media Server URL (Type) to HTTPS, do I need to do anything else? I tested it using:

curl -s "http://192.168.10.10:32400/status/sessions?X-Plex-Token=kjdscvioubcdksajcb"

but nothing was returned. Then I tried:

curl -k "https://192.168.10.10:32400/status/sessions?X-Plex-Token=kjdscvioubcdksajcb"

and that worked. Also, /usr/local/emhttp/plugins/cache-mover/scripts/cache_mover2 doesn't display anything when using HTTPS.

Edited March 2, 20251 yr by shoyrock

1 hour ago, shoyrock said:

do I need to do anything else?

not that im aware off, i dont use enforce https in LAN  added this feature on call

 

1 hour ago, shoyrock said:

curl -s "http://192.168.10.10:32400/status/sessions?X-Plex-Token=kjdscvioubcdksajcb"

but nothing was returned.

may check your network settings in Plex (1st entry)

 

14 minutes ago, alturismo said:

not that im aware off, i dont use enforce https in LAN  added this feature on call

 

may check your network settings in Plex (1st entry)

 

I'm using port forwarding so I have secure connections set to Required 

7 hours ago, shoyrock said:

I'm using port forwarding so I have secure connections set to Required

assuming you mean you have your plex Server directly "open", i see 3 options

 

1/ put your services behind a reverse proxy, then this is obsolete and you can disable secure ... (should be done anyway )

2/ enter your domain instead LAN IP as address

3/ allow LAN addresses without auth

 

sample from 2 + 3

 

 

2/ external domain here ...

3/ LAN addresses need no auth (Network settings tab buttom), you can also enter only 1 direct IP

 

10 hours ago, alturismo said:

assuming you mean you have your plex Server directly "open", i see 3 options

 

1/ put your services behind a reverse proxy, then this is obsolete and you can disable secure ... (should be done anyway )

2/ enter your domain instead LAN IP as address

3/ allow LAN addresses without auth

 

sample from 2 + 3

 

 

2/ external domain here ...

3/ LAN addresses need no auth (Network settings tab buttom), you can also enter only 1 direct IP

 

I allow LAN addresses without auth but that didn't help I'll try the reverse proxy and see if that works 

1 hour ago, shoyrock said:

I allow LAN addresses without auth but that didn't help I'll try the reverse proxy and see if that works 

may rather try your external domain then as "easy" workaround ?

 

and setup your reverse Proxy with time ...

23 minutes ago, alturismo said:

may rather try your external domain then as "easy" workaround ?

 

and setup your reverse Proxy with time ...

I'm using Plex with a VPN so I did not have any need to set up a reverse Proxy port forwarding is done via the VPN. I'm trying to set a reverse Proxy with Duckdns and Nginx Proxy Manager but I'm running into problems 

1 hour ago, shoyrock said:

I'm using Plex with a VPN so I did not have any need to set up a reverse Proxy port forwarding is done via the VPN. I'm trying to set a reverse Proxy with Duckdns and Nginx Proxy Manager but I'm running into problems 

if you use a VPN ... then its a total different story and also, you shouldnt need to port forward to plex and then there is no need for https access only ... seems i missunderstand here something

2 hours ago, alturismo said:

if you use a VPN ... then its a total different story and also, you shouldnt need to port forward to plex and then there is no need for https access only ... seems i missunderstand here something

Even though I'm using a VPN, I still need to use port forwarding; otherwise, I won't have direct access to the server, and it will use the Plex relay. Also, if I set "Secure Connections" to "Preferred," doesn't that allow outside connections to connect to the server insecurely?

1 hour ago, shoyrock said:

Even though I'm using a VPN, I still need to use port forwarding; otherwise, I won't have direct access to the server, and it will use the Plex relay.

then your setup is wrong somewhere, nevermind now, you know what you doing.

 

1 hour ago, shoyrock said:

Also, if I set "Secure Connections" to "Preferred," doesn't that allow outside connections to connect to the server insecurely?

depends what you call "insecure", it can be htto only instead https ...

but has nothing todo with user / pass auth which is needed so or so ... 

12 minutes ago, alturismo said:

then your setup is wrong somewhere, nevermind now, you know what you doing.

 

depends what you call "insecure", it can be htto only instead https ...

but has nothing todo with user / pass auth which is needed so or so ... 

ok ill just keep using HTTP for now 

On 2/25/2025 at 4:40 AM, alturismo said:

may i ask if you figured what was the cause ? in terms which app did change the access time.

I'm starting to lose my mind :')

I can't figure out what is accessing the movie file, I disabled all apps en restarted them again. Now there is no access anymore around 3.30 at night. Now the files still won't be removed beacause they have been accessed but now the acces time is the same as when cache mover is running, so it looks like chache mover is changing te last access time.

 

 File: /mnt/ssd_cache/films/Crank (2006)
  Size: 3               Blocks: 79         IO Block: 16384  directory
Device: 0,36    Inode: 8           Links: 2
Access: (0755/drwxr-xr-x)  Uid: (   99/  nobody)   Gid: (  100/   users)
Access: 2025-03-04 08:05:01.694020223 +0100
Modify: 2025-02-04 22:11:30.607928231 +0100
Change: 2025-02-19 17:22:04.394548804 +0100
 Birth: 2025-02-19 16:42:03.582171137 +0100

 

 

Mar  4 08:05:01 NAS root: cache-cleaner: /mnt/ssd_cache/films/Crank (2006) 0 still in range