1 hour ago, Masterwishx said:

In last version was removed override for cache:prefer array->cache. 

And moving from array to cache only upto 95%. 

this here is a regular Share as far as i see, so not the "weird" cache prefer Share ...

 

 

1 hour ago, Masterwishx said:

@alturismo do you think we need to add override for cache prefer with some options? 

I remember we decided to move to cache only... 

override, nope, move to cache only, yes  so i wouldnt suggest any change ...

 

here its just weird as it looks like the /data Share is handled as

 

cache < array (cache prefer)

 

while his setup is

 

cache > array (standard)

 

so there must be some conflict on his sys ...

26 minutes ago, alturismo said:

may post a screen from your shares folder from your flash, not that there is a second (old) config from data Share

 

Thank you for continued support on this!
Here is the screen from the config/shares location. There is a data.cfg but it does not have mover-tuning related settings. Any other suggestions?

2 hours ago, styagi said:

Here is the screen from the config/shares location. There is a data.cfg but it does not have mover-tuning related settings. Any other suggestions?

 

yep, looking all fine from my point of view ... then im actually out of ideas, sorry ...

 

my last approach would be, delete the mover tuning plugin, install it fresh again, make your settings and see what comes as reporting in logs on this data Share ... 

2 hours ago, styagi said:

Any other suggestions?

You can set override for some share temporary this will couse reapply global config after this share config reading... 

36 minutes ago, Masterwishx said:

You can set override for some share temporary this will couse reapply global config after this share config reading... 

didnt thought about that  very good point ...

 

@styagi like @Masterwishx mentioned, may set a override like you wish it (main setup) and test then please.

I've been following this thread for a while now but can't seem to make sense of my issue.

Here are the settings for my shares:

• appdata: Cache <- Array
• backups: Backup <- Array
• data: Buffer -> Array
• domains: Cache <- Array
• isos: Cache -> Array
• system: Cache <- Array

Cache and Backup are both pools. If I understand how this is intended to function correctly, there should be really no difference between this and the normal move function for cache prefer shares (appdata, backups, domains, system). For cache yes shares (data and isos), I do not have anything overridden so I would have expected that the global settings of 50%/25% would have been used but it doesn't appear that they are. Am I missing something?

 

08:34:36.859 ***************************************************** Mover Tuning Plugin version 2025.03.11 ****************************************************
08:34:36.863 ----------------------------------------------------------------- Global settings ---------------------------------------------------------------
08:34:36.865 Using global (cache) moving threshold: 50 %
08:34:36.867 Using global (cache) freeing threshold: 25 %
08:34:36.871 Age: Automatic (smart caching)
08:34:36.872 After script: /mnt/user/appdata/scripts/moverafter.sh
08:34:36.874 Before script: /mnt/user/appdata/scripts/moverbefore.sh
08:34:36.881 Enable Turbo: yes
08:34:36.884 Skip file list: /mnt/user/appdata/scripts/moverignore.sh
08:34:36.894 Test Mode: yes
08:34:36.896 Notify: errorsOnly
08:34:36.899 Debug Logging: no
08:34:36.900 Validate Filenames: yes
08:34:36.907 Launching before script: /mnt/user/appdata/scripts/moverbefore.sh
08:34:38.136 Before script finished
08:34:38.140 ***************************************************************** FILTERING FILES ***************************************************************
08:34:38.144 ----------------------------------------------------------- Processing [appdata] share ----------------------------------------------------------
08:34:38.157 Primary storage: cache - size: 932GiB - used:  74 % (679GiB)
08:34:38.158 Secondary storage: user0
08:34:38.176 Share Information: Name: appdata - Path: /mnt/*/appdata
08:34:38.179 Moving threshold: 0% (0B) ; Filling up threshold: 95% (885GiB)
08:34:38.180 Mover action: user0->cache (cache:prefer). Pool is below priming threshold percentage:  74% < 95%.
08:34:38.181 => Will smart move newest files from user0 to cache until threshold.
08:36:01.100 Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-03-18T083436.list for appdata
08:36:01.104 ----------------------------------------------------------- Processing [backups] share ----------------------------------------------------------
08:36:01.117 Primary storage: backup - size: 466GiB - used:  53 % (245GiB)
08:36:01.118 Secondary storage: user0
08:36:01.121 Share Information: Name: backups - Path: /mnt/*/backups
08:36:01.124 Moving threshold: 0% (0B) ; Filling up threshold: 95% (443GiB)
08:36:01.124 Mover action: user0->backup (cache:prefer). Pool is below priming threshold percentage:  53% < 95%.
08:36:01.125 => Will smart move newest files from user0 to backup until threshold.
08:36:01.179 Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-03-18T083436.list for backups
08:36:01.183 ------------------------------------------------------------- Processing [data] share -----------------------------------------------------------
08:36:01.196 Primary storage: buffer - size: 1.9TiB - used:   1 % (513MiB)
08:36:01.196 Secondary storage: user0
08:36:01.200 Share Information: Name: data - Path: /mnt/buffer/data
08:36:01.202 Moving threshold: 0% (0B) ; Freeing threshold: 95% (1.8TiB)
08:36:01.203 Mover action: buffer->user0 (cache:yes). Pool is above moving threshold percentage:    1% >= 0%.
08:36:01.204 => Will smart move old files from buffer to user0. Nothing will be moved from user0 to buffer
08:36:01.206 Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-03-18T083436.list for data
08:36:01.210 ----------------------------------------------------------- Processing [domains] share ----------------------------------------------------------
08:36:01.222 Primary storage: cache - size: 932GiB - used:  74 % (679GiB)
08:36:01.223 Secondary storage: user0
08:36:01.226 Share Information: Name: domains - Path: /mnt/*/domains
08:36:01.229 Moving threshold: 0% (0B) ; Filling up threshold: 95% (885GiB)
08:36:01.230 Mover action: user0->cache (cache:prefer). Pool is below priming threshold percentage:  74% < 95%.
08:36:01.230 => Will smart move newest files from user0 to cache until threshold.
08:36:01.233 Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-03-18T083436.list for domains
08:36:01.237 ------------------------------------------------------------- Processing [isos] share -----------------------------------------------------------
08:36:01.249 Primary storage: cache - size: 932GiB - used:  74 % (679GiB)
08:36:01.250 Secondary storage: user0
08:36:01.254 /mnt/cache/isos does not exist. Is the share still used? Consider removing /boot/config/shares/isos.cfg if not.
08:36:01.255 => Skipping
08:36:01.259 ------------------------------------------------------------ Processing [system] share ----------------------------------------------------------
08:36:01.270 Primary storage: cache - size: 932GiB - used:  74 % (679GiB)
08:36:01.271 Secondary storage: user0
08:36:01.275 Share Information: Name: system - Path: /mnt/*/system
08:36:01.277 Moving threshold: 0% (0B) ; Filling up threshold: 95% (885GiB)
08:36:01.278 Mover action: user0->cache (cache:prefer). Pool is below priming threshold percentage:  74% < 95%.
08:36:01.278 => Will smart move newest files from user0 to cache until threshold.
08:36:01.281 Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-03-18T083436.list for system
08:36:01.697 ************************************************************ ANALYSING MOVING ACTIONS ***********************************************************
08:36:01.776 Deciding the action (move/sync/keep) for each file. There are 572799 files, it can take a while...
08:42:57.035 No new files will be moved/synced from primary to secondary
08:42:57.036 No new files will be moved/synced from secondary to primary
08:42:57.037 Launching after script: /mnt/user/appdata/scripts/moverafter.sh
08:42:57.116 After script finished with result: OK
08:42:57.117 Cleaning lock and stop files
08:42:57.121 ****************************************************************** WE ARE DONE ! ****************************************************************

 

On 3/17/2025 at 12:33 PM, Masterwishx said:

You can use befor script option in plugin then

I cannot do this.. because there's more to the script after the Mover is done. I basically pause torrents, run mover, when mover is done resume paused torrents.

 

Being able to just disable the schedule is a must in my eyes, not sure if it's a bug or intended behavior that that setting doesn't stop the Mover.

5 minutes ago, iBoolGuy said:

I cannot do this.. because there's more to the script after the Mover is done. I basically pause torrents, run mover, when mover is done resume paused torrents.

 

Being able to just disable the schedule is a must in my eyes, not sure if it's a bug or intended behavior that that setting doesn't stop the Mover.

There's an after mover script as well.

5 hours ago, iBoolGuy said:

cannot do this.. because there's more to the script after the Mover is done. I basically pause torrents, run mover, when mover is done resume paused torrents.

You can use also after script 

9 hours ago, Ichthus said:

Am I missing something?

For cache prefer we have move only from array to cache until 95% fill up, 

Maybe I can add global setting for cache prefer fill up 0-100% and with some override options also @alturismo is it good idea,sorry I know we don't like this cache item? 

18 hours ago, alturismo said:

may set a override like you wish it (main setup) and test then please.

I can't find why this bug happened but maybe I can add option to force read global config when no override flag? 

the mover suddenly stopped working for me.. test run shows no issues, but when run to move files from share on ssd to array it just does nothing..
test:
/mnt/download_pool/4K
Mover tuning plugin done!

real move:
Mar 19 08:54:01 Plex move: init_debugging: 0

nothing happens..

reinstalled the mover plugin as well..

edit:
mover start in cli however did work.. so now its moving stuff.. no idea why pressing the start button did nothing.

 

edit2:

i take that back.. it looked like it for a moment... but its actually not doing anything...

Edited March 19, 20251 yr by furian

12 minutes ago, furian said:

the mover suddenly stopped working for me.. test run shows no issues, but when run to move files from share on ssd to array it just does nothing..

Without mover logs debug log it's impossible to know whats wrong 

59 minutes ago, Masterwishx said:

Without mover logs debug log it's impossible to know whats wrong 

09:04:23.047 ***************************************************** Mover Tuning Plugin version 2025.03.11 ****************************************>
09:04:23.055 ----------------------------------------------------------------- Global settings --------------------------------------------------->
09:04:23.060 Using global (cache) moving threshold: 80 %
09:04:23.062 Using global (cache) freeing threshold: 40 %
09:04:23.069 Age: 15
09:04:23.088 Notify: no
09:04:23.091 Debug Logging: yes
09:04:23.094 Validate Filenames: yes
09:04:23.112 ***************************************************************** FILTERING FILES ***************************************************>
09:04:23.120 -------------------------------------------------------------- Processing [4K] share ------------------------------------------------>
09:04:23.152 Primary storage: download_pool - size: 1.9TiB - used:  93 % (1.7TiB)
09:04:23.154 Secondary storage: user0

nothing more in the logs... (this was from the mover_tuning log).

Summary log is empty and only Filtered_files log remain..

edit: when trying to stop the mover i got this
mover stop
/usr/local/sbin/mover.old: line 123: kill: (3881860) - No such process
mover: stopped

also when i try to enable the normal mover logging i get this:
Mar 19 08:57:54 nginx: 2025/03/19 08:57:54 [error] 2291550#2291550: *4162441 connect() to unix:/var/run/emhttpd.socket failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.0.100, server: , request: "POST /update.htm HTTP/1.1", upstream: "http://unix:/var/run/emhttpd.socket:/update.htm", host: "server", referrer: "http://server/Settings/Scheduler"

 

might have found the issue... the recycle bin plugin was stuck.. trying to reboot wont even work.. so forcing it off might be the only way.. i'll post here when trying again.

 

well turned out that was it.. nice

Edited March 19, 20251 yr by furian

2 hours ago, Masterwishx said:

Maybe I can add global setting for cache prefer fill up 0-100% and with some override options also @alturismo is it good idea,sorry I know we don't like this cache item? 

i could say now i told you this will be a "mess"

 

now, lets think about it overall

 

cache prefer (cache < array) Shares, goal is (and always been) to keep the files always on cache when there is free space, so your currrent approach is absolutely correct to say 95 % as max target, move all files from cache prefer Shares from array to cache drive.

 

in sum, it should be only a emergency setup for sys relevant Shares ... but ...

 

now, lets see what i thought what will happen, it ll be a ping pong game if there are users abusing this for Shares which are not meant to be used for ... or not considering what will happen on fillup or not running the mover often enough to keep the cache at least free enough for sys Shares and so on ...

 

so i personally would still say, either set your sys Shares to cache only (with Backup Strategy), you should be aware how much space they take approximately ... and set your cache yes Shares decent, then you never should run in any issues ...

 

consider your fillup flow, how much could be in a worse scenario come in in 1 hour, how fast can mover move over from cache to array, how much do i need to free to not run full at all while considering on which treehold i start up ..

 

sample, cache 2 tb

 

/system /appdata /domains << cache only

 

are cache only and using ~ 400 GB (20 %)

 

/download /Media /whatever << cache yes

 

my worst cace fillup would be now downloading or uploading to "cache yes" with 100 MB/s, which results in ~ 350 GB data /h ...

 

so when i now just simple set my mover tuning, run every hour ...

 

fillup treehold 80% (400 GB left), target run down to 50% (1 tb left), sorted by age ... and may set min age like 10 - 20 days, depending on your personal wishes ... you never should collide with anything and never should need cache prefer Shares at all.

 

... which woll only trigger then > 80%

 

now, what seems to happen, /Media /Download Shares gets abused with cache prefer setup, whyever ... and are in sum way larger then they should be and are not meant to be cache prefer as they are even higher in size then cache total ... will end up in bad results and are not really compatible with mover tuning strategy ... those user shouldnt use mover tuning, my personal oppinion

 

when you now consider to use the trehholds from regular setup for cache prefer  ... it will be even more ping pong overall

 

so in the end, you know my opinion  i would support freeing on 1 side cache yes Shares and perm fill up on the other side with cache prefer Shares ... live with it if you want a setup like this or think about it what you do, would be my answer

1 hour ago, alturismo said:

when you now consider to use the trehholds from regular setup for cache prefer  ... it will be even more ping pong overall

 

so in the end, you know my opinion  i would support freeing on 1 side cache yes Shares and perm fill up on the other side with cache prefer Shares ... live with it if you want a setup like this or think about it what you do, would be my answer

When I add an override to the data share using the same settings as the global configuration, the functionality works as I’d expect. This leaves me wondering about the purpose of the global settings if they don’t automatically apply to cache yes shares without an override. I feel like I might be missing their intended role.

 

I’m also unsure about the plugin’s intended use case if it’s not designed for this exact scenario. I’m not trying to be sarcastic—I genuinely don’t grasp the point. This is precisely how the old plugin functioned for me, so I’m struggling to see the difference.

 

On a related note, my data share is a cache yes share that operates on a separate pool from everything else, so there’s no risk of it consuming space needed for a cache prefer share. Plus, my downloader is configured to pause if the mover falls behind and the pool gets too full.

24 minutes ago, Ichthus said:

I’m also unsure about the plugin’s intended use case if it’s not designed for this exact scenario. I’m not trying to be sarcastic—I genuinely don’t grasp the point. This is precisely how the old plugin functioned for me, so I’m struggling to see the difference.

 

On a related note, my data share is a cache yes share

 

im not talking about cache yes Shares ... thats the purpose of this plugin of course

 

im talking about the combination cache yes (moving from cache >> array), using mover tuning already to keep files on demand longer on cache via fillrate treehold, via age, ... AND ALSO running cache prefer Shares (moving from array >> cache) and wondering why these combinations may collide

 

so you  are absolutely right, standard settings shouldnt have any issues and you shouldnt need to override in the Shares itself ... im pretty sure @Masterwishx will solve this as it seems to collide in some cases while there are also cache prefer Shares in the Game.

 

My (personal) intention is to make clear, it makes no sense to expect cache prefer Shares running the same scenario as cache yes Shares ... 2 different stories behind them with 2 different bahaviours and different goals ... especially in combination with mover tuning which will hold files on cache already on cache yes Shares as you set them up by your own rules

1 hour ago, alturismo said:

 

im not talking about cache yes Shares ... thats the purpose of this plugin of course

 

im talking about the combination cache yes (moving from cache >> array), using mover tuning already to keep files on demand longer on cache via fillrate treehold, via age, ... AND ALSO running cache prefer Shares (moving from array >> cache) and wondering why these combinations may collide

 

so you  are absolutely right, standard settings shouldnt have any issues and you shouldnt need to override in the Shares itself ... im pretty sure @Masterwishx will solve this as it seems to collide in some cases while there are also cache prefer Shares in the Game.

 

My (personal) intention is to make clear, it makes no sense to expect cache prefer Shares running the same scenario as cache yes Shares ... 2 different stories behind them with 2 different bahaviours and different goals ... especially in combination with mover tuning which will hold files on cache already on cache yes Shares as you set them up by your own rules

Thank you for the clarification. I believe I have a better grasp of this now.

 

I agree that trying to perfectly balance "cache yes" and "cache prefer" shares might be an overly ambitious goal, given the countless possible scenarios. It’s probably more practical for users to handle this on a case-by-case basis, tailored to their specific needs. I’m currently managing this by keeping my "cache yes" and "cache prefer" in completely separate pools.

 

Regarding the "cache yes" not working as expected (i.e., not picking up the global settings), it seems like this is a recognized issue.

 

Please let me know if I’ve gotten anything wrong.

 

Thanks again for the explanation and all your hard work!

Is there a way to disable the mover schedule but keep the mover options/filters?

When I enable "Disable Mover running on a schedule:" it seems to turn off the plugin.
 

I have a script that runs the mover everyday and I don't want the mover to run on a schedule without the script.

42 minutes ago, Triiiple said:

Is there a way to disable the mover schedule but keep the mover options/filters?

When I enable "Disable Mover running on a schedule:" it seems to turn off the plugin.
 

I have a script that runs the mover everyday and I don't want the mover to run on a schedule without the script.

You could probably delete /config/plugins/dynamix/mover.cron on the flash drive (then from a command prompt)

update_cron

but that file will get regenerated if you make any changes to Settings - Schedules (presumably)

1 hour ago, Ichthus said:

Please let me know if I’ve gotten anything wrong.

 

not that im aware on

 

1 hour ago, Ichthus said:

Regarding the "cache yes" not working as expected (i.e., not picking up the global settings), it seems like this is a recognized issue.

 

i would say yes, but please keep in mind, its all up to @Masterwishx as he is the one who trys to fullfill all wishes here

 

you may could help him by opening or adding in github issues with

 

mover tuning log

share config files (or screens)

 

to may better understand where the issue could be related to ...

 

1 hour ago, Ichthus said:

Thanks again for the explanation and all your hard work!

so all credits for mover tuning to him please

im just trying to help advising and may clear up some scenarios which are discussed here ...

 

im not involved in his development here, so he is the man in charge 👍

 

On 3/18/2025 at 2:56 AM, Masterwishx said:

You needs to set clear folders,clear dataset to Yes

Thanks @Masterwishx!   I was just like a dog going - I think this is my life now hahaha.  One last question but where is this setting?  I thought it would be under mover but its not and its not in scheduler.   

Edited March 19, 20251 yr by cbr600ds2
can't find settings

6 hours ago, alturismo said:

when you now consider to use the trehholds from regular setup for cache prefer  ... it will be even more ping pong overall

I meaned to make global threshold for cache prefer like now fill up upto 95% but users maybe want change it? 

3 hours ago, Ichthus said:

Regarding the "cache yes" not working as expected (i.e., not picking up the global settings), it seems like this is a recognized issue.

If you still have this issue please open issue in github I will try to solve it 

5 hours ago, Ichthus said:

This leaves me wondering about the purpose of the global settings if they don’t automatically apply to cache yes shares without an override

 

Do you have this issue with cache:yes  when primary cache- > secondary cache2 ?