12 hours ago, Dor said:

Done, #44.

Done, test it from github age_mover

On 3/29/2025 at 5:46 PM, steve1977 said:

seems to work well, but some empty folders remain on the cache disk

Check new version if fixed issue

On 3/29/2025 at 9:13 AM, SubbyDew said:

particularly the .list files

Check new version, you can change log folder now, I will try to fix long list in next version 

It would seem that the "Disable Mover running on a schedule" setting is not working as expected. At least I have it set to "Yes" and mover still runs on the previously set schedule... this is with the currently latest version.

 

 

1 hour ago, MellowB said:

would seem that the "Disable Mover running on a schedule" setting is not working as expected.

Was it worked befor? 

Good question! As I had not tried to disable the mover tuning before, I can not say.

Looks like $ in filenames may be causing issues? 

 

I see some posts a few pages back with a similar issue, where the response was $ is not valid in filenames. But $ are valid for filenames both based on spec and obviously, since those files exist and were put there by other unraid processes/containers

 

Apr  2 11:33:15 Tower move: Error: Invalid characters in filename: 'Beast Games (2024) - S01E01 - 1,000 People Fight For $5,000,000 [Release Sources (Streaming Service) - Notifiarr_6_0 WEBDL-720p][EAC3 2.0][h264]-FLUX.mkv'
Apr  2 11:33:15 Tower move: Error: Invalid filename path: 'data/media/tv/Beast Games (2024) {tvdb-447806}/Season 1/Beast Games (2024) - S01E01 - 1,000 People Fight For $5,000,000 [Release Sources (Streaming Service) - Notifiarr_6_0 WEBDL-720p][EAC3 2.0][h264]-FLUX.mkv'
Apr  2 11:33:15 Tower move: Error: Invalid characters in filename: 'Beast Games (2024) - S01E10 - $10,000,000 Coin Flip [Release Sources (Streaming Service) - Notifiarr_6_0 WEBDL-720p][EAC3 2.0][h264]-FLUX.mkv'
Apr  2 11:33:15 Tower move: Error: Invalid filename path: 'data/media/tv/Beast Games (2024) {tvdb-447806}/Season 1/Beast Games (2024) - S01E10 - $10,000,000 Coin Flip [Release Sources (Streaming Service) - Notifiarr_6_0 WEBDL-720p][EAC3 2.0][h264]-FLUX.mkv'
Apr  2 11:33:15 Tower move: Error: Invalid characters in filename: 'Beast Games (2024) - S01E08 - Betray Your Friend For $1,000,000 [Release Sources (Streaming Service) - Notifiarr_6_0 WEBDL-720p][EAC3 2.0][h264]-FLUX.mkv'
Apr  2 11:33:15 Tower move: Error: Invalid filename path: 'data/media/tv/Beast Games (2024) {tvdb-447806}/Season 1/Beast Games (2024) - S01E08 - Betray Your Friend For $1,000,000 [Release Sources (Streaming Service) - Notifiarr_6_0 WEBDL-720p][EAC3 2.0][h264]-FLUX.mkv'

 

Edited April 2, 20251 yr by Terebi

8 hours ago, MellowB said:

It would seem that the "Disable Mover running on a schedule" setting is not working as expected. At least I have it set to "Yes" and mover still runs on the previously set schedule... this is with the currently latest version.

 

 

Pretty sure this is not how that setting works or has ever worked.  The tooltip:

 

"Yes - setting effectively disables this plugin"

 

It disables the plugin, NOT the mover.  This only reverts the mover back to the stock Unraid implementation without the filters.

 

The only way to disable the mover entirely is to remove the secondary storage entirely for each share OR to set an extremely high threshold value for free down and an extremely low value for prime up (i think?  I no longer use array -> cache shares).

But the notification explicitly states that *Mover Tuning plugin* ran and did its thing... which it obviously shouldnt if I disable it.

1 hour ago, ronia said:

It disables the plugin, NOT the mover.  This only reverts the mover back to the stock Unraid implementation without the filters.

No, it should only disable scheduling for plugin. But I checked if plugin disabled, it's still running on schedule but I didn't changed this part at all. As per description you can still run it by mover button. 

 

For run unraid mover by button - move now button follow plugin filters. 

 

So if anyone with php knowledge can help here? 

 

 

1 hour ago, Terebi said:

Looks like $ in filenames may be causing issues? 

You can disable file validation 

3 minutes ago, Masterwishx said:

You can disable file validation 

I can, but why is that a validation rule, when $ is a completely valid char in linux filenames?

30 minutes ago, Terebi said:

completely valid char in linux filenames?

Becouse $ can be used to run commands by filenames, cybersecurity risks

I turned off mover with the "Disable Mover running on a schedule" option a few days ago to help speed up a parity operation, but when I came back to it and looked at the logs, it kept running as normal, on schedule. so there is definitely something up with that option.

 

Just to test, I just turned it off again, and sure enough it ran again the next time my schedule came around.

Mover was working fine until last update (3/30) and I don't understand what's making it not work.   It's set to move files that are older than 10 days old when capacity used is over 70% of cache.  I SUSPECT it's because it says "keep on primary" in the Mover Action log, but I don't know why it is doing that when I haven't changed anything to make that happen, nor do I see a setting that's making that happen.  These files are more than 10 days old, cache usage is over 70%, it looks like all the criteria has been met, but files are not being moved to secondary.  Any advice would be appreciated.

 

excerpt from Mover tuning log:

18:00:06.803 Moving threshold: 70% (1.3TiB) ; Freeing threshold: 70% (1.3TiB)
18:00:06.804 Mover action: cache->user0 (cache:yes). Pool is above moving threshold percentage:  81% >= 70%.
18:00:06.805 => Will smart move old files from cache to user0. Nothing will be moved from user0 to cache
18:00:06.806 Adding Age
18:00:06.807 Age (modification time) 10
18:00:06.873 Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-04-02T180001.list for media

 

excerpt from Mover Action log:

cache|media|yes|1742426843|6731|495037683165|1350342017024|1|12922|keep on primary|/mnt/cache|none|media/media/movies/my_file.mov

 

excerpt from Filtered Files log:

cache|user0|media|yes|1742426843|1350342017024|6731|0|1|12922|/mnt/cache/media/media/movies/my_file.mov

 

Summary log:

PRIMARYSTORAGENAME|FILES_FROM_PRIMARY|SIZE_FROM_PRIMARY|FILES_FROM_SECONDARY|SIZE_FROM_SECONDARY
TOTAL|0|0|0|0|0|0

 

ca.mover.tuning.cfg:

testmode="no"
logging="yes"
movenow="yes"
version="2025.03.30"
moverDisabled="no"
validateFilenames="yes"
debuglogging="no"
notify="no"
advancedSettings="yes"
parity="yes"
moverNice="0"
moverIO="-c 2 -n 0"
force="no"
beforeScript=""
afterScript=""
movingThreshold="70"
freeingThreshold="70"
age="yes"
daysold="10"
advancedFilters="yes"
ctime="no"
atime="no"
sizef="no"
sparsnessf="no"
filelistf="no"
filetypesf="no"
ignoreHidden="no"
omovercfg="yes"
enableTurbo="no"
cleanFolders="yes"
cleanDatasets="no"
synchronizeCache="no"
advancedOptions="yes"
rebalanceShares="no"
resynchronizeCache="no"
omoverthresh="90"
fillupThreshold="0"
loggingFolder="/tmp"
logfilesDaysold="1"
listfilesDaysold="1"

 

7 hours ago, Faceman said:

Just to test, I just turned it off again, and sure enough it ran again the next time my schedule came around.

Yep I can reproduce it, but didn't changed this part as I said, will try to check it but have no experience with php at all, so if some one can help will be cool

6 hours ago, grants169 said:

Moving threshold: 70% (1.3TiB) ; Freeing threshold: 70% (1.3TiB)

Try set 60% for Freeing threshold 

7 hours ago, Masterwishx said:

Try set 60% for Freeing threshold 

Thank You.  I'm not understanding this setting.  Setting to 60% freed space to 78%, I don't understand that math, but the trick for me is to set to 0% to finally move all files older than 10 days.

1 hour ago, grants169 said:

Setting to 60% freed space to 78%, I don't understand that math

This freeing upto 60%, when pool size + file size >= 60%.

So if you have larger file it will not fit upto 60%.

 

Also known issue when zfs have reserved space for datasets or so plugin don't include it in calculation as was described in github issue. 

Edited April 3, 20251 yr by Masterwishx

46 minutes ago, grants169 said:

Setting to 60% freed space to 78%

Maybe you have also cache:prefer? 

Running version 2025.03.30.  Yesterday ran for the first time successfully.  Today mover ran and has error "syntax error near unexpected token `newline'".  It then doesn't appear to generate any of the files (list, log, txt) to process and nothing was moved because of that.  Is there a setting I should look at?

 

Apr  3 03:56:46 Tower root: ---------------------------------------------------------- Processing [Television] share --------------------------------------------------------
Apr  3 03:56:46 Tower root: Primary storage: cache_hd - size: 932GiB - used:  40 % (365GiB)
Apr  3 03:56:46 Tower root: Secondary storage: user0
Apr  3 03:56:46 Tower root: Share Information: Name: Television - Path: /mnt/cache_hd/Television
Apr  3 03:56:46 Tower root: Moving threshold: 0% (0B) ; Freeing threshold: 0% (0B)
Apr  3 03:56:46 Tower root: Mover action: cache_hd->user0 (cache:yes). Pool is above moving threshold percentage:   40% >= 0%.
Apr  3 03:56:46 Tower root: => Will smart move old files from cache_hd to user0. Nothing will be moved from user0 to cache_hd
Apr  3 03:56:46 Tower root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: eval: line 991: syntax error near unexpected token `newline'
Apr  3 03:56:46 Tower root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: eval: line 991: `find "/mnt/cache_hd/Television" -type f -depth -not -name '.placeholder' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|'         -v SPARSENESS='0'         -v PRIMARYSTORAGENAME='cache_hd'         -v SECONDARYSTORAGENAME='user0'         -v SHARENAME='Television'         -v SHAREUSECACHE='yes'         -v PRIMARYSIZETHRESH='0' ' { printf "%s|%s|%s|%s|%d|%d|%d|%d|%d|%d|%s\n", PRIMARYSTORAGENAME, SECONDARYSTORAGENAME, SHARENAME, SHAREUSECACHE, $1, PRIMARYSIZETHRESH, $2, $3, $4, $5, $6}'>>'
Apr  3 03:56:46 Tower root: Updated Filtered filelist: 
Apr  3 03:56:46 Tower root: /tmp/ca.mover.tuning/Filtered_files_2025-04-03T034001.list for Television

 

 

Apr  3 03:56:46 Tower root: ************************************************************ ANALYSING MOVING ACTIONS ***********************************************************
Apr  3 03:56:46 Tower root: Deciding the action (move/sync/keep) for each file. There are 31 files, it can take a while...
Apr  3 03:56:46 Tower root: awk: cmd. line:132:             printf "PRIMARYSTORAGENAME|FILES_FROM_PRIMARY|SIZE_FROM_PRIMARY|FILES_FROM_SECONDARY|SIZE_FROM_SECONDARY\n" > "
Apr  3 03:56:46 Tower root: awk: cmd. line:132:                                                                                                                           ^ unterminated string
Apr  3 03:56:46 Tower root: grep: 
Apr  3 03:56:46 Tower root: /tmp/ca.mover.tuning/Summary_2025-04-03T034001.txt: No such file or directory
Apr  3 03:56:46 Tower root: grep: 
Apr  3 03:56:46 Tower root: /tmp/ca.mover.tuning/Summary_2025-04-03T034001.txt: No such file or directory
Apr  3 03:56:46 Tower root: grep: 
Apr  3 03:56:46 Tower root: /tmp/ca.mover.tuning/Summary_2025-04-03T034001.txt: No such file or directory
Apr  3 03:56:46 Tower root: grep: 
Apr  3 03:56:46 Tower root: /tmp/ca.mover.tuning/Summary_2025-04-03T034001.txt: No such file or directory
Apr  3 03:56:46 Tower root: grep: 
Apr  3 03:56:46 Tower root: /tmp/ca.mover.tuning/Summary_2025-04-03T034001.txt: No such file or directory
Apr  3 03:56:46 Tower root: grep: 
Apr  3 03:56:46 Tower root: /tmp/ca.mover.tuning/Summary_2025-04-03T034001.txt: No such file or directory
Apr  3 03:56:46 Tower root: No new files will be moved/synced from primary to secondary
Apr  3 03:56:46 Tower root: No new files will be moved/synced from secondary to primary

 

On 3/29/2025 at 5:46 PM, steve1977 said:

Shouldn't all folders be deleted / moved?

Is the latest version working fine for delete folders? 

FYI, I enabled "Debug Log Mover Tuning plugin actions" and now it's working.

 

Apr  3 12:51:21 Tower move: ---------------------------------------------------------- Processing [Television] share --------------------------------------------------------
Apr  3 12:51:21 Tower move: Primary storage: cache_hd - size: 932GiB - used:  46 % (424GiB)
Apr  3 12:51:21 Tower move: Secondary storage: user0
Apr  3 12:51:22 Tower move: Share Information: Name: Television - Path: /mnt/cache_hd/Television
Apr  3 12:51:22 Tower move: Moving threshold: 0% (0B) ; Freeing threshold: 0% (0B)
Apr  3 12:51:22 Tower move: Mover action: cache_hd->user0 (cache:yes). Pool is above moving threshold percentage:   46% >= 0%.
Apr  3 12:51:22 Tower move: => Will smart move old files from cache_hd to user0. Nothing will be moved from user0 to cache_hd
Apr  3 12:51:22 Tower move: --------------------------------------------------- Debugging [Television] share Find Function --------------------------------------------------
Apr  3 12:51:22 Tower move: FINDSTR: find "/mnt/cache_hd/Television" -type f -depth -not -name '.placeholder' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|'         -v SPARSENESS='0'         -v PRIMARYSTORAGENAME='cache_hd'         -v SECONDARYSTORAGENAME='user0'         -v SHARENAME='Television'         -v SHAREUSECACHE='yes'         -v PRIMARYSIZETHRESH='0' ' { printf "%s|%s|%s|%s|%d|%d|%d|%d|%d|%d|%s\n", PRIMARYSTORAGENAME, SECONDARYSTORAGENAME, SHARENAME, SHAREUSECACHE, $1, PRIMARYSIZETHRESH, $2, $3, $4, $5, $6}'
Apr  3 12:51:22 Tower move:                                                                                                                                                  
Apr  3 12:51:22 Tower move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2025-04-03T125119.list for Television

 

42 minutes ago, niwmik2 said:

FYI, I enabled "Debug Log Mover Tuning plugin actions" and now it's working.

If disable debug it's not working? 

4 hours ago, niwmik2 said:

Is there a setting I should look at?

Sorry cant reproduce this issue in /tmp: 

 

 

There is new setting for this , you can change parent folder for Logs . By default /tmp , read description