Reynald

Does it move or does it sync? Please check the log, the mover action list (in /tmp/ca.mover.tuning), and sync option, I'd be happy to correct a bug or bad behavior, testing on my own may not be enough

No plan for option about the "cache juggling" yet, as if you set this to anything else than Auto and Synchronise cache files to array, the behavior is the same as before: This will wake the array to synchronise whats on cache to it if files are newer than the last scan/sync. All share content (on cache) will be scanned at each run. Content of Use Cache:Prefer (Array->Cache) will be scan and may or may not (if Folder Caching plugin installed) wake the array up. Set Use cache:Yes in the share settings to not wake the spinners. Cache:Prefer scanning is a new feature, as before Mover Tuning only focussed on Cache:Yes shares. Feature for smart caching was requested for a while, several script attempt (creating dynamic ignore list) have been made, now it's live. Per share threshold was implemented before I "arrive" in this project, but there were no gui option for it. And finally, no, hugenbdd won't update his version, we've hand over, and I have no plan to deprecate it at the moment. Hope this help, do not hesitate if you need some more support :)

Update from last night: ###2024-08-01 - Deleted share error control - SoftStop improvement How softstop works: It will cleanly exit loops (Filtering, Deciding, Moving). While moving/syncing, the file transfer is not interrupted. Can be launched by running this command from terminal or from a script (it was already existing previously): /usr/local/emhttp/plugins/ca.mover.tuning/age_mover softstop there are also, to start age_mover (the engine of Mover Tuning Plugin) /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start and to kill all the process (can lead to unfinished or corrupted file transferts while moving): /usr/local/emhttp/plugins/ca.mover.tuning/age_mover stop

Yes it is: The tuner list all files from cache:prefer and cache:yes from the more recent to the oldest, then it compare the cumulative size to the threshold, for each sharepool. If you want to keep below 80% and that while running the scheduled mover you are at 92%, then the oldest files from cache:prefer and cache:yes representing 12% of space will be moved to array. If you want to keep below 80% and that while running the scheduled mover you are at 67%, then the most recent files from cache:prefer shares on array, representing 13% of cache size, will be moved to cache, not those from cache:yes because they are meant to be moved from cache to array and that is..

Yes, but for Cache:Prefer (Array->Cache) shares only. For cache:yes (Cache->Array), it will keep them if it's not needed to free the cache (i.e usage below threshold). In any case, for freeing the cache, oldest Cache:Yes and Cache:Prefer are moved to array in chronological order until threshold is met

Next version will be verbose about that: if it find a config but the share is not on system

I cannot see mover in the log you provided. Are you sure of your configuration? (schedule mover and mover tuning)

I'm not aware of the 70% option. I guess you mix trigger and free up with another plugin/script (that's how my former "Smart caching script" worked before, and there are some script intended to be ran by this mover tuning plugin, like Terebi's one). Here is an explanation, to make it simple I assume there is no other (size, sparness, etc..) filters. Since previous versions, this plugin filters the cache file list and the move of filtered files are triggered based on threshold. If you filter with AGE=7, passed the threshold all files older than 7 days will be moved. If AGE=0 everything is move. Now (since I maintain this plugin) if you filter with AGE=Automatic, only old files that are above the threshold will be moved. That means that if you set threshold to 70%, when mover is invoked, if you are at 87% it will free the cache down to 70%.

Available shares are found by reading the config files: for SHARECFG in /boot/config/shares/*; do [...] SHARENAME="$(basename "$SHARECFG" .cfg)" Your error is because you delete the share content but not the share via the UnRaid GUI. I'm adding error control in the next version

Synchronize cache files to array will rsync cache=prefer and cache=yes shares to array. Move will rsync --remove-source-files (delete) them, which is basically a deletion for file that have not been modified. Rsync flags are similar to 6.x mover and Unbalanced plugin. This is not yet a per share setting. It's there in the code: https://github.com/R3yn4ld/ca.mover.tuning/blob/b611aaba62b083622598900b8146edd32b2d180e/source/ca.mover.tuning/usr/local/emhttp/plugins/ca.mover.tuning/age_mover#L718 This is specifically for this feature and a better hardkinks management that I stopped using the UnRaid mover

Above bug corrected, thank you @Freender 2024-07-30.0038: Various bug fixes R3yn4ld Freender Enhance internal mover function R3yn4ld

OK, got it ! https://github.com/R3yn4ld/ca.mover.tuning/blob/5747150459a4a362de4fc5020feb8381c0eaa2d0/source/ca.mover.tuning/usr/local/emhttp/plugins/ca.mover.tuning/age_mover#L573 # Make decision to move or keep: action = ($3 != "only" && ($3 == "prefer" && (pool_sums[$1] >= $5 && AGE == -1) || AGE >= 0)) ? "move" : "keep" No decision taken for cache = no and cache = yes shares...

If no sensible data inside, can you post Mover_action please? Else, please tail it. At some point, green number shall be > yellow number and action be move,

@Freender reading your log again. I don't think it's necessary to log "Skipfilelist string:" as the find string (and not the found string 👙) is really large and only usefull for debugging purpose. I may remove that in a next version. What do you (and all others?) think?

1) yes, was a mess to play with options 2) Yes, Filtered_files_*.list, Mover_action_*.list, Mover_tuning*.log, and Summary_*.txt are created even if there are no files to move. The most important file for debugging, in addition to the log, is Mover_action_*.list: With this one, you see all the informations: CACHEPOOLNAME|SHARENAME|SHAREUSECACHE|MODIFICATIONTIME|FILESIZE|POOLSUM|THRESHOLD|NBLINKS|INODE|ACTION|FROM|TO|FILEPATH From your example, you don't expect files to be moved, at the end of the log: decision are no move .

New version: 2024-07-30.0000 Implement above fixes and automatically recreate config file if broken from 2024-07-29 version fresh install

3) Corrected in 2024-07-29.b I had to make some adaptation in getMoverSettings() function to not overwrite defaults when no set for share override. Result: 13:03:47.499 ------------------------------------------------------------ Settings for media share ----------------------------------------------------------- 13:03:47.505 Settings override: 13:03:47.509 ------------------ 13:03:47.520 Age: Automatic (smart caching) 13:03:47.537 Share threshold: 60 % 13:03:47.552 ------------------------------------------------------------------------ ----------------------------------------------------------------------- 13:03:47.562 Cache mode: Yes/Cache->Array. Will smart move old files from Cache to Array. Nothing will be moved from Array to Cache 13:03:47.566 Share Information: Name: media - Path: /mnt/cache/media - Cache mode: yes 13:03:47.590 Pool Information: Name: cache - pool used: 89 % - size: 856872009728 bytes (798 GiB) 13:03:47.606 Default threshold: 95 % - media share threshold: 60 % => Threshold selected: 60 %,s size: 554372903731 bytes (516 GiB) 13:03:48.597 Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-07-29T130347.list for media 13:03:48.606 ----------------------------------------------------------------- Global settings ---------------------------------------------------------------

1) corrected in 2024.07.29.b 2) good idea, implemented !

Updated: 2024-07-29 Automatic Array to Cache Complete rewrite of file listing functions (find, decide to move..) (R3yn4ld) Updated zfs functions getting usage of a pool (R3yn4ld, greetings to (Freender on forums) for pointing out and proposal) Added cache mode "prefer" smart moving in "Automatic age" mode (R3yn4ld) Added option to "repair" Cache:Only (moving everything on share to cache) and Cache:No (moving everything on share to array) shares (R3yn4ld) Added option to synchronize Cache:Yes and Cache:Prefer shares to array so data are parity protected (R3yn4ld) Turbo write mode forcing improvement to not wake spinners if not needed (R3yn4ld) UI improvements (R3yn4ld) Full changelog: R3yn4ld/ca.mover.tuning at automatic-array-to-cache (github.com) It is a huge huge update. I've tested for a couple of weeks. Hopefully you won't find many bugs, please report in any case. Installation: ca.mover.tuning.plg Please remove Community Application one before installing this one (at the moment via Plugins/Install plugin/insert above URL). Please note that you'll have to reconfigure mover tuning. Update: if already installed with above method, you should be able to update normally: (above is the setting page of 2024-07-29 already ) Advise: enable Test Mode as on picture above and verify the Mover_action_{date}T{time}list in the /tmp/ca.mover.tuning directory. This file contain the files that would be moved with Test Mode off. If everything seems OK you can turn off Test mode.

Ah OK, understood. Then the file size filter is effectivly your friend

"Move All from Cache-yes" will move all based on the threhold for this option. Do not select this option and just set "Only move at/to this threshold of used cache space:" and "Move files off cache based on age?"

The report are in the file in /tmp/Mover. You can check the list of what would be moved with test mode: no to keep media library pictures on ssd drives, set the share(s) to "Cache=only". In latest unraid version, do not set a secondary storage for it.

Ok. And is the age_mover running now?

Actually 'per pool threshold' is not working, so with this plugin, you would set a 70% threshold and the files of each shares above 70% would be moved to array. That's not a problem as you cache drive is about 20%

Very strange because creating symlinks is part of the installation of the latest plugin. Please try this code in console and report: if [[ -e /usr/local/bin/move ]]; then echo "Creating Soft link for 6.12 RC8 or newer since /usr/local/sbin/move is now located in /usr/local/bin/move" if [[ -e /usr/local/sbin/move ]]; then rm /usr/local/sbin/move; fi ln -s /usr/local/bin/move /usr/local/sbin/move elif [[ -e /usr/libexec/unraid/move ]]; then echo "Creating Soft link for 7.0 or newer since /usr/local/sbin/move is now located in /usr/libexec/unraid/move" if [[ -e /usr/local/sbin/move ]]; then rm /usr/local/sbin/move; fi ln -s /usr/libexec/unraid/move /usr/local/sbin/move fi