Reynald

Hi, Sorry for late answer @Camnomis. Your cache is in a state I've not tested that much: Move All from Primary->Secondary shares option is selected and pool is above move all threshold percentage: 100% > 95%. I will investigate and push a revised version. In the meantime, you may change this setting to no: Move All from Primary->Secondary (cache:yes) shares when disk is above a certain percentage: No Your cache will be freed down to 80%. If you want to mimic the option, set the freeing threshold to 0% and it will move everything of the cache as would the Move All option

Oups, forgot to announce. Push one hour ago: 2024.08.17 Fix settings override not reverting (R3yn4ld)

Thank you. Can you explain what is your data structure please? What do you store in what king (single, cache->array, array->cache..) of share?

Wow it can take... a while. What is "as I would like" ? If 9 millions files on SSD it can be not that long. If on spinners, hum, please tell us how long it was, that's interesting. Yes, at the moment and until filelist is store in database, it will be everytime mover runs. (it won't overlap and skip if a mover instance is already runing) I have an optimisation idea I can set quickly if something else than auto age is selected (processing like 2023 version), but it depends of usecases

If your system share is set to Cache->Array (cache:yes) or Array->Cache (cache:prefer), then if your usage is above occupation threshold then yes, mover is moving older files from cache. Good settings for Appdata and System share are to set just a Primary (Cache) and no Secondary (cache:only). You can then use the advanced settings "Rebalance share" of Mover Tuning to move back the files where they're suppose to be: I can't answer your question about ZFS docker folder as I myself use a vDisk for docker (in previous implementations of ZFS I've found it laggy and it was even crashing my kernel).

2024.08.15.2118 Fix ignore list reserved space double quoting (Thanks silver226 see forum post) (R3yn4ld) Better empty folder cleaner (R3yn4ld) Rewritten to rmdir parent directory of a moved file if empty (drawbacks: will let multidirectory dirs alive) Added option to enable/disable empty folder cleaner added in Settings UI UI improvements, settings sorted 2024.08.15.0025 Even Better cache priming (hopefully) (R3yn4ld) Rewriten Ignore filelist from file and filetypes filtering functions (major) Improve calculating size of filtered files and filetypes Update calculation from basic/bc to numfmt. Removed bc option Added verification for not breaking hardlinks when an hardlinked file is filtered Added testmode to cleaning empty folder function, and a min depth of 2. Fix ctime bug (R3yn4ld) @Silver226you may have some inconsistency with your _byte thing. Now, file and folder relies on du -sh (a little less precise but get compressed size instead of apparent size on zfs systemfiles). But overall, you bring me on the right track, thank you for that

Done Testing ignore file list and releasing

I'm going to investigate the share overrides, I didn't touch this part for a while...

The code block was misplaced and that's why it picks to many dirs. I added options to tune or disable empty folder cleaner. It will be difficult to log un Mover_tuning log as well, but I'll give it a try

Great job, thank you for your review and patches ! I'm going to implement this

To check for mover tuning files, you can: open a terminal and then: cd /tmp/ca.mover.tuning/ ls Regarding your option "Mover - logging" should not made difference for Mover Tuning (or maybe prevent mover tuning logs to appear, I have to check/test) I will check about verbosity in different logging option cases.

The folder is /tmp/ca.mover.tuning/ You can ls to it in the terminal, or use a File manager. There is nothing else in the syslog? No error, nothing?

Hi, This message is very normal, it is the mechanism to start the mover taking these options into account: Do you have some verbosity from the mover? It should if this option is enabled: This option will also create a log file: /tmp/ca.mover.tuning/Mover_tuning_(date).log Three other files are also created in /tmp/ca.mover.tuning, the most usefull are Filtered_filelist and Mover_action.

Here is how folder cleaner works: - It looks for shares where files have been previously moved - It find empty folder (litteraly, empty, not any file inside) - It remove the folder Since 2024.08.15, it also: - Apply filter from exclusion - Have a min depth of 2: /mnt/*/share/dir1/dir2 and won't delete below dir3 then I don't think it's related, because it won't delete any file (it would throw error if there are files in a folder it attempt to delete)

2024.08.15 Even Better cache priming (hopefully) (R3yn4ld) Rewriten Ignore filelist from file and filetypes filtering functions (major => test mode) Improve calculating size of filtered files and filetypes Update calculation from basic/bc to numfmt. Removed bc option Added verification for not breaking hardlinks when an hardlinked file is filtered Added testmode to cleaning empty folder function, and a min depth of 2. Fix ctime bug (R3yn4ld)

Got it Doing some testing and pushing an update later

Hi, I think I have to rewrite the function, it's too efficient at cleaning it seems. I went the easy, quick, and efficient way by looking for empty dirs since root of a share where files were moved. I've just added some verification and limits: added a dry-run in testmode, and do not go below a min depth of 2. For ctime, I have to debug it yes. It's quiet inefficient (long process) to look at ctime when filtering files and not all filesystem handle ctime correctly (especially with Copy On Write), but there is a real use case for that. Oh, and thank you for the report

Hardlinks are preserved. It was the case in 2023 versions also. A file won't be moved on cache without all files hardlinked to it Every hardlinked files will be recreated on array. You can check Mover_action file in /tmp/ca.mover.tuning/: the NBLINKS column represent the number of hardlinks to a file. Then you have the inode, which represent the unique file on disk. The filesize of hardlinks is set to zero (to not be counted x times when calculating sizes) The mover sends all the inode to the same batch to rsync with the --hard-links option, so it recreates hardlinks on destination Your settings seems great for what you are trying to achieve.

Hi, Thank you for your report For ctime, I'm going to check that again. For deletion empty directory, you can create a .placeholder or any other empty file in those you do not want to be deleted. In my tests I did not encounter that as I always have some dusts in usenet folders. I will try something cleaner.

2024.08.12.0852 Repair/optimize cache priming (R3yn4ld). Adding check to bc (un)installation routine 2024.08.12.0023 Add bc (un)install option (R3yn4ld) Force test mode only on major upgrade, keep on minor (R3yn4ld) Better cache priming (R3yn4ld)

Hi @Faceman /./ in rsync is for relative path. From rsync man: It's also how unbalanced plugin works. That's a bug, it's corrected (see below)

If bc is installed it's used whatever the "install routine" and it's parameter. I made a typo on line 386, thanks. I will reduce verbosity in case bc is detected by installation routine. The setting has no effect if bc is already installed. We know bc is working when we have size with two decimals. If not, it's rounded to the nearest integer (also for TiB which make the information useless)

Yes I did, because I made some major changes in filtering or decision handling. It's stated in the plugin installation window: Updating config file - Forcing test mode There shouldn't be any major changes in the next few weeks. The next planned feature is smart caching TV shows, which will optimize storage and playback. We'll introduce this through a beta branch before a full release.

2024.08.11 Allow operation without array if multiple pools (R3yn4ld): Fixed fatal error bug Fixed error message about mover.pid and softstop file when installing the plugin or booting Unraid @ramjam824 shall be OK for you now

Hello, This is a bug. I added these command in install script to clean leftovers of a previous interrupted execution. Added some controls for next version. These two errors do not affect the program's functionality.