On 10/2/2024 at 6:38 PM, Terebi said:

The mover does not appear to be cleaning up empty directories anymore using the age mover.  the "Clean Empty Folders" setting is set to true. 

 

The logs shall tell you why. In example:

Quote

16:33:56.490 270549 files remaining from caches to array  2.5TiB
16:33:56.529 Moving "/mnt/cache/./media/_downloads/torrents/complete/sonarr/Pokemon S18 - AMZN WEB-DL 1920x1080 FR [ABYSS]/Pokemon S18E43 - AMZN WEB-DL 1920x1080 FR [ABYSS].mkv"  to  /mnt/user0/ 
Not deleting dir containing 29 files: /mnt/cache/media/_downloads/torrents/complete/sonarr/Pokemon S18 - AMZN WEB-DL 1920x1080 FR [ABYSS]

[...]

16:44:47.173 270528 files remaining from caches to array  2.5TiB
16:44:47.211 Moving "/mnt/cache/./media/_downloads/torrents/complete/sonarr/Your.Honor.US.S02E02.MULTi.DV.HDR.2160p.WEB-DL.DD5.1.H265-R3MiX/Your.Honor.US.S02E02.MULTi.DV.HDR.2160p.WEB-DL.DD5.1.H265-R3MiX.nfo"  to  /mnt/user0/ 
Deleting empty dir /mnt/cache/media/_downloads/torrents/complete/sonarr/Your.Honor.US.S02E02.MULTi.DV.HDR.2160p.WEB-DL.DD5.1.H265-R3MiX
 

 

Edited October 6, 20241 yr by Reynald

On 10/5/2024 at 6:11 AM, Soulplayer said:

I'm seeing the mover doing something that I do not understand. It is moving the files from my cache to my array, but the share is set Array -> Cache-docker. So that does not make sense? Any idea why it is doing this? Is this a setting I wrongfully set, or is it a bug?

 

edit: and the next run it moves everything back… and then again.  I disabled it for now. 

 

In the log you've shown mover plugin says it will prime your cache up to the threshold. Can you please show several logs of it moving back and forth?

 

I've not extensively tested with age not set. I prefer to use it with 'automatic age' (plugin is extensively tested with this option)

Seems there is a bug in the plugin that causes it to not work with shares that have a space in the name.

Just found this in the logs after failing to figure out why my cache kept filling up no matter what settings I used.

 

Oct  8 16:15:33 Tower move: Primary storage: new_cache - size: 900GiB - used: 49 % (458GiB)
Oct  8 16:15:33 Tower move: Secondary storage: user0
Oct  8 16:15:33 Tower move: Share Information: Name: TV Shows - Path: /mnt/cache/TV Shows
Oct  8 16:15:33 Tower move: Moving threshold: 5% (45GiB) ; Freeing threshold: 5% (45GiB)
Oct  8 16:15:33 Tower move: Mover action: cache->user0 (cache:yes). Pool is above moving threshold percentage:  49% >= 5%.
Oct  8 16:15:33 Tower move: => Will smart move old files from cache to user0. Nothing will be moved from user0 to new_cache
Oct  8 16:15:33 Tower move: find: '/mnt/cache/TV': No such file or directory
Oct  8 16:15:33 Tower move: find: 'Shows': No such file or directory
Oct  8 16:15:33 Tower move: awk: fatal: cannot open file `-v' for reading: No such file or directory
Oct  8 16:15:33 Tower move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-10-08T161531.list for TV Shows

 

Unraid does warn "We highly recommend to make your life easier and avoid special characters."

So while the space isn't a symbol they explicitly disallow, I wouldn't be using spaces for share names where you know scripts will be running commands on them. sometimes they dont get sanitised/escaped into "/mnt/cache/TV\ Shows/" properly

 

 

edit: I suspect it could be fixed with some tweaks to the scripts, but simply renaming a couple of shares and adjusting container mappings to suit would solve it for everything.

Edited October 9, 20241 yr by Faceman

A quick clarification for anyone in the know:

 

Does the 'Move' button on the 'Main' page follow the rules set via Mover Tuner?

 

I confirm I have the relevant settings done (after quite a bit of fiddling and trial and error), so i know 100% that the 'Move Now' under the schedule settings works flawlessly and follows my custom settings, however I don't want to risk pressing the button on the main page unless i know for sure it will also follow those rules (as it means it will screw with my Sabnzbd etc and mean i have to manually move files to rectify it which means halting it and stuffing around with MC or dynamix again which i've done way too many times as i figured my way through the settings lol)

Edited October 14, 20241 yr by jpw007

On 10/4/2024 at 9:11 PM, Soulplayer said:

I'm seeing the mover doing something that I do not understand. It is moving the files from my cache to my array, but the share is set Array -> Cache-docker. So that does not make sense? Any idea why it is doing this? Is this a setting I wrongfully set, or is it a bug?

 

edit: and the next run it moves everything back… and then again.  I disabled it for now. 

 

 

Mine is doing the same thing. See:

 

Edited October 15, 20241 yr by Brenex

I'm seeing on 7.0.0-beta 3 that if I run a 'move now' from within a share page, eventually the logs of whatever is being moved stop showing up in the syslog. I can see the data is still happening and if I click move now again I get the 'mover already running' prompt, but I'm unclear why the existing job stops posting to the log.

 

I have not tested this on the scheduled mover job or the main page move button. I'm not sure if anyone else has reported seeing this behavior, whether with or without the plugin installed.

 

Is there any way to try and retrieve and reattach the job to the syslog somehow? Thanks.

On 9/12/2024 at 8:14 PM, Reynald said:

 

Script configuration in your case is like this:

 

 

#!/bin/bash

 

shareName=media

/usr/local/emhttp/plugins/ca.mover.tuning/share_mover {$shareName} >> /var/log/syslog &

 

 

(There was a typo in my previous proposal, you've nicely seen it 😊)

 

You can run it from terminal and check for any error.

 

You can check syslog (from UnRaid webui) to see if mover is running and get feeback

If I run this from unraid terminal and then check syslogs I get this....

Oct 16 11:00:01 Tower root: Starting Mover
Oct 16 11:00:01 Tower root: ionice -c 2 -n 0 nice -n 0 /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start
over tuning plugin done!

Seems to finish in less than second and doesn't actually run.

Hi to all,

i'm new to the mover tuning plugin and i need some help / clarification about the behavior of the mover tuning plugin.

 

Unraid Version: 6.12.13

Mover Tuning Version: 2024.09.05.0222
Issue: Mover not syncing

 

My mover tuning default settings. All relevant shares use the default settings for the moment.

 

 

 

Issue is, that every time i start the mover, independent if dry-run is on or off, files are not synced to secondary (Option: Syncronize Primary files to Secondary is YES).

e.g.: 2 shares: documents and system both use the default mover tuning settings. Only minimum free space share setting differs. 

All files of both shares are on cache (done with unbalanced plugin).

 

system

 

Dokumente

 

BUT:

When i start the mover, the files of "Dokumente" share are not touched (not moved, not synced) and the files of system share are moved to secondary, no sync.

 

Quote

PRIMARYSTORAGENAME|SHARENAME|SHAREUSECACHE|MODIFICATIONTIME|FILESIZE|POOLSUM|THRESHOLD|NBLINKS|INODE|ACTION|FROM|TO|FILEPATH
cache|system|yes|1729114303|21474836480|21474836480|800163069952|1|1688263|move from primary|/mnt/cache|/mnt/user0|system/docker/docker.img

 

My expectation is that both shares stay on primary (cache) and are only synced to secondary as backup.

And if threshold of 95% of used space on cache is tripped then the mover will move/delete files from cache of both shares until used space on cache is below 80%.

 

I also deleted and re-installed the mover tuning plugin but without luck.

 

Hope you all can help or clarify my issue.

2 hours ago, MrSteel said:

My expectation is that both shares stay on primary (cache) and are only synced to secondary as backup.

I think you misunderstand something here.   A file is EITHER on primary storage OR secondary storage.   It will not exist in both locations.

Thanks for your reply but for me the description of this functions is clear: Sync Primary files to Secondary for backup BUT use Primary for modifications. If this is not the case, please clarify. Thanks in advance.

OK I am confused. Didn't see anything in first page or FAQ.

Can someone reply and tag me?

 

I have a Mover Tuning by Squid with last version being last year? (this is the one I have installed)
And then there is a different one, that updated like 50 days ago, that refers to this same thread, but is not made by Squid?
Is this a replacement and I should remove the old and get this?

 

 

5 hours ago, NLS said:

Is this a replacement and I should remove the old and get this?

 

its already the 2nd replacement ... so yes, you should replace it.

On 10/18/2024 at 3:02 PM, MrSteel said:

for me the description of this functions is clear: Sync Primary files to Secondary for backup

in terms of description you are right, may give it a test with dry run and logging enabled so the dev can take a look.

 

your /system/docker/docker.img is prolly not working due "in use" ... as note, while a file is in use it wont be touched ... and as the docker.img is always in use while dockers are running ...

 

may also the 1st rule comes in ... cache treehold ... yours is set to 80 %, so may you are below 80 % and it ll skip anyway and not syncing ... so this "special" backup case also only triggers while upper treehold, dry run, try ...

 

im not using this function myself so cant tell if it should work or not ...

OK installed the latest version.

1) I wish I could set the trigger of cache empty percentage to 0% (to always use mover) instead of just 5%. Just 5% actually covers me ok, but still my OCD says there should be a "disable this filter" setting (aka 0%).

2) Remove empty folders doesn't work. It actually not only left my empty share folder in my cache after mover run, but also 16 empty subfolders!

 

 

Where does the mover log go to when using this plugin and it is triggered by schedule? I can't see any logs in the syslog when mover is running via schedule. I wanted to send cache to array twice per month.

 

I have set;

Mover Logging: Enabled

Log Mover Tuning plugin actions: Yes

Force move of all files on a schedule: Yes

Cron Schedule to force move all of files:

59 22 10,24 * *

 

Force turbo write during mover: Yes

 

but during the move it did not use turbo write mode. I was looking to see the logs to see the issue.

 

I'm on 7.0.0 Beta 3.

 

After the schedule completed, triggering mover manually via the Dashboard shows the mover logs in the syslog as expected and I can see turbo being set and then returned back to auto.

 

 

Edited October 24, 20241 yr by Presjar

a bit of an edge case but a nice feature might be time (hour) based threshholds.

 

So I could say nightly, I want to empty the cache down to 80%, even if only 81% is used. 

 

But then I want to run the mover hourly, but only have it do something if current usage is above 90%. 

 

That way if things are unexpectedly full, it can get cleaned up before the cache is full.  But we aren't eating i/o, spinup, perf, etc for normal filling of the cache during the day. 

6 hours ago, Terebi said:

a bit of an edge case but a nice feature might be time (hour) based threshholds.

 

before this is ballooned endless for such edge cases, may rather just use a cron user script to either

 

1/ sed the treehold to your wishes by your times in the config file

2/ copy them over from your edited versions in the ranges when you want to use different settings

 

which is located here

 

/boot/config/plugins/ca.mover.tuning/ca.mover.tuning.cfg

 

What is the issue with my install here. I tried to install this I am getting this error

plugin: installing: ca.mover.tuning.plg
Executing hook script: pre_plugin_checks
plugin: downloading: ca.mover.tuning.plg ... done

Executing hook script: pre_plugin_checks


6.12.13

plugin: unable to create parent directory for /boot/config/plugins/ca.mover.tuning/ca.mover.tuning-2024.09.05.0222-x86_64-1.txz
Executing hook script: post_plugin_checks

 

any chance of adding a Script to run before mover but having it check the filters first and if the mover isn't going to run because of the filters then the script to run before mover doesn't get ran?

12 hours ago, jcofer555 said:

any chance of adding a Script to run before mover but having it check the filters first and if the mover isn't going to run because of the filters then the script to run before mover doesn't get ran?

what you expect is some sort of "dry run" before the mover runs normally then

 

also some edge case which i would recommend to write something special yourself and let it run before your mover triggers.

1 hour ago, alturismo said:

what you expect is some sort of "dry run" before the mover runs normally then

 

also some edge case which i would recommend to write something special yourself and let it run before your mover triggers.

no not wanting a dry run situation. i'm trying to prevent qbittorrent from having to pause my torrents if the mover isn't going to run because of my mover tuning filters. the trash guide mover python script will pause and start back the torrents regardless if the mover actually starts or not. though a script that checks how full my cache is and then runs the mover tuning command might work. currently i have a script monitoring every 1 minute if mover is running and then perform the functions if it is. just don't know if that'll ever goof anything up if while the mover is running the python script is going through and pausing my torrent rather than having them paused before the mover actually starts. i'll look into the before mover starts way of checking how full the cache is and then invoking the mover tuning moving command.

12 minutes ago, jcofer555 said:

no not wanting a dry run situation.

actually, yes ... as the commads follows each other and the mover dont know if yes or no yet ...

 

run your script, run mover, ... so your edge case would rather be

 

run mover (dry), if mover will take action, run script, run mover ...  if not, dont run the script ... let the mover "do nothing"

 

as the mover is running AFTER script ... no matter what

 

so either you make a "outside" check (user scripts as sample) before your sceduled mover triggers and may just replace the cfg like mentiones 3 posts upper yours "without" the pre run script option OR include the pre run script Option ...

 

or something totally different

2 minutes ago, alturismo said:

actually, yes ... as the commads follows each other

 

run your script, run mover, ... so your edge case would rather be

 

run mover (dry), if mover will take action, run script, run mover ...  if not, dont run the script ... let the mover "do nothing"

 

as the mover is running AFTER script ... no matter what

 

so either you make a "outside" check (user scripts as sample) before your sceduled mover triggers and may just replace the cfg like mentiones 3 posts upper yours "without" the pre run script option OR include the pre run script Option ...

 

or something totally different

can you provide me the command to launch mover tuning from terminal?

1 minute ago, jcofer555 said:

can you provide me the command to launch mover tuning from terminal?

actually that wasnt what i mentioned, but ... starting mover from terminal is simple as typing

 

mover

 thats it