1 hour ago, froggles said:

So I don't think this works?

exactly, subfolders wont work this way ...

 

1 hour ago, froggles said:

so don't want to have the innefficiency of copying them across to a different share when this occurs).

if its the same disk (cache pool) its a matter of setup, but nevermind ... otherwise alot of Media Tools etc would be hurting disk usage when moving Media Files around

 

may other solution, whatever you use to backup your files, when the job is done, just use a script which will move them away nightly, hourly, whatever ... either by a simple user script, or depending on your used tools something like "run after ..."

 

you should find a solution anyway

On 11/13/2024 at 3:41 PM, nebb00 said:

I have a number of items which will not move due to hardlinks, any ideas?

 

unraid move: Expected 2 files, got 1. Not moving "/mnt/cache/./data/media/audiobooks/REDACTED/REDACTED/REDACTED.m4b"  to prevent breaking hardlinks

didnt have this problem on the old fork. what am I missing now? Does mover not recreate the hardlinks anymore?

I have got the same problem, have you found a solution to it?
From what I read here, it seems to be intended behaviour as it has not found all hard-linked files in the to-be-move list. My current setup relies on breaking hard links, so this doesn't work for me.

I am regularly getting a syslog error which shouldn't be an error I guess?

 

Edit: Ok, mgutt just told me that this message comes from his script rather than yours. So please ignore this. Sorry!

Edited December 8, 20241 yr by SirLupus
Clarify

This new mover has several bugs.

 

Auto age is not moving any files.  It says it's going to, but it doesn't, so cache is filling up over time.  Setting a hard '2 days' works.
Mover is moving files off of cache shares to the array even though there is ample space available on the share, and the share is cache preferred.

Mover is moving the same file back and forth between the cache share and array.

 

Dec 10 06:00:40 tower root: Moving "/mnt/cache/./system/libvirt/libvirt.img"  to  /mnt/user0/ 
Dec 10 06:00:48 tower root: Deleting empty dir /mnt/cache/system/libvirt
Dec 10 06:00:48 tower root: 1 files remaining from caches to array  30GiB
Dec 10 06:00:48 tower root: Moving "/mnt/cache/./domains/Windows 10/vdisk1.img"  to  /mnt/user0/ 
Dec 10 06:07:48 tower root: Deleting empty dir /mnt/cache/domains/Windows 10

Dec 10 07:01:01 tower root: Moving "/mnt/disk5/./system/libvirt/libvirt.img"  to  /mnt/cache/ 
Dec 10 07:01:09 tower root: Deleting empty dir /mnt/disk5/system/libvirt
Dec 10 07:01:09 tower root: 1 files remaining from array to caches 30GiB
Dec 10 07:01:09 tower root: Moving "/mnt/disk6/./domains/Windows 10/vdisk1.img"  to  /mnt/cache/ 
Dec 10 07:06:27 tower root: Deleting empty dir /mnt/disk6/domains/Windows 10

Dec 10 08:00:39 tower root: Moving "/mnt/cache/./system/libvirt/libvirt.img"  to  /mnt/user0/ 
Dec 10 08:00:46 tower root: Deleting empty dir /mnt/cache/system/libvirt
Dec 10 08:00:46 tower root: 1 files remaining from caches to array  30GiB
Dec 10 08:00:46 tower root: Moving "/mnt/cache/./domains/Windows 10/vdisk1.img"  to  /mnt/user0/ 
Dec 10 08:07:25 tower root: Deleting empty dir /mnt/cache/domains/Windows 10

 

Again, there is no reason to move thils file.  The 'domains' share is cache preferred and is only at 35% usage, and Mover is configured for a threshold of 50% and a free down/prime up to 50%.

This file is only 30gb in size and the cache volume is 4TB.  With the file on the cache volume, it's still well below the 50% threshold.
 

Edited December 10, 20241 yr by foomanjee

4 minutes ago, foomanjee said:

Again, there is no reason to move thils file.  The 'domains' share is cache preferred and is only at 35% usage, and Mover is configured for a threshold of 50% and a free down/prime up to 50%.

i would say there is your ....

 

think about it, you play "ping pong" ... it should be something like 70 / 30 or 50 / 25 or ....

 

now you say, upper 50 go down under 50 ... so 1 byte over 50 will trigger a move to 1 byte under 50, and so on ...

 

but yes, it shouldnt be triggered while its under 50 ... but i guess its the combo of "ping pong" together with "cache prefered"

You're right, and I get that - the thing about it though is that this file is only 30gb on a 4tb cache where there is 2.5tb free space.  This 30gb file sitting on cache does not push the free space beyond the 50% mark.

I've switched to 60% threshold and free down/prime up to 50%, and it's still moving the file.  The cache is not at 60%.  It's decided to pick up yet another file, and says it's going to move from the array to cache, but it's actually moving things in the other direction.
 

Dec 10 10:26:46 tower move: ------------------------------------------------------------ Processing domains share -----------------------------------------------------------
Dec 10 10:26:46 tower move: Primary storage: cache - size: 3.7TiB - used:  36 % (1.4TiB)
Dec 10 10:26:46 tower move: Secondary storage: user0
Dec 10 10:26:46 tower move: Share Information: Name: domains - Path: /mnt/*/domains
Dec 10 10:26:46 tower move: Moving threshold: 60% (2.2TiB) ; Freeing threshold: 50% (1.9TiB)
Dec 10 10:26:46 tower move: Mover action: user0->cache (cache:prefer). Pool is below priming threshold percentage:  36% < 50%.
Dec 10 10:26:46 tower move: => Will smart move newest files from user0 to cache until threshold. Older files will be moved from cache to user0.
Dec 10 10:26:46 tower move: Adding Age
Dec 10 10:26:46 tower move: Age (modification time) 2
Dec 10 10:26:46 tower move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-12-10T102614.list for domains
  
Dec 10 10:26:47 tower move: ------------------------------------------------------------- Processing system share -----------------------------------------------------------
Dec 10 10:26:47 tower move: Primary storage: cache - size: 3.7TiB - used:  36 % (1.4TiB)
Dec 10 10:26:47 tower move: Secondary storage: user0
Dec 10 10:26:47 tower move: Share Information: Name: system - Path: /mnt/*/system
Dec 10 10:26:47 tower move: Moving threshold: 60% (2.2TiB) ; Freeing threshold: 50% (1.9TiB)
Dec 10 10:26:47 tower move: Mover action: user0->cache (cache:prefer). Pool is below priming threshold percentage:  36% < 50%.
Dec 10 10:26:47 tower move: => Will smart move newest files from user0 to cache until threshold. Older files will be moved from cache to user0.
Dec 10 10:26:47 tower move: Adding Age
Dec 10 10:26:47 tower move: Age (modification time) 2
Dec 10 10:26:47 tower move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-12-10T102614.list for system

  
Dec 10 10:26:48 tower move: Moving "/mnt/cache/./system/libvirt/libvirt.img"  to  /mnt/user0/ 
Dec 10 10:26:55 tower move: Deleting empty dir /mnt/cache/system/libvirt
Dec 10 10:26:55 tower move: 1 files remaining from caches to array  30GiB
Dec 10 10:26:55 tower move: Moving "/mnt/cache/./domains/Windows 10/vdisk1.img"  to  /mnt/user0/ 

 

 

 

EDIT / Addition:
2 hours later and it's still bouncing these two files back and forth, even while set to 60/50

 

Dec 10 11:00:44 tower root: Primary storage: cache - size: 3.7TiB - used:  35 % (1.3TiB)
Dec 10 11:00:44 tower root: Secondary storage: user0
Dec 10 11:00:44 tower root: Share Information: Name: domains - Path: /mnt/*/domains
Dec 10 11:00:44 tower root: Moving threshold: 60% (2.2TiB) ; Freeing threshold: 50% (1.9TiB)
Dec 10 11:00:44 tower root: Mover action: user0->cache (cache:prefer). Pool is below priming threshold percentage:  35% < 50%.
Dec 10 11:00:44 tower root: => Will smart move newest files from user0 to cache until threshold. Older files will be moved from cache to user0.
Dec 10 11:00:44 tower root: Adding Age
Dec 10 11:00:44 tower root: Age (modification time) 2
Dec 10 11:00:44 tower root: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-12-10T110002.list for domains
  
Dec 10 11:00:47 tower root: Moving "/mnt/disk5/./system/libvirt/libvirt.img"  to  /mnt/cache/ 
Dec 10 11:00:54 tower root: Deleting empty dir /mnt/disk5/system/libvirt
Dec 10 11:00:54 tower root: 1 files remaining from array to caches 30GiB
Dec 10 11:00:54 tower root: Moving "/mnt/disk6/./domains/Windows 10/vdisk1.img"  to  /mnt/cache/ 
Dec 10 11:01:19 tower apcupsd[2300]: Communications with UPS lost.
Dec 10 11:08:02 tower root: Deleting empty dir /mnt/disk6/domains/Windows 10
Dec 10 11:08:02 tower root: 0 files remaining from array to caches 0B
  
--  
  
Dec 10 12:00:43 tower root: Primary storage: cache - size: 3.7TiB - used:  36 % (1.4TiB)
Dec 10 12:00:43 tower root: Secondary storage: user0
Dec 10 12:00:43 tower root: Share Information: Name: domains - Path: /mnt/*/domains
Dec 10 12:00:43 tower root: Moving threshold: 60% (2.2TiB) ; Freeing threshold: 50% (1.9TiB)
Dec 10 12:00:43 tower root: Mover action: user0->cache (cache:prefer). Pool is below priming threshold percentage:  36% < 50%.
Dec 10 12:00:43 tower root: => Will smart move newest files from user0 to cache until threshold. Older files will be moved from cache to user0.
Dec 10 12:00:43 tower root: Adding Age
Dec 10 12:00:43 tower root: Age (modification time) 2
Dec 10 12:00:43 tower root: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-12-10T120002.list for domains
  
Dec 10 12:00:45 tower root: Moving "/mnt/cache/./system/libvirt/libvirt.img"  to  /mnt/user0/ 
Dec 10 12:00:52 tower root: Deleting empty dir /mnt/cache/system/libvirt
Dec 10 12:00:52 tower root: 1 files remaining from caches to array  30GiB
Dec 10 12:00:52 tower root: Moving "/mnt/cache/./domains/Windows 10/vdisk1.img"  to  /mnt/user0/ 
Dec 10 12:01:36 tower apcupsd[2300]: Communications with UPS lost.
Dec 10 12:07:58 tower root: Deleting empty dir /mnt/cache/domains/Windows 10
Dec 10 12:07:58 tower root: 0 files remaining from caches to array  0B

 

Edited December 10, 20241 yr by foomanjee

Hey, i think mover tuning plugin breaks the new feature "empty disk"
When i run

mover start -e disk2 |& logger &

i get this output in syslog:
 

Dec 12 00:14:16 Tower root: mover: started
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs unmount 'cache/backup' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs destroy 'cache/backup' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs mount 'cache/backup' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs unmount 'cache/filerun' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs destroy 'cache/filerun' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs mount 'cache/filerun' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs unmount 'cache/nextcloud' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs destroy 'cache/nextcloud' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs mount 'cache/nextcloud' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs unmount 'cache/safe' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs destroy 'cache/safe' 2>&1
Dec 12 00:14:16 Tower shfs: /usr/sbin/zfs mount 'cache/safe' 2>&1
Dec 12 00:14:16 Tower root: mover: finished

Though when i run
 

mover.old start -e disk2 |& logger &

it works and empties the disk:
 

Dec 12 00:19:31 Tower root: mover: started
Dec 12 00:19:33 Tower root: move: /mnt/disk2/.... Success
Dec 12 00:20:09 Tower root: move: /mnt/disk2/.... Success
Dec 12 00:20:10 Tower root: move: /mnt/disk2/.... Success
Dec 12 00:20:54 Tower root: move: /mnt/disk2/.... Success
Dec 12 00:21:30 Tower root: move: /mnt/disk2/.... Success
Dec 12 00:21:30 Tower root: move: /mnt/disk2/.... Success

 

Edited December 11, 20241 yr by Joly0

I think there is an issue with the mover tuning plugin combined with 7 RC1:

 

 

 

Mover dit nothing on RC1 when activated manually. Tuning was setup to have move button not follow the tuning rules. After removing the plugin all was working again.

I'd suspect the update to RC1 replaces the plugin mover script with the old mover again and you should be able to get it working again after re-installing the plugin unless unraid has changed something significant, but with RC1 being more of less the same as Beta4 it shouldnt be too much of an issue.

 

when installing mover tuning it renames the stock mover to "mover.old" and puts its own in for the scheduler to execute.

Seeking help as a new unraid user (first post here!) who configured this plugin hoping to move files from my cache_ssd drive once it hits 75-80% to move to my array and stop around 50%.

But unfortunately somehow it hasnt been working and my drive is now nearly 95% full. Would someone be able to review my settings to check what I am doing wrong here?

Thanks in advance!

 

 

it says the mover is running, so whats in the logs? perhaps it's jammed on something and it's filling up faster than it is clearing.

 

Also, try turning on file move based on age, set to auto. that could kick it into action.

 

to stop the frozen mover use "mover stop" in the terminal, then click move now to start it again.

On 12/12/2024 at 9:54 PM, Faceman said:

I'd suspect the update to RC1 replaces the plugin mover script with the old mover again and you should be able to get it working again after re-installing the plugin unless unraid has changed something significant, but with RC1 being more of less the same as Beta4 it shouldnt be too much of an issue.

 

when installing mover tuning it renames the stock mover to "mover.old" and puts its own in for the scheduler to execute.

This cannot be the case as you need to reboot to activate the new release which gives the plugin a chance during the boot process to repeat replacing the stock mover file.   Something else is causing the problem.

I'll do some deep diving when I update from Beta4 to the RC then.

17 hours ago, Faceman said:

it says the mover is running, so whats in the logs? perhaps it's jammed on something and it's filling up faster than it is clearing.

 

Also, try turning on file move based on age, set to auto. that could kick it into action.

 

to stop the frozen mover use "mover stop" in the terminal, then click move now to start it again.

Woah, stopping the mover and clicking move seems to have moved the files! Not sure what was wrong earlier..

 

Thanks a lot for the help!

i'm trying to have a script run after mover finishes and so i've put in a path of /mnt/cache/data/computer/unraidstuff/script.sh in the Script to run after mover (No checks, always runs): field and confirmed the path is correct, i set 777 perms on the file and enabled mover logging, ran mover and it says file not found. any ideas what i can try? it's just a bash script to remove empty folders as the clean empty folders setting doesn't seem to work for my needs. i tested this script in user scripts and it runs and works as intended

i am running unraid 7 rc1 and all other parts of the mover tuning works just doesn't run the after script because of the not found error

Edited December 18, 20241 yr by jcofer555
additional info

On 11/25/2024 at 11:11 PM, mcgerk said:

Same here, ignoring files doesn‘t work. (7.0.0-beta4)

Has this been addressed for 7.0.0-rc.1?

On 12/19/2024 at 3:05 AM, Janis.S said:

Has this been addressed for 7.0.0-rc.1?

rc2 is out now as well, might try on both

So, I was also having issues with compatability with unraid 7 betas and the release candidate

Just checked the community apps, and there appear to be 2 different versions of this plugin, both of which point back to this same support thread.

The one from "Squid's" repository does not appear to have been updated since December 2023 (this is the one I had), and the new one from "Reynald's" repository has only been created July this year, and has been updated much more recently, and it does include support for 7 and some additional features.

No idea if there has been an "official" change to the new plugin, but now that I know what to search the thread for I was able to find a few people saying pretty much the same thing.

long story short, I had to uninstall the old plugin (take a screenshot of your config so its easier to set up the new one, settings to not transfer across) and then install the new plugin, then reconfigure, and boom, shes all good again.

yea it's a common pitfall, almost every reddit post about mover tuning issues was due to people running the old version without realizing it.

updated to RC2 over the weekend and still no issues to report.

Hello, I'm looking for some help with Mover Tuner. Few months ago installed Reynald's app, checked it's up to date 2024.09.05.0222. Unraid version 6.12.14

I'm getting errors in the Syslog, for example:

/mnt/cache/Applications does not exist. Is the share still used? Consider removing /boot/config/shares/Applications.cfg if not.

At one point I must have told that share to use cache, but the share now says to only use Array

 

So I open terminal and remove the Applications.cfg. But it keeps coming back. There's about 5 of these cfg files that I want to remove, but they all keep coming back. The rm command completes successfully

/boot/config/shares# rm Applications.cfg -v
removed 'Applications.cfg'

But ls -als shows the file is still there with a new timestamp of now

 

What's going on!??

 

Edit: So I've just realised that these cfg files are not a part of Mover Tuner, but of the share config of Unraid.

I've uninstalled the Mover Tuner

 

It now can be run manually using "Move Now" button and all the "unprotected files" warnings have disappeared from my shares list, except for appdata. The Mover is skipping .log files, and .mca files for some reason:

skip: /mnt/cache/appdata/binhex-minecraftserver-2/minecraft/logs/screen.log

move: skip: /mnt/cache/appdata/binhex-minecraftserver-2/minecraft/logs/latest.log

move: skip: /mnt/cache/appdata/binhex-minecraftserver-2/minecraft/world/region/r.0.0.mca 

move: skip: /mnt/cache/appdata/binhex-minecraftserver-2/minecraft/world/region/r.0.1.mca

move: skip: /mnt/cache/appdata/binhex-minecraftserver-2/minecraft/world/region/r.1.0.mca 

etc

 

I just want it to move everything

 

Thanks

Edited December 23, 20241 yr by Arran Coote

Did you make sure that the docker service was disabled while doing this?   Mover will ignore open files so you need to make sure there is nothing that can keep open files you want to be moved.

Hello all,

 

After updating to the new (non-squid) version of tuned mover, I am no longer able to call the "/usr/local/emhttp/plugins/ca.mover.tuning/age_mover" from a custom script as I used to, since it seems to ignore all passed arguments and only pull variables from the "/boot/config/plugins/ca.mover.tuning/ca.mover.tuning.cfg" that is set from the SETTINGS page.

 

Is there any way I can run this new update script with custom variables passed as arguments like I was able to do with Squid's version?

 

Thanks

I'm trying to invoke the mover with the "Move Now" button after enabling the option to use the plugin version with said button, and am encountering an error message about "/usr/local/sbin/move" not existing, then failing to create the symbolic link because it already exists. Any ideas?

 

Dec 29 15:11:17 Denton emhttpd: shcmd (6066204): /usr/local/sbin/mover start |& logger -t move &
Dec 29 15:11:17 Denton root: Starting Mover
Dec 29 15:11:17 Denton root: Forcing turbo write on
Dec 29 15:11:17 Denton kernel: mdcmd (198): set md_write_method 1
Dec 29 15:11:17 Denton kernel: 
Dec 29 15:11:17 Denton root: ionice -c 2 -n 0 nice -n 0 /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start 3 0 0 '' '' '' '' no '' '' '' 0
Dec 29 15:11:17 Denton move: Log Level: 1
Dec 29 15:11:17 Denton move: /usr/local/sbin/move does not exists, creating soft link
Dec 29 15:11:17 Denton move: ln: failed to create symbolic link '/usr/local/sbin/move': File exists
Dec 29 15:11:17 Denton move: mover: started
Dec 29 15:11:18 Denton move: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 250: /usr/local/sbin/move: No such file or directory

 

The last line repeats many times, presumably for each of the files that match the age requirement.