[Plugin] Mover Tuning

hugenbdd · February 17

10 minutes ago, g.strange42 said:

Just to give more Info, I tried reinstalling it twice, and stopping all the dockers that use the cache.

Could you post a screen shot of your settings?

Could be due to Priority for mover process or Priority for disk I/O, something may have changed with those in this release.... But I won't know till I get a chance to test more.

g.strange42 · February 17

Just now, hugenbdd said:

Could you post a screen shot of your settings?

Could be due to Priority for mover process or Priority for disk I/O, something may have changed with those in this release.... But I won't know till I get a chance to test more.

Other than adding an ignore list all setting are default.

modem7 · February 18

On 9/12/2023 at 12:32 PM, 2TC said:
When I boot up my server I get the following error in the output of the screen
mv: cannot stat '/usr/local/bin/mover': No such file or directory
it is not really an issue, I searched the web and I found this post talking about this and the addon "move tuning plugin", which I assume is "CA Mover Tuning". The post talks about deleting the plugin. Now I do not have an issue with the server not starting, just that the entry is there.

I looked at the '/usr/loval/bin/mover' which is a shell script
#!/bin/bash
PPPID=`ps h -o ppid= $PPID 2>/dev/null`
P_COMMAND=`ps h -o %c $PPPID 2>/dev/null`
/usr/local/emhttp/plugins/ca.mover.tuning/mover.php $P_COMMAND $1
I looked at the file mentioned, and the mover.php file is where it is supposed to be, seems everything is working as intended, except that I get the error when booting it, surely I could just ignore it. But I rather fix it than leave it there.

I'm also getting this error.

Not quite sure if I should be concerned about it or not.

BazzT92 · February 23

Since i installed this MOVER TUNING Plugin under 6.12.8 i am also getting this error now.

mv: cannot stat '/usr/local/bin/mover': No such file or directory

Mover won't start,

had to uninstall the Plugin,

start mover, stop mover,

install the plugin again,

now it moves,

but I dont know how long 😑.

wgstarks · February 23

2 hours ago, BazzT92 said:
Since i installed this MOVER TUNING Plugin under 6.12.8 i am also getting this error now.
mv: cannot stat '/usr/local/bin/mover': No such file or directory
Mover won't start,

had to uninstall the Plugin,

start mover, stop mover,

install the plugin again,

now it moves,

but I dont know how long 😑.

I’m also getting the same error.

Swarles · February 25

@2TC @modem7 @BazzT92 @wgstarks
Hmm I can't seem to recreate this issue after having updated to 6.12.8. Would someone with the issue be able to clarify the following:

Does this error message happen whenever Mover is trying to start or during boot of the machine?
Does this error message happen during scheduled mover or when using "Move" button on main page?
(With "Move now button uses mover tuning" enabled.)

Can you verify that the file exists manually?

ls -l /usr/local/bin/mover

Output:
-rwxr-xr-x 1 root root 161 Feb 25 11:05 /usr/local/bin/mover*

Are the permissions the same as above?

Check the cron entry and verify

cat /etc/cron.d/root

Output:
# Generated mover schedule:
40 3 * * * /usr/local/sbin/mover |& logger

Check if your output is the same.

If you verified both steps 3 and 4 to be true, try uninstalling the plugin, reboot, and reinstall.
You can backup your settings with the following:
```
cp -r /boot/config/plugins/ca.mover.tuning /boot/config/ca.mover.tuning.backup
```
Once you have reinstalled, you can restore your settings with:
```
mv -f /boot/config/ca.mover.tuning.backup /boot/config/plugins/
```
Let me know if issue persists.

Edited February 25 by Swarles

wgstarks · February 25

For me the error has only showed up on reboot in the console. Not in the system log.

Started mover manually (didn't want to wait an hour) and see this in the system log-

Feb 24 20:42:21 Brunnhilde emhttpd: shcmd (129): /usr/local/sbin/mover &> /dev/null &
Feb 24 20:42:21 Brunnhilde root: Starting Mover
Feb 24 20:42:21 Brunnhilde root: ionice -c 2 -n 0 nice -n 0 /usr/local/sbin/mover.old

Seems to work properly.

root@Brunnhilde:~# ls -l /usr/local/bin/mover
-rwxr-xr-x 1 root root 161 Feb 24 20:28 /usr/local/bin/mover*
root@Brunnhilde:~#

# Generated mover schedule:
0 */1 * * * /usr/local/sbin/mover &> /dev/null

Tried uninstall-reboot-reinstall. Didn't work. performed another reboot after the reinstall and still seeing the error.

Swarles · February 25

42 minutes ago, wgstarks said:
For me the error has only showed up on reboot in the console. Not in the system log.

Started mover manually (didn't want to wait an hour) and see this in the system log-
Feb 24 20:42:21 Brunnhilde emhttpd: shcmd (129): /usr/local/sbin/mover &> /dev/null &
Feb 24 20:42:21 Brunnhilde root: Starting Mover
Feb 24 20:42:21 Brunnhilde root: ionice -c 2 -n 0 nice -n 0 /usr/local/sbin/mover.old 
Seems to work properly.
root@Brunnhilde:~# ls -l /usr/local/bin/mover
-rwxr-xr-x 1 root root 161 Feb 24 20:28 /usr/local/bin/mover*
root@Brunnhilde:~#
# Generated mover schedule:
0 */1 * * * /usr/local/sbin/mover &> /dev/null
Tried uninstall-reboot-reinstall. Didn't work. performed another reboot after the reinstall and still seeing the error.

Oh okay during boot makes sense, I have found the issue.

line 113 of "ca.mover.tuning.plg"

if [[ ! -f /usr/local/bin/mover.old ]]; then mv /usr/local/bin/mover /usr/local/bin/mover.old; fi

I uninstalled the plugin and rebooted and it would appear that unraid does not initialise a "mover" file in "/usr/local/bin/mover" (on 6.12.8 at least). "move" is located in "bin" and "mover" is located in "sbin".

I don't know enough about unraid to know why this is the case or what the configuration is in older OS. It's probably important for people on older OS. I can confirm that this is not an issue to worry about though.

We could change it so it checks if "mover" exists rather than if "mover.old" doesn't exist, thoughts @hugenbdd?
If it is really bothering anyone, for the time being you can remove the line in "/boot/config/plugins/ca.mover.tuning.plg" or suppress its output.

Swarles · February 25

FAQ - Frequently Asked Questions

I'm putting these here for some of the common problems, and will try to update it as more questions/issues come along.

How do I enable logs / provide logs?
Spoiler
1. Go to Settings>Scheduler>Mover Settings
2. Set "Mover Logging" to Enabled.
3. Go to the Mover Tunning settings and set "Test Mode" to "Yes"
4. Set "Move Now button follows plug-in filters" to "Yes".
5. On the Main tab press the "Move" button to run the mover. Running in test mode will generate logs/lists but not make actual moves.
6. Go to "/tmp/Mover" and get the latest version of the requested file.
7. You can revert test mode and logging after this is done.
  
  NOTE: Please put the logs in a code block ( </> ) inside a spoiler (the eye logo) so the comment is not massive.
Mover wont run after installing Mover Tunning?
Spoiler
Check share.cfg - "cache not enabled"
Spoiler
cat "/boot/config/share.cfg"

Check to see if the following line exists 'shareCacheEnabled="yes"'.
If not or the file doesn't exist:

echo 'shareCacheEnabled="yes"' >> /boot/config/share.cfg
Check ca.mover.tuning.cfg - "ca.mover.tuning.cfg does not exist"
Spoiler
cat "/boot/config/plugins/ca.mover.tuning"

If the file does not exist or it appears fields are missing, go to "Settings>Scheduler>Mover Tuning", make some changes and save. See if it exists now. If not report the issue.
Check to see your share names are correct.
Spoiler
ls "/boot/config/shares"

Make sure the "[share_name].cfg" matches up exactly with the name of your share (case sensitive). If not, rename the share in "Shares>{share_name]" to something different even if it appears correct in this tab. Save it and then you can reverse this process. Verify that it appears correctly now as above.
Mover isn't ignoring files or folders?
Spoiler
1. Make sure your [skip_list_name].txt is readable.
2. Make sure path entries in your skip list go through the "[cache_pool_name]" NOT "user". eg.
```
/mnt/user/data/media/linux-isos   <--Bad
/mnt/cache/data/media/linux-isos  <--Good
/mnt/cache/data/usenet/incomplete <--Good
```
3. Some things to note:
  1. You can put individual files as well as directories in the skip list.
  2. Don't append directory paths with /*, unless you really want to.
  3. I recommend keeping the list on the boot drive so the mover will never touch it. Otherwise make the path to the list, the first entry on the list.
  4. Make the list using 'nano' to ensure it has unix style line breaks.

Issue Reporting Template

Quote

Unraid Version: 6.12.8

Mover Tuning Version: 2023.12.19
Issue: eg. Mover not running.

Edited March 4 by Swarles

Revan335 · February 25

6 hours ago, Swarles said:

FAQ - Frequently Asked Questions

Please ad to Recommended Posts.

hugenbdd · February 25

11 hours ago, Swarles said:
Oh okay during boot makes sense, I have found the issue.

line 113 of "ca.mover.tuning.plg"
if [[ ! -f /usr/local/bin/mover.old ]]; then mv /usr/local/bin/mover /usr/local/bin/mover.old; fi
I uninstalled the plugin and rebooted and it would appear that unraid does not initialise a "mover" file in "/usr/local/bin/mover" (on 6.12.8 at least). "move" is located in "bin" and "mover" is located in "sbin".

I don't know enough about unraid to know why this is the case or what the configuration is in older OS. It's probably important for people on older OS. I can confirm that this is not an issue to worry about though.

We could change it so it checks if "mover" exists rather than if "mover.old" doesn't exist, thoughts @hugenbdd?
If it is really bothering anyone, for the time being you can remove the line in "/boot/config/plugins/ca.mover.tuning.plg" or suppress its output.

the mover.old file is so that we can restore the original mover binary. The location changed several releases ago but I'm not certain what version. If we remove this then it will not work for older releases. I will try and work on a way around this to satisfy older and newer releases this week. Which I think will be a "longer" if statement, and to check for the file before mv'ing it and also creating another if to create a copy for the sbin location file.

KnifeFed · February 26

On 2/25/2024 at 11:30 AM, Revan335 said:

Please ad to Recommended Posts.

And ad a link in the plugin description 😉

noonamer · February 27

Thanks for the decent plugin!
I'm curious: can I trigger mover for only single user share from command line?

nebb00 · February 27

Is it possible to ignore a directory including all sub items?

but then again i cannot get the exclude files to even work.

Quote

cat /mnt/cache/excluded_moves.txt
/mnt/cache/data/downloads/bentest.txt

Ignore files listed inside of a text file: Yes

File list path: /mnt/cache/excluded_moves.txt

Quote

mvlogger: Cache Pool Name: cache
mvlogger: Share Path: /mnt/cache/data
mvlogger: Complete Mover Command: find "/mnt/cache/data" -depth | /usr/local/sbin/move -d 1
file: /mnt/cache/data/downloads/bentest.txt
mvlogger: Tue Feb 27 16:09:00 GMT 2024

Edited February 27 by nebb00

Renegade605 · February 27

Today I had my cache drive fill but the mover not move any files. I've determined the cause to be the method of reading used disk space on ZFS.

Move All threshold is set to 80%, and the cache pool's used space is 97%. But, there are datasets on the cache pool with reservations and quotas, so the actual used space is 74% even though there is no space remaining for some shares.

image.png.2ce40b4f0f450df48c8dc60367489878.png

Can the mover tuning plugin evaluate used space on a per-dataset basis instead of a whole pool basis? Or is there some other solution that would solve this problem?

EDIT TO ADD: I also notice now that the mover does not seem to be respecting the Ignore hidden files and directories option.

Feb 27 09:41:55 Global-Dynamics move: file: /mnt/cache/movies/.radarr/Arrivl.2016.MULTi.2160p.UHD.BluRay.x265-OohLaLa/t82hgxa6svubn8lh42nkbjs4vjd5bfn50b/Arrival.2016.MULTi.2160p.UHD.BluRay.x265-OohLaLa/Proof/arrival.2016.multi.2160p.uhd.bluray.x265-oohlala.proof.jpg

Also Important

This error brought my attention that the mover destroys empty datasets when finished.

Please remove this step or at least make it an option to be decided by the user!

Datasets have settings that should be maintained and they should never be destroyed without explicit instruction from the sysadmin. For now I've taken a snapshot of the empty datasets to force this step to fail.

Edited February 27 by Renegade605

Swarles · February 29

On 2/28/2024 at 1:20 AM, noonamer said:

Thanks for the decent plugin!
I'm curious: can I trigger mover for only single user share from command line?

The way it's done on the share page is an onclick event that calls the following:

$.post("/plugins/ca.mover.tuning/moveShareNow.php?Share='<?echo $shareName?>'");

The "moveShareNow.php" script executes the following:

exec("/usr/local/emhttp/plugins/ca.mover.tuning/share_mover {$shareName} >> /var/log/syslog &", $output, $retval);

You can definitely call that script from cmd and just provide the share name as an argument:

bash /usr/local/emhttp/plugins/ca.mover.tuning/share_mover 'shareName'

NOTE: The share_mover script in the current plugin does not consider the mover tuning settings and will move everything off that share

Swarles · February 29

On 2/28/2024 at 2:58 AM, nebb00 said:

Is it possible to ignore a directory including all sub items?

but then again i cannot get the exclude files to even work.

Ignore files listed inside of a text file: Yes

File list path: /mnt/cache/excluded_moves.txt

See item #3 on the FAQ.
Make sure the file is readable, double check the file is actually in that location, it's likely that the mover could have moved it after the first run. If it still isn't working, you'll have to provide the latest ".logs" files for me to see what is going on. See item #1 on the FAQ for providing logs

Swarles · February 29

On 2/28/2024 at 3:35 AM, Renegade605 said:

Today I had my cache drive fill but the mover not move any files. I've determined the cause to be the method of reading used disk space on ZFS.

Move All threshold is set to 80%, and the cache pool's used space is 97%. But, there are datasets on the cache pool with reservations and quotas, so the actual used space is 74% even though there is no space remaining for some shares.

Can the mover tuning plugin evaluate used space on a per-dataset basis instead of a whole pool basis? Or is there some other solution that would solve this problem?

I think it would be possible but It's not something I would want to implement in the current mover tuning code base.

I am currently in the process of building the mover tuning plugin from the ground up fully in php, using OOP practices and I'll think about the best way to implement evaluating a threshold on a per dataset basis. I'm not sure if I would ship it in the first release but the idea of migrating to OOP would be to make implementing stuff like this easier.

For the time being it will remain a whole pool basis unless someone else wants to implement it. In light of that which one do you think would be better to use going forward?

zpool get -o value capacity
OR
df --output=pcent

On 2/28/2024 at 3:35 AM, Renegade605 said:
EDIT TO ADD: I also notice now that the mover does not seem to be respecting the Ignore hidden files and directories option.
Feb 27 09:41:55 Global-Dynamics move: file: /mnt/cache/movies/.radarr/Arrivl.2016.MULTi.2160p.UHD.BluRay.x265-OohLaLa/t82hgxa6svubn8lh42nkbjs4vjd5bfn50b/Arrival.2016.MULTi.2160p.UHD.BluRay.x265-OohLaLa/Proof/arrival.2016.multi.2160p.uhd.bluray.x265-oohlala.proof.jpg

That's weird, can you confirm if that file appears if you perform the following:

find '/mnt/cache/movies' -depth -not -path '*/\.*'

If not, you might need to confirm what find string is being used in the logs.

On 2/28/2024 at 3:35 AM, Renegade605 said:

Also Important

This error brought my attention that the mover destroys empty datasets when finished.

Please remove this step or at least make it an option to be decided by the user!

Datasets have settings that should be maintained and they should never be destroyed without explicit instruction from the sysadmin. For now I've taken a snapshot of the empty datasets to force this step to fail.

I couldn't find any code relating to destroying datasets which leads me to believe this is a feature baked into the built in mover.

I could possibly add a check to not send datasets to the mover when that setting is enabled.
This will definitely be an option when I finish building the new mover tuning plugin.

Renegade605 · February 29

12 hours ago, Swarles said:
For the time being it will remain a whole pool basis unless someone else wants to implement it. In light of that which one do you think would be better to use going forward?
zpool get -o value capacity
OR
df --output=pcent

Both have issues. zpool commands use the raw capacity of the pool (including parity drives, etc.) and ignore the properties of the underlying filesystem. df is too aware of the filesystem and ignores everything in the datasets.

image.png.c1443ceb08d8974f776267160c092f58.png

After a little playing around... How about zfs get instead?

image.png.3f9eb97bd58a1a6eed14cb2276431c76.png

318 / (318 + 580) = 35.41%

More than close enough to 342GB / 965GB = 35.44%

(Looks like just imprecision with GB vs GiB)

Get precise with the -p flag.

image.png.bb76267840efea2b9f2900f2ca14e637.png

Plus some bash magic: (there was definitely an easier way to do this if it doesn't have to be a one line command lol)

expr `zfs get -Hp -o value used cache` * 100 / `zfs get -Hp -o value used,available cache | awk '{s+=$1} END {print s}'`

image.png.5e2243a7cc1c832bfb9d188ba8be49f8.png

Tested again while letting the pool fill up:

image.png.68a28c415323bb25526c489da143863e.png

12 hours ago, Swarles said:
That's weird, can you confirm if that file appears if you perform the following:
find '/mnt/cache/movies' -depth -not -path '*/\.*'
If not, you might need to confirm what find string is being used in the logs.

I just realized this was for a "Move All" so that's got to be why.

12 hours ago, Swarles said:

I couldn't find any code relating to destroying datasets which leads me to believe this is a feature baked into the built in mover.

Grrr Unraid

12 hours ago, Swarles said:

I could possibly add a check to not send datasets to the mover when that setting is enabled.
This will definitely be an option when I finish building the new mover tuning plugin.

Awesome! Thanks a lot. For now, the snapshot of datasets while empty is working, other than cluttering up the logs with errors.

Edited February 29 by Renegade605

isvein · March 4

Hello

How is the file for the "Ignore files listed inside of a text file" function supposed to be formatted and can it be used to ignore complete folders too?

Swarles · March 4

4 minutes ago, isvein said:

Hello

How is the file for the "Ignore files listed inside of a text file" function supposed to be formatted and can it be used to ignore complete folders too?

See #3 on the FAQ.

noonamer · March 4

On 2/29/2024 at 3:27 AM, Swarles said:
The way it's done on the share page is an onclick event that calls the following:
$.post("/plugins/ca.mover.tuning/moveShareNow.php?Share='<?echo $shareName?>'");
The "moveShareNow.php" script executes the following:
exec("/usr/local/emhttp/plugins/ca.mover.tuning/share_mover {$shareName} >> /var/log/syslog &", $output, $retval);
You can definitely call that script from cmd and just provide the share name as an argument:
bash /usr/local/emhttp/plugins/ca.mover.tuning/share_mover 'shareName'
NOTE: The share_mover script in the current plugin does not consider the mover tuning settings and will move everything off that share

there is no way to use it with Tunner settings?

Maybe, there is a way to move just particular files? I think "/usr/local/sbin/move", but I'm not sure it will handle hardlinks?

Swarles · March 4

1 hour ago, noonamer said:

there is no way to use it with Tunner settings?

Yeah unfortunately the way the individual share mover script is implemented it doesn’t use the mover tuning settings. It honestly wouldn’t be the hardest thing to implement so I might do it, but I’m also working on revamping the plug-in from the ground up so that’s where my limited time is being spent at the moment.

For now the easiest thing you can do if you want to use the tuning settings but move only 1 share (or a few), would be to put the path for all the other shares in the skip files list. eg.

/mnt/cache1/share1
/mnt/cache1/share2
/mnt/cache2/backups
etc

This will work exactly as expected, albeit quite inefficient as it will still loop through each share but just find no files.

1 hour ago, noonamer said:

Maybe, there is a way to move just particular files? I think "/usr/local/sbin/move", but I'm not sure it will handle hardlinks?

Honestly I’m not really sure how the built in mover handles hard-links, but I can tell you that when the mover tuning plugin detects a hard-link in the list of files that should be moved, it sends the entire list of files as a single group to the mover (sbin/move). Otherwise it sends only 1 file path at a time, I didn’t implement this and so I am not sure if this a requirement or not.

Edited March 4 by Swarles

noonamer · March 5

23 hours ago, Swarles said:
Yeah unfortunately the way the individual share mover script is implemented it doesn’t use the mover tuning settings. It honestly wouldn’t be the hardest thing to implement so I might do it, but I’m also working on revamping the plug-in from the ground up so that’s where my limited time is being spent at the moment.

For now the easiest thing you can do if you want to use the tuning settings but move only 1 share (or a few), would be to put the path for all the other shares in the skip files list. eg.
/mnt/cache1/share1
/mnt/cache1/share2
/mnt/cache2/backups
etc
This will work exactly as expected, albeit quite inefficient as it will still loop through each share but just find no files.

Honestly I’m not really sure how the built in mover handles hard-links, but I can tell you that when the mover tuning plugin detects a hard-link in the list of files that should be moved, it sends the entire list of files as a single group to the mover (sbin/move). Otherwise it sends only 1 file path at a time, I didn’t implement this and so I am not sure if this a requirement or not.

I have custom scripts which filter files by some criteria and I would like to move just these files. But, also the issue with hardlinks within particular files I want to move their hardlinks. So I would love if I can just drop file paths to the mover and it does all the magic for me

Swarles · March 5

7 hours ago, noonamer said:

I have custom scripts which filter files by some criteria and I would like to move just these files. But, also the issue with hardlinks within particular files I want to move their hardlinks. So I would love if I can just drop file paths to the mover and it does all the magic for me

In that case you should be able to echo the list of file paths and pipe them into "/usr/local/sbin/move -d".

[Plugin] Mover Tuning

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Squid

hugenbdd

hugenbdd

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

FAQ - Frequently Asked Questions

Issue Reporting Template

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Also Important

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation