[Plugin] Mover Tuning

[knights_of_pine]

On 3/27/2024 at 3:33 AM, ezek1el3000 said:

 

 

I am still getting the following error on boot:

 

mv: cannot stat '/usr/local/bin/mover': No such file or directory

Server boots up fine and only hangs for a couple of seconds on this error. Latest CA Version 2023.12.19

 

 

The error has only showed up on (re)boot in the console

 

ls -l /usr/local/bin/mover

Output:
-rwxr-xr-x 1 root root 161 Mar 26 15:01 /usr/local/bin/mover*

 

cat /etc/cron.d/root

Output: # Generated mover schedule:
0 8 * * * /usr/local/sbin/mover &> /dev/null

 

I am getting this same error.  first reboot in a few months to update to 6.12.9

[cui]

Hi, please tell me which one has higher priority, setting up mover rules in shared files or setting up mover rules in scheduled tasks. I actually only want to make special mover move rules for certain shared folders, should I disable mover tuning in scheduled tasks and enable mover tuning in shared folders so that I can achieve my purpose. Thanks for the plugin, looking forward to your reply

 

[Swarles]

15 hours ago, cui said:

Hi, please tell me which one has higher priority, setting up mover rules in shared files or setting up mover rules in scheduled tasks. I actually only want to make special mover move rules for certain shared folders, should I disable mover tuning in scheduled tasks and enable mover tuning in shared folders so that I can achieve my purpose. Thanks for the plugin, looking forward to your reply

 

Share override settings take higher priority than the global mover tuning settings.

 

If you disable mover tuning it will also disable the share override settings. 

 

Leave everything default in the global "scheduler settings" and the mover will behave has default. Then you can make changes in the individual shares. If you want a share to not use the mover at all, disable the mover in that shares configuration

[cui]

On 4/8/2024 at 7:01 AM, Swarles said:

Share override settings take higher priority than the global mover tuning settings.

 

If you disable mover tuning it will also disable the share override settings. 

 

Leave everything default in the global "scheduler settings" and the mover will behave has default. Then you can make changes in the individual shares. If you want a share to not use the mover at all, disable the mover in that shares configuration

    Thank you for your answer

[MILDEW]

    I am still having an issue with Mover not running with this plugin. I have done some testing with diagnostic logs. These tests are all in one consecutive run after reboot.

 

    My first test was after reinstalling plugin and rebooting. I invoked "Move Now button follows plug-in filter", applied, and attempted invoke Mover via button. After clicking, it would immediately return to its previous state. Seemingly without running Mover. I believe I attempted this 4 times. I then reverted "Move Now button follows plug-in filter" to its previous state and invoked mover via button. I then successfully invoked Mover via WebGUI button and pulled diagnostic logs.

 

towerone-diagnostics-20240414-1305 [test_one(fresh start-set_tune_to_buton_&_it_stopped_immediately-disabled_tune_filter_with_button_&_it_worked)].zip

 

    My following test started from previous state. I then invoked "mover stop" via terminal, invoked the "Log when not moving due to rules" option via Tuner, and repeated previous test by attempting to invoke mover via WebGUI button. Results were the same. I again pulled diagnostic logs.

 

towerone-diagnostics-20240414-1313 [test_two(same_as_previous_but_ran_mover_stop_then_turned_on_tuner_logging-still_same_outcome)].zip

 

      My final test began where the previous left off. I toggled "Only move at this threshold of used cache space"  from 0% to 5%, applied, changed it back, applied, then I invoked mover via the "Force move of all files on a schedule" option. This resulted in he Mover button greying out and Mover "running" seemingly without ever moving files. If let sit long enough it will have a random read with a much higher write (reads in double digit KB/s and writes much higher in the triple digit KB/s). I then pulled the diagnostic log. I have watched this same action previously for hours, and am again watching now. The only time it stops is with a Mover top command. The array_cache never gets smaller and the array never gets bigger, even if left to sit for days. I have done this test several times over the last week or so.  Toggling "Only move at this threshold of used cache space" had no relevance, but thought I would mention it as I happened to do this time that had the same outcome as not doing it.

 

towerone-diagnostics-20240414-1324 [test_three(again_same_as_before-toggled_5_&_0-started_with_cron-said_running_infinetly-no_movment_except_few_random_KB_with_higher_w_than_r)].zip

 

    If anyone can help me with this I would be grateful. I removed my previous comment, as this is likely better written and I previously didn't get any response.

Edited April 14 by MILDEW
Attached Logs

[Swarles]

On 4/15/2024 at 4:30 AM, MILDEW said:

    I am still having an issue with Mover not running with this plugin. I have done some testing with diagnostic logs. These tests are all in one consecutive run after reboot.

 

    My first test was after reinstalling plugin and rebooting. I invoked "Move Now button follows plug-in filter", applied, and attempted invoke Mover via button. After clicking, it would immediately return to its previous state. Seemingly without running Mover. I believe I attempted this 4 times. I then reverted "Move Now button follows plug-in filter" to its previous state and invoked mover via button. I then successfully invoked Mover via WebGUI button and pulled diagnostic logs.

 

towerone-diagnostics-20240414-1305 [test_one(fresh start-set_tune_to_buton_&_it_stopped_immediately-disabled_tune_filter_with_button_&_it_worked)].zip 357.4 kB · 2 downloads

 

    My following test started from previous state. I then invoked "mover stop" via terminal, invoked the "Log when not moving due to rules" option via Tuner, and repeated previous test by attempting to invoke mover via WebGUI button. Results were the same. I again pulled diagnostic logs.

 

towerone-diagnostics-20240414-1313 [test_two(same_as_previous_but_ran_mover_stop_then_turned_on_tuner_logging-still_same_outcome)].zip 351.24 kB · 0 downloads

 

      My final test began where the previous left off. I toggled "Only move at this threshold of used cache space"  from 0% to 5%, applied, changed it back, applied, then I invoked mover via the "Force move of all files on a schedule" option. This resulted in he Mover button greying out and Mover "running" seemingly without ever moving files. If let sit long enough it will have a random read with a much higher write (reads in double digit KB/s and writes much higher in the triple digit KB/s). I then pulled the diagnostic log. I have watched this same action previously for hours, and am again watching now. The only time it stops is with a Mover top command. The array_cache never gets smaller and the array never gets bigger, even if left to sit for days. I have done this test several times over the last week or so.  Toggling "Only move at this threshold of used cache space" had no relevance, but thought I would mention it as I happened to do this time that had the same outcome as not doing it.

 

towerone-diagnostics-20240414-1324 [test_three(again_same_as_before-toggled_5_&_0-started_with_cron-said_running_infinetly-no_movment_except_few_random_KB_with_higher_w_than_r)].zip 368.13 kB · 0 downloads

 

    If anyone can help me with this I would be grateful. I removed my previous comment, as this is likely better written and I previously didn't get any response.

Hey sorry for the delay on this, I would very much like to get you up and running. Unfortunately I will need some logs from Mover Tuning which don't get saved with the diagnostics zip. If you follow the steps here in #1 to get the logs that should allow me to figure out exactly what is going wrong

Don't worry about doing 3 runs, just the 1 should be fine.

[Sebby]

Hi,

 

I'm on unraid 6.12.10 and mover tuning released on 2023.12.19

 

I have over 600gb on my cache and my settings are

 

Even with forcing mover to run, it still doesn't move any of the files. Could this be a mover tuning issue or a hardlinks issue?

[Kilrah]

You've set to only move files that are more than 15 days old, that might be why.

Edited April 29 by Kilrah

[Sebby]

43 minutes ago, Kilrah said:

You've set to only move files that are more than 15 days old, that might be why.

Most of the files are over a month old. I understand the settings but its not moving files that are well over 15 days.

 

I was trying to run it nightly for files over 15 days old but they are just accumulating and not moving

Edited April 29 by Sebby

[Swarles]

On 4/29/2024 at 8:43 PM, Sebby said:

Most of the files are over a month old. I understand the settings but its not moving files that are well over 15 days.

 

I was trying to run it nightly for files over 15 days old but they are just accumulating and not moving

What are all your settings?

[Kees Fluitman]

Im guessing. since this plugin hasn't been updated the last few months, it is most likely starting to break?

Edited May 5 by Kees Fluitman

[Swarles]

On 5/6/2024 at 3:42 AM, Kees Fluitman said:

Im guessing. since this plugin hasn't been updated the last few months, it is most likely starting to break?

Unraid itself hasn't really had any big updates/changes recently that impact how the mover works. So it shouldn't be starting to break, but it does appear to be not working for some.

[Oxxy]

Like quite a few other people here, I am getting "No such file or directory' when running the mover. Its been updated, so I am stuck with unprotected data on the cache.

 

I've had to copy it manually somewhere else until this is sorted.

 

Running 6.12.1.

Edited May 15 by Oxxy

[Swarles]

1 hour ago, Oxxy said:

Like quite a few other people here, I am getting "No such file or directory' when running the mover. Its been updated, so I am stuck with unprotected data on the cache.

 

I've had to copy it manually somewhere else until this is sorted.

 

Running 6.12.1.

This may sound obvious but have you checked if the file or directory it is trying to move exists? It could be a number of things wrong. 

 

Is there any more info you can provide? See the FAQ on how to provide logs.

[Oxxy]

13 hours ago, Swarles said:

This may sound obvious but have you checked if the file or directory it is trying to move exists? It could be a number of things wrong. 

 

Is there any more info you can provide? See the FAQ on how to provide logs.

Thanks, I did get the diagnostics sorted, but could see if has a heap of personal information in it, so didnt really want to add them to my post just above.

 

EDIT - I have removed the HBA (9300-16i) and moved to an older one I had lying around. For now its working. I did attach the diagnostics though.

tower-diagnostics-20240516-0722.zip

Edited May 15 by Oxxy

[Swarles]

1 hour ago, Oxxy said:

Thanks, I did get the diagnostics sorted, but could see if has a heap of personal information in it, so didnt really want to add them to my post just above.

 

EDIT - I have removed the HBA (9300-16i) and moved to an older one I had lying around. For now its working. I did attach the diagnostics though.

tower-diagnostics-20240516-0722.zip 151.67 kB · 0 downloads

Oh okay it looks like the folders it can't find are the following:

/mnt/cache_ssd/Tech/pfSense/pfSense Backups/April 30

And

/mnt/cache_ssd/Tech/pfSense2/pfSense Backups/April 30

 

This is a peculiar error as it's happening on the share mover not the main mover, the share mover does minimal processing, it basically just sends the share path to the unraid default mover.

 

It seems like you figured out your problem was related to your HBA. Great work! I have no idea how you deduced that or why it would cause issues though. 

 

You can delete your diagnostics. If you want me to investigate or the issue hasn't gone let me know, I'll probably need you to confirm that that exact path does exist and if there is anything peculiar about its permissions or properties.

[Oxxy]

22 hours ago, Swarles said:

 

It seems like you figured out your problem was related to your HBA. Great work! I have no idea how you deduced that or why it would cause issues though. 

 

 

Appreciate the follow up. I found the original HBA issue by transferring a heap of small files and randomly it would slow to a crawl. Looking at logs shows a heap of mp3sas errors. Thankfully I had spare hardware available. Interesting that this same card (9300-16i) has caused issues for others.

 

The other mover issue, not sure on whats up with that. I noticed a while ago mover wasn't always working for shares that had a space character in the share name. But i did notice the little warning saying the share wasn't protected, when looking at the share on the command line, I could see the folders and they just wouldn't move at all, and attempts had the "No such file or directory" thing.

[Swarles]

1 hour ago, Oxxy said:

The other mover issue, not sure on whats up with that. I noticed a while ago mover wasn't always working for shares that had a space character in the share name. But i did notice the little warning saying the share wasn't protected, when looking at the share on the command line, I could see the folders and they just wouldn't move at all, and attempts had the "No such file or directory" thing.

I think usually the "Some or all files unprotected" with the orange icon is related to not having redundancy. So on the array that could be no parity, or a compromised parity. On the cache that could mean you only have one cache drive or just no redundancy. 

 

Spaces can cause lots of issues in bash but for the most part the plugin tries to encompass everything in quotes so that they don't cause issues. It's possible that there is something unaccounted for however. If you get the errors again just drop another comment here and we can try figure it out. It's annoying but there's so many things it could be that it's difficult to know for sure what might be causing it. 

[solidno8]

Is there a way to have mover work on a hidden share i.e. .tmp? The original script does honor the hidden share and move files off of the cache but I am unable to get the plugin mover to see it. I tried staging a .tmp.cfg file but that doesn't seem to work. However, I was to pass the share name to the share_mover script and that does work i.e share_mover '.tmp'

[jaimbo]

This plugin is currently broken on 6.13.0-beta.2 I think due to the mover binary being relocated:

 

<6.12: /usr/local/bin/move

>6.13: /usr/libexec/unraid/move

 

Just making sure this is on the radar to be fixed before release

[hugenbdd]

Thank You

WIll keep an eye on for when 6.13 is released and try to get an update out quickly when it is.

[StylishQuoter]

I may have run into a bug.

 

Running mover 2023.12.19 on unraid 6.12.10

 

```

Jun  3 03:00:02 root: mover: started
Jun  3 03:00:02 root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun  3 03:00:02 root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun  3 03:00:02 root: skip: /mnt/disk2/appdata/tautulli/tautulli.db-wal
Jun  3 03:00:02 root: skip: /mnt/disk2/appdata/tautulli/tautulli.db-shm
Jun  3 03:00:02 root: mover: finished

```

 

Thanks!

[Odon]

Hello colleagues,

 

First of all, thank you developers, without you this would not be possible.

 

There is some possibility that in a future version within the options "Script to run before move" and "Script to run after move" it will be possible to pass a script with parameters like /appdata/loquesea/mibonitoscript.sh --option "pepe".

 

This is very useful if you want to make the most of the function's options, otherwise you have to have a script with each option.

PS: maybe I've missed something and if you can pass a script with parameters.

 

Thank you,

 

Odon

[Sebby]

On 5/3/2024 at 8:16 PM, Swarles said:

What are all your settings?

Sorry I didn't see this.

 

This is what I was using when I was trying to work on this.

[malaki86]

Having an issue with the Mover Tuning plugin. This is in my main log.

 

Jun 14 20:00:01 Neo root: Starting Mover
Jun 14 20:00:01 Neo root: ionice -c 2 -n 0 nice -n 0 /usr/local/emhttp/plugins/ca.mover.tuning/age_mover start 0 0 0 '' '' '' '' '' '' '' '' 50
Jun 14 20:00:01 Neo root: Log Level: 1
Jun 14 20:00:01 Neo root: mover: started
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 372: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")
Jun 14 20:00:02 Neo root: skip: /mnt/disk1/appdata/radarr/radarr.db
Jun 14 20:00:02 Neo root: skip: /mnt/disk1/appdata/radarr/supervisord.log
Jun 14 20:00:03 Neo root: skip: /mnt/disk2/appdata/emby/data/activitylog.db
Jun 14 20:00:03 Neo root: skip: /mnt/disk2/appdata/emby/data/authentication.db
Jun 14 20:00:03 Neo root: skip: /mnt/disk2/appdata/emby/data/displaypreferences.db
Jun 14 20:00:03 Neo root: skip: /mnt/disk2/appdata/emby/data/library.db
Jun 14 20:00:03 Neo root: skip: /mnt/disk2/appdata/emby/data/users.db
Jun 14 20:00:03 Neo root: skip: /mnt/disk2/appdata/binhex-sabnzbd/admin/history1.db
Jun 14 20:00:03 Neo root: file: /mnt/disk2/appdata/binhex-qbittorrentvpn/qBittorrent/config/ipc-socket
Jun 14 20:00:03 Neo root: move_object: /mnt/disk2/appdata/binhex-qbittorrentvpn/qBittorrent/config/ipc-socket File exists
Jun 14 20:00:03 Neo root: file: /mnt/disk2/appdata/qbittorrent/qBittorrent/ipc-socket
Jun 14 20:00:03 Neo root: move_object: /mnt/disk2/appdata/qbittorrent/qBittorrent/ipc-socket File exists
Jun 14 20:00:03 Neo root: skip: /mnt/disk2/appdata/organizrv2/log/nginx/status_errors.log
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/appdata/organizrv2/log/nginx/access.log
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/appdata/organizrv2/log/nginx/error.log
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/appdata/organizrv2/log/php/error.log
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/appdata/bazarr/db/bazarr.db
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/appdata/HandBrake/xdg/cache/fontconfig/605eb33399efa08596766eba8c7361d5-le64.cache-8
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/appdata/HandBrake/xdg/cache/fontconfig/6ba42ae0000f58711b5caaf10d690066-le64.cache-8
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/appdata/HandBrake/log/nginx/error.log
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/appdata/HandBrake/log/nginx/access.log
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/sysLog/syslog-192.168.1.5.log
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/system/docker/docker.img
Jun 14 20:00:04 Neo root: skip: /mnt/disk2/system/libvirt/libvirt.img
Jun 14 20:00:04 Neo root: mover: finished