Hi all,

I made a new version of the plugin. This fork replaces @hugenbdd's plugin which is not maintained anymore and will land in community applications if your feedbacks are positive.

 

It worth the upgrade if you want:

- Automatic age

- Compatibility with 7.0.0 without playing with symlinks

- Minor fixes

Please see README / Changelog for more informations: R3yn4ld/ca.mover.tuning (github.com).

 

I use it hourly and it shall work correctly. I had not made extensive testing on 7.0.0 however (just a couple of days, seems OK). Please report any bug/problem.

 

Here is the plugin: ca.mover.tuning.plg

 

Please remove hugenbdd's one before installing (at the moment via Plugins/Install plugin/insert above URL).

Please note that you'll have to reconfigure mover tuning.

 

Advise: enable Test Mode and verify the Mover_Tuning_{date}T{time}list in the /tmp/Mover directory. This file contain the files that would be moved by original mover with Test Mode off.

If everything seems OK you can turn off Test mode.

 

@SimonF, would you please check my plugin script and tell me if it seems OK for you to publish on CA? I've worked on renaming and symlinking files and hope I did not break reverse compatibility (should be OK but I'd like a review please)

 

 

 

Edited July 29, 20241 yr by Reynald
Links and typo

Does force turbo writes when not enabled in unraid settings work with that?

Can you be more specific about this force turbo problem you encounter @Yock please?

 

I have not touched this part, yet.

Unraid 7 beta has a bug where the setting "Tunable (md_write_method)" does not apply until after a reboot of the server.

 

I THINK it's what stopping the mover plugin from working for me as i have that setting set to read/modify/write, in the plugin i want it set to force turbo write on (as in reconstructive write) and so the mover doesn't start for me even with this fix to the plugin.

Oh OK I see.

 

Is there another workaround than rebooting the server that you can me point to so I can see if something can be made by mover plugin please?

However, I cannot see how this bug can prevent mover plugin to work, as there is no error control on the Force Turbo Write piece of code

 

There is probably something else.

 

Are list created in /tmp/Mover ?

Is test mode enable ? (it shall be set to "no" if above lists are OK )

I don't think there is a workaround.

 

It's sadly not possible for me right now to get near to anything administrative stuff on my server atm as i'm not home and don't want to use a computer/network not owned by me for that.

But what i remember from logs is all it did was say something about starting the whole process, setting turbo writes on then next line was setting turbo writes off again and then nothing more.

Test mode was disabled.

 

Sorry i'm of practically no help but as said i'm not near the server atm.

On 6/30/2024 at 3:34 PM, wuudogg said:

Thinking I may need to rm /usr/local/sbin/move first ?

 

Edit confirmed.

 

Issued from terminal:

rm /usr/local/sbin/move

ln -s /usr/libexec/unraid/move /usr/local/sbin/move

 

Working great. Thank you!

Saved me from rolling back/pulling my hair out! Thanks! 😄

25 minutes ago, Yock said:

I don't think there is a workaround.

 

It's sadly not possible for me right now to get near to anything administrative stuff on my server atm as i'm not home and don't want to use a computer/network not owned by me for that.

But what i remember from logs is all it did was say something about starting the whole process, setting turbo writes on then next line was setting turbo writes off again and then nothing more.

Test mode was disabled.

 

Sorry i'm of practically no help but as said i'm not near the server atm.

 

 No problem Yock.

 

The behavior you describe is likely that you have filter preventing moving files. 

Between the two turbo writes line in the log you should see something like:

ionice $ioLevel nice -n $niceLevel $age_mover_str

or

ionice $ioLevel nice -n $niceLevel /usr/local/sbin/mover.old $options

 

Here is the behavior I have with 7.0.0-beta.1 and 2024-07-07 mover plugin:

 

(edit: it seems that force turbo write is working as we have mdcmd returns)

Edited July 8, 20241 yr by Reynald

The plugin was with default settings except for the force turbo writers.

 

Sounds good that it works, i will look into it again when i get back to the server. Thanks for the help!

Couldn't help my self so borrowed a secure computer and tested.

 

Jul 8 12:27:49 YockS emhttpd: shcmd (87): /usr/local/sbin/mover start &> /dev/null &

Jul 8 12:27:49 YockS kernel: mdcmd (36): set md_write_method 1

Jul 8 12:27:49 YockS kernel:

Jul 8 12:27:49 YockS kernel: mdcmd (37): set md_write_method 0

Jul 8 12:27:49 YockS kernel:

 

Is what i get.

1 minute ago, Yock said:

Couldn't help my self so borrowed a secure computer and tested.

...

Is what i get.

 

Can you please post screenshots of mover and mover tuning scheduler settings?

Just noticed on the picture my loggin is off.. Would probably help with that on... So...

 

Jul 8 12:55:56 YockS kernel: mdcmd (46): set md_write_method 1

Jul 8 12:55:56 YockS kernel:

Jul 8 12:55:56 YockS move: Usage: /usr/local/sbin/mover.old start [-e] <disk_name>

Jul 8 12:55:56 YockS move: /usr/local/sbin/mover.old stop|status

Jul 8 12:55:56 YockS move: <disk_name> must match pattern 'disk[0-9]*'

Jul 8 12:55:56 YockS kernel: mdcmd (47): set md_write_method 0

Jul 8 12:55:56 YockS kernel:

Please enable Mover logging.

 

What is the first parameter of Mover Tuning settings ? This one:

 

With your settings, Mover Tuning filter are applied only on scheduled (cron) run, that is on the 28th of the month.

You need to set this to also invoke Mover Tuning on "Move Now" button press:

1 minute ago, Yock said:

Just noticed on the picture my loggin is off.. Would probably help with that on... So...

 

Jul 8 12:55:56 YockS kernel: mdcmd (46): set md_write_method 1

Jul 8 12:55:56 YockS kernel:

Jul 8 12:55:56 YockS move: Usage: /usr/local/sbin/mover.old start [-e] <disk_name>

Jul 8 12:55:56 YockS move: /usr/local/sbin/mover.old stop|status

Jul 8 12:55:56 YockS move: <disk_name> must match pattern 'disk[0-9]*'

Jul 8 12:55:56 YockS kernel: mdcmd (47): set md_write_method 0

Jul 8 12:55:56 YockS kernel:

 

OK so there is a bug with the mover.old invocation for unraid 7.0.0+.
As a workaround, you can keep your default parameters and set this to Yes:

or remove this plugin as you do not use filters

Copying some files to cache now to test, going to take a couple of minutes.

Haha nearly killed my moms pets now when i yelled out in joy.

It is working now! But if i have schedule on i can't manually move and vice versa, no biggie though. As a switch between turbo write on and off is really what i wanted anyways.

 

I can't thank you enough for your help!

1 minute ago, Yock said:

It is working now! But if i have schedule on i can't manually move and vice versa, no biggie though.

 

I'll investigate this one after looking at this error:

move: Usage: /usr/local/sbin/mover.old start [-e] <disk_name>

 

10 hours ago, Reynald said:

 

  

 

Hi all,

I made a new version of the plugin. This fork replaces @hugenbdd's plugin which is not maintained anymore and will land in community applications if your feedbacks are positive.

 

It worth the upgrade if you want:

- Automatic age

- Compatibility with 7.0.0 without playing with symlinks

- Minor fixes

Please see README / Changelog for more informations: R3yn4ld/ca.mover.tuning (github.com).

 

I use it hourly and it shall work correctly. I had not made extensive testing on 7.0.0 however (just a couple of days, seems OK). Please report any bug/problem.

 

Here is the plugin: ca.mover.tuning.plg

 

Please remove hugenbdd's one before installing (at the moment via Plugins/Install plugin/insert above URL).

Please note that you'll have to reconfigure mover tuning.

 

Advise: enable Test Mode and verify the Mover_Tuning_{date}T{time}list in the /tmp/Mover directory. This file contain the files that would be moved by original mover with Test Mode off.

If everything seems OK you can turn off Test mode.

 

@SimonF, would you please check my plugin script and tell me if it seems OK for you to publish on CA? I've worked on renaming and symlinking files and hope I did not break reverse compatibility (should be OK but I'd like a review please)

 

 

 

Looks good to me. symlink is created correctly. You will need an xml and to speak to @Squidfor ca

Great, thank you @SimonF

Do you have any guidance to the doc for the XML please?

10 minutes ago, Reynald said:

Great, thank you @SimonF

Do you have any guidance to the doc for the XML please?

Here are my xmls for CA. 

 

https://github.com/SimonFair/unraid-plugins then if you let squid know the repo he will add to CA

1 hour ago, SimonF said:

Here are my xmls for CA. 

 

https://github.com/SimonFair/unraid-plugins then if you let squid know the repo he will add to CA

 

Thank you, repo created: https://github.com/R3yn4ld/unraid-plugins and PM sent to @Squid   (I lamely reuse original ca.mover.tuning XML)

Pushed new version:

https://github.com/R3yn4ld/ca.mover.tuning/raw/master/plugins/ca.mover.tuning.plg

 

2024-07-10

- Unraid 7.0.0 compatibility (/ 6.x bug?): original mover now works with "Move Now button follows plug-in filters" set to off. Bug discussed here: 

 

 

Edited July 10, 20241 yr by Reynald

I plan to add "Automatic age" to array->cache mover for shares with cache set to "Prefer" (= Array -> Cache).
I also plan to add some "Smart caching" option like I did in the following (deprecated) script: https://forums.unraid.net/topic/92126-smart-caching-script/

 

 

Both of these function will require a rewrite of plugin internal filelist functions, but I think it worth it.

 

My plan is to, under the hood:

1. build a single list of files that were created/modified since the last plugin run (first run will be quite long)
2. add (in a future version, tunable with option) seeding torrent, played series, and other "preventing to sleep" things
3. sort the list by age (as actually done by "Automatic Age" settings)
   Then with the list containing all needed information to do smart things:
4. emptying old files from cache to array
5. smartly fill the cache 

How do you all feel about this? Any advice/feature request?

 

2 hours ago, Reynald said:

I also plan to add some "Smart caching" option like I did in the following (deprecated) script: https://forums.unraid.net/topic/92126-smart-caching-script/

 

the "smart" TV Show Function looks promising

 

very nice ideas and smart additions ... thanks for your efforts on this project