Not sure if it's this plugin or mover in general or just a 6.12.x issues but unraid doesn't appear to be killing mover process properly when a shutdown is launched causing it to want to do a parity check. 

 

I reported this as a bug in 6.12 and was told to use 'mover stop' but surely this is built into unraids shutdown sequence!?

 

Am now on 6.12.2 and same thing's happening

3 hours ago, dopeytree said:

Not sure if it's this plugin or mover in general or just a 6.12.x issues but unraid doesn't appear to be killing mover process properly when a shutdown is launched causing it to want to do a parity check. 

 

I reported this as a bug in 6.12 and was told to use 'mover stop' but surely this is built into unraids shutdown sequence!?

 

Am now on 6.12.2 and same thing's happening

the binary mover changed locations in 6.12 from older releases. 

For me on a slightly older release.

# which mover

/usr/local/sbin/mover

 

Can you try the command     which mover

and post the results?  There is a stop option for mover but it may not hitting the right file.

On 7/3/2023 at 11:35 AM, Swarles said:

The behaviour would skip all files and subfolders of the path given.

@Swarles+1 this would be awesome.  Would resolve several related 'asks' 🙂 

the ignore hidden folders i.e named .folder doesn't seem to work?

 

3 hours ago, dopeytree said:

the ignore hidden folders i.e named .folder doesn't seem to work?

Hmm this is not expected behaviour and I have tested on my unraid to be working.

A couple things:

1. Make sure that if the particular share with ".folder" is using its own settings (mover settings override) that the setting to ignore hidden files is set to yes in that share. If it is not using its own settings the global settings must be set to yes.
2. Ensure that the type of share is valid (Primary: Cache, Mover: cache-->array), other share types do not use mover settings.
3. Ensure that you're not trying to move files by hitting the "Move ALL" button in the share settings.
4. Ensure that you do not have "Move All from Cache-Yes shares when disk is above a certain percentage:" set to Yes in your share/mover settings. If this successfully executes, it will move all the files off the cache not following settings.
5. Set "Test Mode:" and "Move Now button follows plug-in filters:" to yes, also enable logs. Then run the mover and in "cd /tmp/Mover" use "cat Custom_Mover_Tuning_*latest*.list" this will show the files that will move. None of the files in .folder should be here. IF they are, do the following: "cat Mover_Tuning_*latest*.log". You can post the logs in a spoiler here or send to me privately.
   Spoiler

   Alternatively if you look through your logs yourself; find "Skipfiletypes string: ...." (there should be 1 for each cache-->array share), that is (most of) the find string being used so you can then run that in your your terminal and it will output all the paths that the mover will attempt to move for that share. It should not include paths for .folder or its contents.

    

On 7/6/2023 at 4:58 PM, unr41dus3r said:

Can i provide any info for you to check this problem?

I did missconfigured my smtp settings and the mail failed, so the mover/cron is using the smtp settings.

A couple things that could be helpful:

1. If you could check if you have mover logging "Enabled", if it is, try disabling and see if you still get the emails (this is not a fix obviously but would be useful info). Alternatively, if it is disabled, try enabling it and see if you get a lot more logs in the email.
2. Do you get the email when you manually run the mover? Before you do this, know that if you do not have "Move Now button follows plug-in filters:" set to Yes, it will ignore mover tuning settings. Set this to Yes first if it'll mess things up for you.
3. Enable "Test Mode:" to Yes and then manually run the mover, do you get the email?

On 7/6/2023 at 4:58 PM, unr41dus3r said:

I did already an uninstall of the plugin, is this deleting all related information to the plugin?

Yeah if you uninstall and also do a restart all relevant plugin files should be gone, you can check this by checking if "/boot/config/plugins/ca.mover.tuning" exists. 

 

In your first message you state:

On 6/24/2023 at 4:12 AM, unr41dus3r said:

Edit:
This happens since upgrade to 6.12 RC Releases

Is this email happening for any other plugins or cron entries in "/etc/cron.d/root"?

I'm inclined to believe this is something triggered outside of the Mover Tuning Plugin because the code is not written to do this but I'll still try to figure it out.

 

EDIT: From what I have found in other posts, people who tend to get these kinds of "cron for user root" emails, tend to be related to errors or issues. It's some kind of unraid feature. The logs you are getting, are regular "echo" logs that appear in syslog. Maybe it's related to the way the script is executed or something else I haven't figured out yet.

Edited July 8, 20233 yr by Swarles

Could use some advice:  

I run two 2tb m.2's in raid0 for my cache.  I would like to keep files on the cache drive for as long as possible before the mover takes over and puts them on the array.  Seems to work fine, it will fill up to around 80ish % then all of a sudden, bam....moves everything to the array.  

This has been happening for a while, and its really hard to diagnose, because it take a bit of time to fill up the cache

Here's my mover settings

 

Here's my tuning settings:

 

Is it something i have setup wrong?  Ultimately i'd like to have it keep 60 days worth of media on the cache, then move it to the array.  I'm a little confused on what to have "Only move at this threshold of used cache space" set to if i only want the 60 day old media kept

35 minutes ago, danimal86 said:

Could use some advice:  

I run two 2tb m.2's in raid0 for my cache.  I would like to keep files on the cache drive for as long as possible before the mover takes over and puts them on the array.  Seems to work fine, it will fill up to around 80ish % then all of a sudden, bam....moves everything to the array.  

This has been happening for a while, and its really hard to diagnose, because it take a bit of time to fill up the cache

Here's my mover settings

Here's my tuning settings:

 

Is it something i have setup wrong?  Ultimately i'd like to have it keep 60 days worth of media on the cache, then move it to the array.  I'm a little confused on what to have "Only move at this threshold of used cache space" set to if i only want the 60 day old media kept

Ever thing will move because you have "Move All From Cache-Yes shares when disk is above a certain percentage:" set to "Yes" and the threshold set to 85%.  You can set this to "No" however, there is a possibility you could use 100% of cache.  But, if you monitor it and get your "days old" setting just right, you can keep 60 days or close to 60 days with no issues depending on how much you download each day.  (i.e. You are trying to match the amount that gets moved off each night with about the same amount you download each day)

14 minutes ago, hugenbdd said:

Ever thing will move because you have "Move All From Cache-Yes shares when disk is above a certain percentage:" set to "Yes" and the threshold set to 85%.  You can set this to "No" however, there is a possibility you could use 100% of cache.  But, if you monitor it and get your "days old" setting just right, you can keep 60 days or close to 60 days with no issues depending on how much you download each day.  (i.e. You are trying to match the amount that gets moved off each night with about the same amount you download each day)

Thank you!  Cant believe i didn't put 2 & 2 together with that setting.  Seems like it could be helpful to have that more towards the top with the other settings.  

 

Thats really what i want to do, kinda fine tune the days to keep on cache so it keeps cache around 80% full, ( seems like i ingest 1tb/month, so that would leave some extra room in there) and it will keep drives from spinning up and have fast play times.  

On 7/8/2023 at 7:54 AM, Swarles said:

A couple things that could be helpful:

1. If you could check if you have mover logging "Enabled", if it is, try disabling and see if you still get the emails (this is not a fix obviously but would be useful info). Alternatively, if it is disabled, try enabling it and see if you get a lot more logs in the email.
2. Do you get the email when you manually run the mover? Before you do this, know that if you do not have "Move Now button follows plug-in filters:" set to Yes, it will ignore mover tuning settings. Set this to Yes first if it'll mess things up for you.
3. Enable "Test Mode:" to Yes and then manually run the mover, do you get the email?

Yeah if you uninstall and also do a restart all relevant plugin files should be gone, you can check this by checking if "/boot/config/plugins/ca.mover.tuning" exists. 

 

In your first message you state:

Is this email happening for any other plugins or cron entries in "/etc/cron.d/root"?

I'm inclined to believe this is something triggered outside of the Mover Tuning Plugin because the code is not written to do this but I'll still try to figure it out.

 

EDIT: From what I have found in other posts, people who tend to get these kinds of "cron for user root" emails, tend to be related to errors or issues. It's some kind of unraid feature. The logs you are getting, are regular "echo" logs that appear in syslog. Maybe it's related to the way the script is executed or something else I haven't figured out yet.

 

Thanks for the response. Tested your steps

 

1.) Logging was disabled. I enabled the setting and tested a scheduled force run and recieved the same error (same log in syslog)

2.) I still dont recieve the email when i manually start the mover process and enable "Move Now button follows plug-in filters:"

3.) No difference if i enable Test Mode or disable, Log looks the same

4.) Did a uninstall, sadly didnt help. (uninstall, reboot, install)

5.) No other plugin or log message looks like the message from the mover schedule.

 

So sadly no solution, but still interesting, that also the "Move Now button follows plug-in filters:" does not create the error. Only if the task runs with the cron.d

 

About my comment, that i recieve the message since 6.12 RC. What also is possible that i recieve the message since i use a ZFS cache pool, but cant say this for sure.

Is it possible that it happens because of ZFS and something like "zfs destroy/create/..."

 

Edit:
I also recieve the error if i disable all alerts in Unraid. Really strange

 

 

Edit 2:
I am no expert with Linux CLI etc. and maybe it nothing but one thing is confusing me.

mover.old in the /etc/cron.d/root is the only command which uses the parameter "2>/dev/null" to send the log into the black hole.

 

All other commands in my root file are using "&> /dev/null" or "> /dev/null 2>&1" or > "/dev/null 2>&1", which are sending no mails.

 

Maybe its nothing but still one of the little things.

Edited July 11, 20233 yr by unr41dus3r

Can you modified the Plugin for Localization? 

 

54 minutes ago, Revan335 said:

Can you modified the Plugin for Localization? 

 

doesn't really tell me how to add for plug-in's.  If you can post a github link of another plug-in that has translations, I will take a look.

Can you post a example? @kennymc.c

 

This have German Language https://github.com/itimpi/parity.check.tuning

14 hours ago, unr41dus3r said:

Edit 2:
I am no expert with Linux CLI etc. and maybe it nothing but one thing is confusing me.

mover.old in the /etc/cron.d/root is the only command which uses the parameter "2>/dev/null" to send the log into the black hole.

 

All other commands in my root file are using "&> /dev/null" or "> /dev/null 2>&1" or > "/dev/null 2>&1", which are sending no mails.

 

Maybe its nothing but still one of the little things.

I'm no expert either lol so you're doing good! Interesting results!
I actually think your 2nd edit is on to something. I completely overlooked that your question was in response to the "Force move of all files on a schedule" setting and was assuming you were getting the emails from the normal mover schedule. I was testing the wrong things so my apologies.

 

Use the following to change the "2" to an "&". So "2>/dev/null" to "&>/dev/null".

nano /etc/cron.d/root

Then ctrl+O (the letter) to save and ctrl+X to exit.
Let me know if this fixes your issues  

On 7/3/2023 at 10:16 PM, dopeytree said:

Is there a way I can tell mover tuning to avoid moving files in a folder.

 

i.e do not move nzbget intermediate downloads folder 'data/usenet/intermediate'

Hey so when I initially responded I think I may have been wrong and you might be able to achieve this by just putting the path to the folder in the skip files list (in the file). Just make sure that if you are wanting to use the path "/mnt/cache/data/usenet/intermediate" there are no other paths that contain this same path as part of it. e.g. "/mnt/cache/data/usenet/intermediate1234".

I noticed my error in the original response because my implementation of skip folders wasn't behaving as expected and turns out it was because the folders were being skipped anyway due to the original code. Not sure why I thought it didn't work when I first tested it. Regardless will still be working on a more robust solution.

Edited July 12, 20233 yr by Swarles

5 hours ago, Swarles said:

I'm no expert either lol so you're doing good! Interesting results!
I actually think your 2nd edit is on to something. I completely overlooked that your question was in response to the "Force move of all files on a schedule" setting and was assuming you were getting the emails from the normal mover schedule. I was testing the wrong things so my apologies.

 

Use the following to change the "2" to an "&". So "2>/dev/null" to "&>/dev/null".

nano /etc/cron.d/root

Then ctrl+O (the letter) to save and ctrl+X to exit.
Let me know if this fixes your issues  

 

Thanks

 

I think it isnt working to edit the file normally. Is it possible to reload the root file? When i chance the time for example it doesnt will be executed.

When i use update_cron command to reload cron (i found in the forums) the command will be resettet to 2>

 

Edit:

But i found something else.

When i use "/usr/local/sbin/mover.old start 2>/dev/null" in the CLI the information about

mover: started
mover: finished

is written in the CLI

 

When i use "/usr/local/sbin/mover.old start &> /dev/null" there is no output in the CLI

Edited July 12, 20233 yr by unr41dus3r

2 hours ago, unr41dus3r said:

When i use "/usr/local/sbin/mover.old start &> /dev/null" there is no output in the CLI

Yeah this is because the &> is sending both stderr and stdout to /dev/null while "2>" only sends stderr to the void. 
 

2 hours ago, unr41dus3r said:

I think it isnt working to edit the file normally. Is it possible to reload the root file? When i chance the time for example it doesnt will be executed.

I think you could achieve this by making the edit in the updateCron.php file.

nano /usr/local/emhttp/plugins/ca.mover.tuning/updateCron.php

This change will be reverted when your server restarts and the plugin installs.

 

Unfortunately I can't test these commands because I don't have a test rig and this will forgo my current mover settings.
Unless it has something to do with the way mover.old is being called in the cron schedule (2> /dev/null), I believe this behaviour is not because of the Mover Tuning Plugin. I don't think that "mover.old" is a Mover Tuning file.

Some more questions:

1. When you ran "/usr/local/sbin/mover.old start 2> /dev/null" in CLI did you you get the email?
2. If you uninstall the Mover Tuning Plugin. What mover cron schedule do you have in "cat /etc/cron.d/root"?

Edited July 12, 20233 yr by Swarles

1 hour ago, Swarles said:

Yeah this is because the &> is sending both stderr and stdout to /dev/null while "2>" only sends stderr to the void. 
 

I think you could achieve this by making the edit in the updateCron.php file.

nano /usr/local/emhttp/plugins/ca.mover.tuning/updateCron.php

This change will be reverted when your server restarts and the plugin installs.

 

Unfortunately I can't test these commands because I don't have a test rig and this will forgo my current mover settings.
Unless it has something to do with the way mover.old is being called in the cron schedule (2> /dev/null), I believe this behaviour is not because of the Mover Tuning Plugin. I don't think that "mover.old" is a Mover Tuning file.

Some more questions:

1. When you ran "/usr/local/sbin/mover.old start 2> /dev/null" in CLI did you you get the email?
2. If you uninstall the Mover Tuning Plugin. What mover cron schedule do you have in "cat /etc/cron.d/root"?

 

Its fixed ! As i see in my short testing. (will see definitly tonight)

 

I did edit the "/usr/local/emhttp/plugins/ca.mover.tuning/updateCron.php" file and replaced the logging part:

".../usr/local/sbin/mover.old start &> /dev/null\n\n ..."

 

Then i set a new time in the advanced mover schedule setting and the mover run without sending an email!

 

About your questions

1. it never send an email (didnt matter if i used 2 or &)

2. didnt test now it because &> fixed it (as i remember the mover.old cron line was removed and only the original mover schedule was there)

 

This are the 2 mover lines in my cron file now:
 

# Generated mover schedule:
0 */6 * * * /usr/local/sbin/mover |& logger

# Generated schedule for forced move
0 */24 * * * /usr/local/sbin/mover.old start &> /dev/null

 

Is mover.old correct in my system?

Edited July 12, 20233 yr by unr41dus3r

Great! Do let us know if the email ends up coming through again or not. Otherwise I'll ask hugendbb if we can change it permanently. Also keep in mind it will be reset if your server restarts (until it is implemented) 

 

47 minutes ago, unr41dus3r said:

Is mover.old correct in my system?

Not entirely sure what you mean but the location in sbin is correct and the file SHOULD exist (if that's what you meant).

Edited July 12, 20233 yr by Swarles

19 hours ago, Revan335 said:

Can you post a example? @kennymc.c

 

This have German Language https://github.com/itimpi/parity.check.tuning

 

21 hours ago, hugenbdd said:

doesn't really tell me how to add for plug-in's.  If you can post a github link of another plug-in that has translations, I will take a look.

 

I am not a plugin developer but there is a GitHub Repo for every language package which are all based on this one: https://github.com/unraid/lang-en_US

Basically you just have to separate or duplicate all language specific content into one or more text files, which correspond to the format mentioned there.

If I look at https://github.com/itimpi/parity.check.tuning/blob/439e7f7aff64beaec4cc8449a74b431579ca34fa/source/usr/local/emhttp/plugins/parity.check.tuning/Parity.check.tuning.page from the parity check tuning plugin probably every text string found in the translated text file will be replaced with the one from the currently active translation package. But this is something you would probably have to ask other developers.

The developer requirements for multi-language support are covered in this section of the online documentation accessible via the ‘Manual’ link at the bottom of the GUI or the DOCS link at the top of each forum page.

1 hour ago, itimpi said:

The developer requirements for multi-language support are covered in this section of the online documentation accessible via the ‘Manual’ link at the bottom of the GUI or the DOCS link at the top of each forum page.

Is this link available?
It is necessary for plugin developers to take appropriate action within their plugins to enable them to utilize the multi-language support. The details on what is required are covered in the design guide here.  "Here" <- https://wiki.unraid.net/images/5/5e/Multi-language_Support_Design_Guide.pdf   Doesn't seem to work.

27 minutes ago, hugenbdd said:

Is this link available?
It is necessary for plugin developers to take appropriate action within their plugins to enable them to utilize the multi-language support. The details on what is required are covered in the design guide here.  "Here" <- https://wiki.unraid.net/images/5/5e/Multi-language_Support_Design_Guide.pdf   Doesn't seem to work.

Best to look at other plugins.

 

You need the translation parts.https://github.com/SimonFair/USB_Manager/blob/85cb99a5aca0b633184fec3d557587aec36802c6/source/USBDevices.php#L14

 

Value to be translated in php are between _()_ and _() is javascript

 

echo "<td>"._('Volume(Storage)    ')."</td><td>"._('Mappin

14 hours ago, Swarles said:

Great! Do let us know if the email ends up coming through again or not. Otherwise I'll ask hugendbb if we can change it permanently. Also keep in mind it will be reset if your server restarts (until it is implemented) 

 

Not entirely sure what you mean but the location in sbin is correct and the file SHOULD exist (if that's what you meant).

 

It is working fine! No email tonight and the mover force task ran.

This means the change from "2>/dev/null\" to "&> /dev/null\" fixed my problem with the emails!

Hi, still on 6.11.5

 

I keep randomly getting a mover error with the plugin every now and then:

 

cat: /boot/config/plugins/ca.mover.tuning/ca.mover.tuning.cfg: No such file or directory
Jul 13 08:00:01 redacted root: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 675: [: 87: unary operator expected

 

I have not changed anything in the config in over a month and it worked before.

 

Does anyone have a clue what the cause could be?