[Plugin] Parity Check Tuning

itimpi · July 12, 2023

On 7/11/2023 at 10:16 AM, dopeytree said:

What's the best way to look at a file in the ram system i.e /etc/cron.d/root

I'm still working out how unraids terminal commands work compared to normal linux.

If you use the option to open a console session from the Unraid GUI then you run commands just like on any other Linux system. Because Unraid is a cut-down Slackware based system then the command available may be limited to the more common (or Unraid specific) ones. The NerdTools plugin can be used to install additional commands.

On 7/11/2023 at 10:16 AM, dopeytree said:

The system error is this thats causing the conflict:

Jul 11 10:06:30 Moulin-rouge Parity Check Tuning: ERROR: marker file found for both scheduled and automatic

That is an internal consistency check for a state condition that I do not expect to happen, but it should (in most cases) let things run without a problem. Was this error with the latest version of the plugin (2023.07.08) as if so I might give you instructions on how you could gather additional diagnostics to help identify the cause so I can correct for it.

dopeytree · July 13, 2023

moulin-rouge-diagnostics-20230713-0426.zip

GMAsterAU · July 21, 2023

On 7/5/2023 at 9:20 AM, itimpi said:

That seems a valid Use Case so I may go ahead and add the option to close down the server when an array operation finished. Of course I may then find it is not as easy to implement as I currently think is the case

I just noticed that on Version: 2023.07.08 an option for 'Shutdown server when array operation completes:' exists, however it is greyed out. Is there a condition that has to be given to enable it?

Thank you for all the great work with this plugin

itimpi · July 21, 2023

7 hours ago, GMAsterAU said:

I just noticed that on Version: 2023.07.08 an option for 'Shutdown server when array operation completes:' exists, however it is greyed out. Is there a condition that has to be given to enable it?

If you read the change log you will see that it mentions this option is added to the GUI, but is currently disabled due the supporting code not being ready. The next update will allow this option to be selected.

GMAsterAU · July 21, 2023

9 hours ago, itimpi said:

If you read the change log you will see that it mentions this option is added to the GUI, but is currently disabled due the supporting code not being ready. The next update will allow this option to be selected.

Thank you Dave, I didn't think to look.

I was working on a solution to essentially implement a similar function using user scripts. Maybe you find some of the code useful in implementing the logic:

#!/bin/bash

#VARS=$(awk -F= '/\[/{prefix=$0; next} $1{$1=$1; print prefix $0}' OFS='=' "/var/local/emhttp/var.ini")
check_interval=300
parity="off"
counter=0
Script="/boot/config/plugins/user.scripts/scripts/PLACEHOLDER/script"

####check if parity check is running
paritycheck(){
VARS=$(awk -F= '/\[/{prefix=$0; next} $1{$1=$1; print prefix $0}' OFS='=' "/var/local/emhttp/var.ini")
test=$(grep -wn "mdResyncPos" <<< "$VARS")
test=$(grep -oP '"\K[^"\047]+(?=["\047])' <<< "$test")
#echo "$test"
if [ $test == 0 ] && [ "$counter" == 0 ];
then
  echo "Parity Check / Sync / Rebuild not running."
  echo "Stopping here"
  exit 0
elif [ $test == 0 ] && [ "$counter" != 0 ];
then
  parity="off"
  runscript
elif [ $test != 0 ];
then
   if [ "$counter" == 0 ];
   then
     echo "Parity Check / Sync / Rebuild in progress."
     parity="on"
   else
   :
   fi
fi
}

####continue to recheck if parity is running
paritycheckstatus () {
while [ "$parity" == "on" ]
	do
	
  		echo "Parity operation still running"
  		echo "Waiting $check_interval seconds to check again"
                ((counter=counter+1))
                echo "$counter"
  		sleep $check_interval
                paritycheck            
done
}

####run script when parity check has completed
runscript() {
bash "$Script"
}

##execute
arraycheck
paritycheck
paritycheckstatus

My Bash is not the cleanest but it did do the job technically; however I had issues with executing the whole thing reliably from the main server to start it, execute the parity check and then shut down again.

itimpi · July 21, 2023

25 minutes ago, GMAsterAU said:

Thank you Dave, I didn't think to look.

I was working on a solution to essentially implement a similar function using user scripts. Maybe you find some of the code useful in implementing the logic:

#!/bin/bash

#VARS=$(awk -F= '/\[/{prefix=$0; next} $1{$1=$1; print prefix $0}' OFS='=' "/var/local/emhttp/var.ini")
check_interval=300
parity="off"
counter=0
Script="/boot/config/plugins/user.scripts/scripts/PLACEHOLDER/script"

####check if parity check is running
paritycheck(){
VARS=$(awk -F= '/\[/{prefix=$0; next} $1{$1=$1; print prefix $0}' OFS='=' "/var/local/emhttp/var.ini")
test=$(grep -wn "mdResyncPos" <<< "$VARS")
test=$(grep -oP '"\K[^"\047]+(?=["\047])' <<< "$test")
#echo "$test"
if [ $test == 0 ] && [ "$counter" == 0 ];
then
  echo "Parity Check / Sync / Rebuild not running."
  echo "Stopping here"
  exit 0
elif [ $test == 0 ] && [ "$counter" != 0 ];
then
  parity="off"
  runscript
elif [ $test != 0 ];
then
   if [ "$counter" == 0 ];
   then
     echo "Parity Check / Sync / Rebuild in progress."
     parity="on"
   else
   :
   fi
fi
}

####continue to recheck if parity is running
paritycheckstatus () {
while [ "$parity" == "on" ]
	do
	
  		echo "Parity operation still running"
  		echo "Waiting $check_interval seconds to check again"
                ((counter=counter+1))
                echo "$counter"
  		sleep $check_interval
                paritycheck            
done
}

####run script when parity check has completed
runscript() {
bash "$Script"
}

##execute
arraycheck
paritycheck
paritycheckstatus

My Bash is not the cleanest but it did do the job technically; however I had issues with executing the whole thing reliably from the main server to start it, execute the parity check and then shut down again.

I already have the code implemented and am currently testing that it works as expected. It was not a lot of code as most of the building block needed were already included, just not being used to trigger the shutdown at the appropriate point.

I have included a safety check/escape in that you will get a notification that a shutdown is pending near the start of the check, but as long as you unset the option before the array operation finishes then the shutdown is aborted.

itimpi · July 22, 2023

2023-07-21 update

Fix: Some internal changes to try and minimise chance of race conditions
New: Option to shutdown server after array operation completes now ready for use

Johann · August 1, 2023

I'm getting a lot of errors in my syslog with the following, is this normal?

Thank you!

itimpi · August 1, 2023

1 hour ago, Johann said:

I'm getting a lot of errors in my syslog with the following, is this normal?

Thank you!

This is an internal check in the plugin for a state I did not think would occur. Things should still work OK.

Were you doing anything unusual in terms of array operations leading up to this? The diagnostics might give me a clue.

if you know how to recreate it then I would appreciate it if was possible to get a log with the plugin’s Testing logging mode active so I can see what triggered it.

Sacred · August 2, 2023

It would be very cool if there was another function to trigger the spindown at the start of the break. 😊

itimpi · August 2, 2023

4 hours ago, Sacred said:

It would be very cool if there was another function to trigger the spindown at the start of the break. 😊

Do not see much point in this. Just set the spindown delay to be a short delay at the Unraid level.

Sacred · August 2, 2023

2 hours ago, itimpi said:

Do not see much point in this. Just set the spindown delay to be a short delay at the Unraid level.

Again, this depends on the different uses of the users. Shortening spindown is not a solution - because that would lead to the disks being stopped too often in everyday work. 😑

At the same time, the disks don't have to run forever after the parity pause until someone needs data from the NAS. At the same time, the NAS should already be quiet at the beginning of the day - i.e. the patches should remain in spindown.

Everyone surely has his own preferences how he would like to use this. I'm not saying that the wish should have a high priority on the todo list. But it should not be such a big effort to set a trigger point for Spindown. 😊

estrim · August 21, 2023

QQ -

It appears as soon as I have a process that contains the words 'mover' my parity-check pauses (as per my settings) - anyone seen that?

E.g:

Aug 22 00:24:24 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 22 00:24:24 unraid01 Parity Check Tuning: Send notification: mover running:  
Aug 22 00:24:24 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 22 00:24:24 unraid01 Parity Check Tuning: DEBUG:   detected that mdcmd had been called from sh with command mdcmd nocheck PAUSE 
Aug 22 00:24:29 unraid01 Parity Check Tuning: DEBUG:   ...Pause Automatic Correcting Parity-Check
Aug 22 00:24:29 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for scheduled pause and resume
Aug 22 00:24:29 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for 6 minute interval monitoring
Aug 22 00:24:30 unraid01 Parity Check Tuning: DEBUG:   Updated cron settings are in /boot/config/plugins/parity.check.tuning/parity.check.tuning.cron
Aug 22 00:24:30 unraid01 kernel: mdcmd (48): nocheck PAUSE
Aug 22 00:24:30 unraid01 kernel: 
Aug 22 00:24:30 unraid01 kernel: md: recovery thread: exit status: -4
Aug 22 00:24:30 unraid01 Parity Check Tuning: Paused: Automatic Correcting Parity-Check (1.9% completed)

When:

root@unraid01:/mnt/user/my_repo# ps -ef | grep move
root     11163 10958  0 00:22 ?        00:00:00 tee -a /mnt/user/appdata/mystuff/log/unraid01-doingmoverXfer.log

I tried to figure out in the php how it 'greps' for the mover running but not very comfortable with it.

Thanks

itimpi · August 22, 2023

11 hours ago, estrim said:

QQ -

It appears as soon as I have a process that contains the words 'mover' my parity-check pauses (as per my settings) - anyone seen that?

E.g:

Aug 22 00:24:24 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 22 00:24:24 unraid01 Parity Check Tuning: Send notification: mover running:  
Aug 22 00:24:24 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 22 00:24:24 unraid01 Parity Check Tuning: DEBUG:   detected that mdcmd had been called from sh with command mdcmd nocheck PAUSE 
Aug 22 00:24:29 unraid01 Parity Check Tuning: DEBUG:   ...Pause Automatic Correcting Parity-Check
Aug 22 00:24:29 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for scheduled pause and resume
Aug 22 00:24:29 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for 6 minute interval monitoring
Aug 22 00:24:30 unraid01 Parity Check Tuning: DEBUG:   Updated cron settings are in /boot/config/plugins/parity.check.tuning/parity.check.tuning.cron
Aug 22 00:24:30 unraid01 kernel: mdcmd (48): nocheck PAUSE
Aug 22 00:24:30 unraid01 kernel: 
Aug 22 00:24:30 unraid01 kernel: md: recovery thread: exit status: -4
Aug 22 00:24:30 unraid01 Parity Check Tuning: Paused: Automatic Correcting Parity-Check (1.9% completed)

When:

root@unraid01:/mnt/user/my_repo# ps -ef | grep move
root     11163 10958  0 00:22 ?        00:00:00 tee -a /mnt/user/appdata/mystuff/log/unraid01-doingmoverXfer.log

I tried to figure out in the php how it 'greps' for the mover running but not very comfortable with it.

Thanks

At the moment the test is very simple as it looks for any process containing 'mover' using the statement:

	exec ("ps -ef | grep mover", $output);
	$ret = (count($output)> 2) ? 1 : 0;

and then looks at number of entries in $output. Perhaps need to change that test to make it more restrictive or change it in some other way?

Just to check - is there a good reason your code does NOT want the parity check to be paused while your process is running?

estrim · August 22, 2023

11 hours ago, itimpi said:
At the moment the test is very simple as it looks for any process containing 'mover' using the statement:
	exec ("ps -ef | grep mover", $output);
	$ret = (count($output)> 2) ? 1 : 0;
and then looks at number of entries in $output. Perhaps need to change that test to make it more restrictive or change it in some other way?

Thanks for having a look and fishing that out!

Looks like looking for /usr/local/sbin/mover is probably a better move (but a pain if they ever move the binary).

11 hours ago, itimpi said:

Just to check - is there a good reason your code does NOT want the parity check to be paused while your process is running?

Since in my case it is a log file which is always open and in the process table - yeah, however I simply changed the name to work around it for the time being.

Many thanks!

itimpi · August 23, 2023

9 hours ago, estrim said:

Looks like looking for /usr/local/sbin/mover is probably a better move (but a pain if they ever move the binary).

I'll look at changing it to check for 'bin/mover' which is probably generic enough that it will be found whatever the location of the binary and should not fall foul of a Use Case like yours.

estrim · August 23, 2023

On 6/14/2023 at 12:14 PM, itimpi said:

That is expected behaviour as the moment the plugin detects a Manual Check has been started and you have set the option to run manual checks in increments it will automatically pause the check if it is outside the increment period (ready to restart it when the next increment starts).

Run into something odd myself that is kind of like the above - I have no increment windows set, my parity is checking due to a unclean shutdown and I see this:

# grep -E 'Parity|md:' /var/log/syslog
...

Aug 23 00:48:38 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 00:54:23 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 01:00:32 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 01:00:32 unraid01 Parity Check Tuning: Send notification: mover running:  
Aug 23 01:00:33 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 01:00:33 unraid01 Parity Check Tuning: DEBUG:   detected that mdcmd had been called from sh with command m
dcmd nocheck PAUSE 
Aug 23 01:00:38 unraid01 Parity Check Tuning: DEBUG:   ...Pause Automatic Correcting Parity-Check
Aug 23 01:00:38 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for 6 minute interval monitoring
Aug 23 01:00:38 unraid01 Parity Check Tuning: DEBUG:   Updated cron settings are in /boot/config/plugins/parity.c
heck.tuning/parity.check.tuning.cron
Aug 23 01:00:38 unraid01 kernel: md: recovery thread: exit status: -4
Aug 23 01:00:38 unraid01 Parity Check Tuning: Paused: Automatic Correcting Parity-Check (2.8% completed)
Aug 23 01:06:35 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 01:06:35 unraid01 Parity Check Tuning: Send notification: mover no longer running:  
Aug 23 01:06:36 unraid01 Parity Check Tuning: Array operation not resumed - outside increment window: Automatic Correcting Parity-Check (2.8% completed)
Aug 23 01:12:37 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 01:18:29 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 01:24:35 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 01:30:33 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
...

Config -

# cat /boot/config/plugins/parity.check.tuning/parity.check.tuning.cfg 
parityTuningScheduled="0"
parityTuningManual="0"
parityTuningAutomatic="0"
parityTuningFrequency="0"
parityTuningResumeCustom="5,15,25,35,45,55 * * * *"
parityTuningResumeDay="0"
parityTuningResumeHour="0"
parityTuningResumeMinute="15"
parityTuningPauseCustom="0,10,20,30,40,50 * * * *"
parityTuningPauseDay="0"
parityTuningPauseHour="0"
parityTuningPauseMinute="10"
parityTuningNotify="0"
parityTuningRecon="0"
parityTuningClear="0"
parityTuningRestart="1"
parityTuningMover="1"
parityTuningCABackup="1"
parityTuningHeat="0"
parityTuningHeatHigh="3"
parityTuningHeatLow="8"
parityTuningHeatNotify="1"
parityTuningHeatShutdown="0"
parityTuningHeatCritical="2"
parityTuningHeatTooLong="30"
parityTuningLogging="1"
parityTuningLogTarget="0"
parityTuningMonitorDefault="17"
parityTuningMonitorHeat="7"
parityTuningMonitorBusy="6"
parityTuningMonitorPartial="1"
parityProblemType="sector"
parityProblemStartSector="0"
parityProblemStartPercent="0"
parityProblemEndSector="100"
parityProblemEndPercent="0"
parityProblemCorrect="0"
parityTuningBackup="1"
parityTuningShutdown="0"

I am therefore wondering why I'm getting "Aug 23 01:06:36 unraid01 Parity Check Tuning: Array operation not resumed - outside increment window: Automatic Correcting Parity-Check".

Tell me I'm doing something silly please

itimpi · August 23, 2023

6 minutes ago, estrim said:

Run into something odd myself that is kind of like the above - I have no increment windows set, my parity is checking due to a unclean shutdown and I see this:

# grep -E 'Parity|md:' /var/log/syslog
...

Aug 23 00:48:38 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 00:54:23 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 01:00:32 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 01:00:32 unraid01 Parity Check Tuning: Send notification: mover running:  
Aug 23 01:00:33 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 01:00:33 unraid01 Parity Check Tuning: DEBUG:   detected that mdcmd had been called from sh with command m
dcmd nocheck PAUSE 
Aug 23 01:00:38 unraid01 Parity Check Tuning: DEBUG:   ...Pause Automatic Correcting Parity-Check
Aug 23 01:00:38 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for 6 minute interval monitoring
Aug 23 01:00:38 unraid01 Parity Check Tuning: DEBUG:   Updated cron settings are in /boot/config/plugins/parity.c
heck.tuning/parity.check.tuning.cron
Aug 23 01:00:38 unraid01 kernel: md: recovery thread: exit status: -4
Aug 23 01:00:38 unraid01 Parity Check Tuning: Paused: Automatic Correcting Parity-Check (2.8% completed)
Aug 23 01:06:35 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 01:06:35 unraid01 Parity Check Tuning: Send notification: mover no longer running:  
Aug 23 01:06:36 unraid01 Parity Check Tuning: Array operation not resumed - outside increment window: Automatic Correcting Parity-Check (2.8% completed)
Aug 23 01:12:37 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 01:18:29 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 01:24:35 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 01:30:33 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
...

Config -

# cat /boot/config/plugins/parity.check.tuning/parity.check.tuning.cfg 
parityTuningScheduled="0"
parityTuningManual="0"
parityTuningAutomatic="0"
parityTuningFrequency="0"
parityTuningResumeCustom="5,15,25,35,45,55 * * * *"
parityTuningResumeDay="0"
parityTuningResumeHour="0"
parityTuningResumeMinute="15"
parityTuningPauseCustom="0,10,20,30,40,50 * * * *"
parityTuningPauseDay="0"
parityTuningPauseHour="0"
parityTuningPauseMinute="10"
parityTuningNotify="0"
parityTuningRecon="0"
parityTuningClear="0"
parityTuningRestart="1"
parityTuningMover="1"
parityTuningCABackup="1"
parityTuningHeat="0"
parityTuningHeatHigh="3"
parityTuningHeatLow="8"
parityTuningHeatNotify="1"
parityTuningHeatShutdown="0"
parityTuningHeatCritical="2"
parityTuningHeatTooLong="30"
parityTuningLogging="1"
parityTuningLogTarget="0"
parityTuningMonitorDefault="17"
parityTuningMonitorHeat="7"
parityTuningMonitorBusy="6"
parityTuningMonitorPartial="1"
parityProblemType="sector"
parityProblemStartSector="0"
parityProblemStartPercent="0"
parityProblemEndSector="100"
parityProblemEndPercent="0"
parityProblemCorrect="0"
parityTuningBackup="1"
parityTuningShutdown="0"

I am therefore wondering why I'm getting "Aug 23 01:06:36 unraid01 Parity Check Tuning: Array operation not resumed - outside increment window: Automatic Correcting Parity-Check".

Tell me I'm doing something silly please

That looks like a bug as because you have no increment Window set it should resume when mover finishes.

I think I can see what is going wrong as it looks like I do not check whether running in increments is set at that point so I will check that I can recreate your issue, and issue a fix if I am right about the cause.

itimpi · August 23, 2023

@estrim Hopefully the release I have just pushed will fix both of the points you have raised. Please let me know if this is not the case or if you spot any other anomaly.

nearcatch · August 23, 2023

Would these settings be correct in Unraid 6.12? I expected the default parity check to start on schedule and then be stopped by this plugin, but it didn't work.

Instead, I got these log messages, and the parity check never stopped at 08:00. I had to manually pause it.

image.png.3a1621a2a0308b1a1e3b86abe59ebcaf.png

estrim · August 23, 2023

8 hours ago, itimpi said:

@estrim Hopefully the release I have just pushed will fix both of the points you have raised. Please let me know if this is not the case or if you spot any other anomaly.

Thanks for the quick update!

I did the update as per the screenshot I'm running 2023.08.23

I also see the file:

# md5sum /boot/config/plugins/parity.check.tuning/parity.check.tuning-2023.08.23.txz 
be35b73c7c37eee02f3da99688833180  /boot/config/plugins/parity.check.tuning/parity.check.tuning-2023.08.23.txz

However my logs say:

Aug 23 16:27:40 unraid01 Parity Check Tuning: Versions: Unraid 6.12.3, Plugin Version: 2023.07.21
Aug 23 16:27:40 unraid01 Parity Check Tuning: Configuration: Array#012(#012    [parityTuningScheduled] => 0#012    [parityTuningManual] => 0#012    [parityTuningAutomatic] => 0#012    [parityTuningFrequency] => 0#012    [parityTuningResumeCustom] => 5,15,25,35,45,55 * * * *#012    [parityTuningResumeDay] => 0#012    [parityTuningResumeHour] => 0#012    [parityTuningResumeMinute] => 15#012    [parityTuningPauseCustom] => 0,10,20,30,40,50 * * * *#012    [parityTuningPauseDay] => 0#012    [parityTuningPauseHour] => 0#012    [parityTuningPauseMinute] => 10#012    [parityTuningNotify] => 0#012    [parityTuningRecon] => 0#012    [parityTuningClear] => 0#012    [parityTuningRestart] => 1#012    [parityTuningShutdown] => 0#012    [parityTuningMover] => 1#012    [parityTuningBackup] => 1#012    [parityTuningHeat] => 0#012    [parityTuningHeatHigh] => 3#012    [parityTuningHeatLow] => 8#012    [parityTuningHeatNotify] => 1#012    [parityTuningHeatShutdown] => 0#012    [parityTuningHeatCritical] => 2#012    [parityTuningHeatTooLong] => 30#012    [parityTuningLogging] => 1#012    [parityTuningLogTarget] => 0#012    [parityTuningMonitorDefault] => 17#012    
Aug 23 16:27:40 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for 6 minute interval monitoring
Aug 23 16:27:40 unraid01 Parity Check Tuning: DEBUG:   Updated cron settings are in /boot/config/plugins/parity.check.tuning/parity.check.tuning.cron

I continued assuming the version text was possibly not updated and unfortunately the 'outside increment window' message still appears.

# grep -E 'Parity|md:|move' /var/log/syslog

Aug 23 23:00:02 unraid01 root: mover: started
...
Aug 23 23:00:29 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 23:00:29 unraid01 Parity Check Tuning: Send notification: mover running:  
Aug 23 23:00:30 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 23:00:30 unraid01 Parity Check Tuning: DEBUG:   detected that mdcmd had been called from sh with command mdcmd nocheck PAUSE 
Aug 23 23:00:35 unraid01 Parity Check Tuning: DEBUG:   ...Pause Automatic Correcting Parity-Check
Aug 23 23:00:35 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for 6 minute interval monitoring
Aug 23 23:00:35 unraid01 Parity Check Tuning: DEBUG:   Updated cron settings are in /boot/config/plugins/parity.check.tuning/parity.check.tuning.cron
Aug 23 23:00:35 unraid01 kernel: md: recovery thread: exit status: -4
Aug 23 23:00:35 unraid01 Parity Check Tuning: Paused: Automatic Correcting Parity-Check (13.6% completed)
Aug 23 23:00:57 unraid01 root: mover: finished
Aug 23 23:07:07 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 23:07:07 unraid01 Parity Check Tuning: Send notification: mover no longer running:  
Aug 23 23:07:08 unraid01 Parity Check Tuning: Array operation not resumed - outside increment window: Automatic Correcting Parity-Check (13.6% completed)
Aug 23 23:12:40 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 23:18:30 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused

Thanks!

itimpi · August 24, 2023

11 hours ago, estrim said:

Thanks for the quick update!

I did the update as per the screenshot I'm running 2023.08.23

I also see the file:

# md5sum /boot/config/plugins/parity.check.tuning/parity.check.tuning-2023.08.23.txz 
be35b73c7c37eee02f3da99688833180  /boot/config/plugins/parity.check.tuning/parity.check.tuning-2023.08.23.txz

However my logs say:

Aug 23 16:27:40 unraid01 Parity Check Tuning: Versions: Unraid 6.12.3, Plugin Version: 2023.07.21
Aug 23 16:27:40 unraid01 Parity Check Tuning: Configuration: Array#012(#012    [parityTuningScheduled] => 0#012    [parityTuningManual] => 0#012    [parityTuningAutomatic] => 0#012    [parityTuningFrequency] => 0#012    [parityTuningResumeCustom] => 5,15,25,35,45,55 * * * *#012    [parityTuningResumeDay] => 0#012    [parityTuningResumeHour] => 0#012    [parityTuningResumeMinute] => 15#012    [parityTuningPauseCustom] => 0,10,20,30,40,50 * * * *#012    [parityTuningPauseDay] => 0#012    [parityTuningPauseHour] => 0#012    [parityTuningPauseMinute] => 10#012    [parityTuningNotify] => 0#012    [parityTuningRecon] => 0#012    [parityTuningClear] => 0#012    [parityTuningRestart] => 1#012    [parityTuningShutdown] => 0#012    [parityTuningMover] => 1#012    [parityTuningBackup] => 1#012    [parityTuningHeat] => 0#012    [parityTuningHeatHigh] => 3#012    [parityTuningHeatLow] => 8#012    [parityTuningHeatNotify] => 1#012    [parityTuningHeatShutdown] => 0#012    [parityTuningHeatCritical] => 2#012    [parityTuningHeatTooLong] => 30#012    [parityTuningLogging] => 1#012    [parityTuningLogTarget] => 0#012    [parityTuningMonitorDefault] => 17#012    
Aug 23 16:27:40 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for 6 minute interval monitoring
Aug 23 16:27:40 unraid01 Parity Check Tuning: DEBUG:   Updated cron settings are in /boot/config/plugins/parity.check.tuning/parity.check.tuning.cron

I continued assuming the version text was possibly not updated and unfortunately the 'outside increment window' message still appears.

# grep -E 'Parity|md:|move' /var/log/syslog

Aug 23 23:00:02 unraid01 root: mover: started
...
Aug 23 23:00:29 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 23:00:29 unraid01 Parity Check Tuning: Send notification: mover running:  
Aug 23 23:00:30 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check running
Aug 23 23:00:30 unraid01 Parity Check Tuning: DEBUG:   detected that mdcmd had been called from sh with command mdcmd nocheck PAUSE 
Aug 23 23:00:35 unraid01 Parity Check Tuning: DEBUG:   ...Pause Automatic Correcting Parity-Check
Aug 23 23:00:35 unraid01 Parity Check Tuning: DEBUG:   Created cron entry for 6 minute interval monitoring
Aug 23 23:00:35 unraid01 Parity Check Tuning: DEBUG:   Updated cron settings are in /boot/config/plugins/parity.check.tuning/parity.check.tuning.cron
Aug 23 23:00:35 unraid01 kernel: md: recovery thread: exit status: -4
Aug 23 23:00:35 unraid01 Parity Check Tuning: Paused: Automatic Correcting Parity-Check (13.6% completed)
Aug 23 23:00:57 unraid01 root: mover: finished
Aug 23 23:07:07 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 23:07:07 unraid01 Parity Check Tuning: Send notification: mover no longer running:  
Aug 23 23:07:08 unraid01 Parity Check Tuning: Array operation not resumed - outside increment window: Automatic Correcting Parity-Check (13.6% completed)
Aug 23 23:12:40 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused
Aug 23 23:18:30 unraid01 Parity Check Tuning: DEBUG:   Automatic Correcting Parity-Check paused

Thanks!

Perhaps you need to reboot to get the correct version to register although I am not sure why! I will see if I can reproduce as I normally would have rebooted when testing.

If after rebooting you can reproduce the problem then please enable the Testing mode logging in the plugin in its settings and after reproducing the error let me have your system's diagnostics so I can see exactly why you still get the error. Maybe the fix I added has a corner case I did not allow for

estrim · August 25, 2023

On 8/24/2023 at 10:52 AM, itimpi said:

Perhaps you need to reboot to get the correct version to register although I am not sure why! I will see if I can reproduce as I norm

If after rebooting you can reproduce the problem then please enable the Testing mode logging in the plugin in its settings and after reproducing the error let me have your system's diagnostics so I can see exactly why you still get the error. Maybe the fix I added has a corner case I did not allow for

Sure thing - changed the logging, rebooted -

You'll see the plugin start at 13:17:22, I clicked 'resume' at 13:17:43, at 14:28:25 you will see the 'outside increment window' message, at 14:44:07 the mover finishes running and mentions the same 'outside increment window' message.

Thanks

Edited September 3, 2023 by estrim

itimpi · August 28, 2023

OK - I think I have finally cracked getting the pause/resume to work correctly in all the cases around mover running. I have reworked my test environment so I can do tests more rapidly going forward which should help with turnaround.

Before I make this release public I am interested if there on any feedback on the changes to notifications from the plugin I am making if the notifications option is enabled in the plugin as follows:

Pause/Resume are going to be displayed as warning (orange) notifications from the plugin instead of normal (green) as they are at the moment. This is consistent with the core Unraid approach for notifications about array operations.
Finished message will be displayed as normal (green) if no errors were found during an array operation and alert (red) if they were.
Any other error I detect will be displayed as alert (red) notifications.
Not sure at the moment if there are any other conditions that I want to treat as warnings (orange) but there may be.
Pure information notifications will be displayed as normal (green) ones.
I will display the mover/backup running/finishing messages as normal (green) notifications if I detect them during a parity operation regardless of whether the option to pause/resume the operation is set for these cases. I thought that even for those not using the plugin to handle pause/resume this is useful information. The pause/resume will only happen if it is set and the notification about this will be warning (orange) as indicated earlier for pause/resume.

Any thoughts on this?

Malachi · August 29, 2023

I've run into some behavior I'm not sure is expected. I was running a parity sync when appdata backup started. The sync operation was paused, as expected, but then not resumed when it finished. It was outside the increment window, but I would have only expected that to apply to parity checks, not syncs, especially since I have increments disabled for Parity-Sync/Data Rebuild.

297413068_2023-08-2901_05_53-Window.png.af96e0187a40da4829ae41d0288e9ede.png

[Plugin] Parity Check Tuning

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

itimpi

itimpi

itimpi

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation