[Plugin] Parity Check Tuning


Recommended Posts

3 hours ago, itimpi said:

This warning is because with the settings you have set in the screenshot then the increment time would be 14 hours and it is unusual for anyone using this plugin to want the increment to be that long.   It is only a warning as common mistake is (as the message indicates) for users to get the pause and resume time the wrong way around.

Just had another look at this - the resume time is 22:00 and the pause time is 12:00 (mid-day) so the increment IS 14 hours so the warning is correct.    if you meant the pause time to be midnight then you select the first entry from the hours drop-down as the time.

Link to comment

I recently installed this plugin and completed my first parity check in 8 hour segments. The check took 25 hours and now the History section on my Main page says: Duration: 1 hour, 1 minute, 32 seconds which was the amount done during the last day only.

 

Is this normal or am I doing something wrong. 

Link to comment
6 hours ago, RichardU said:

I recently installed this plugin and completed my first parity check in 8 hour segments. The check took 25 hours and now the History section on my Main page says: Duration: 1 hour, 1 minute, 32 seconds which was the amount done during the last day only.

 

Is this normal or am I doing something wrong. 

That is because Unraid is only counting the last increment.

 

The plugin will have put the correct information into the History file and if you have the plugin set to issue notifications then those messages also give the correct information.

 

I must have another look to see if I can convince Unraid to display the correct information.

  • Like 1
Link to comment

Hey Everyone! I've been trying to get the "Increment Frequency/Custom" working for what I need, but I'm struggling.

 

I have cooling issues with my Unraid, and what my goal is to allow the the Parity Check to 'Pause when disks overheat', then have the Custom Increment frequency pause the Parity operations for ~30mins to let the disks cool down, and then resume (or at least check if the disks are cooled down enough) and then Resume parity operations.

 

Clearly my cron skills are weak, is there an "Increment Resume Time" and "Increment Pause Time" that someone can suggest?

 

(thanks again for all the awesome features in the Parity Check Tuning plugin!)

 

Link to comment
20 minutes ago, rsbuc said:

Hey Everyone! I've been trying to get the "Increment Frequency/Custom" working for what I need, but I'm struggling.

 

I have cooling issues with my Unraid, and what my goal is to allow the the Parity Check to 'Pause when disks overheat', then have the Custom Increment frequency pause the Parity operations for ~30mins to let the disks cool down, and then resume (or at least check if the disks are cooled down enough) and then Resume parity operations.

 

Clearly my cron skills are weak, is there an "Increment Resume Time" and "Increment Pause Time" that someone can suggest?

 

(thanks again for all the awesome features in the Parity Check Tuning plugin!)

 

I think you are over-thinking this!     You only want to set the increment pause/resume times to define the maximum time period you want the parity check to potentially run.

 

You then set the temperature related pause resume values and as long as you are within the increment period the plugin will pause/resume the check based on disk temperatures.    You may also want to have aggressive spin down times on the drives as experience has shown that simply keeping them spinning even if no I/O is taking place significantly extends the cool down time.

Link to comment

Hello. Thanks for providing this most useful plugin, itimpi! Unfortunately I ran into a small problem with it when I last restarted my Unraid server. When this plugin's installed (specifically, version 2022.12.05 on Unraid version 6.11.5), the root of the filesystem gets its permissions set to 777. This causes OpenSSH to refuse to allow login with public/private key pairs with the error message "Authentication refused: bad ownership or modes for directory /" because it's worried about some other user nefariously providing extra "authorised" keys. If I uninstall the plugin and restart, / is back to 755 which satisfies OpenSSH.

 

The workaround, of course, is to manually set the permissions of / after start up, but it would be better if this plugin could avoid changing them in the first place.

Link to comment
17 hours ago, itimpi said:

I think you are over-thinking this!     You only want to set the increment pause/resume times to define the maximum time period you want the parity check to potentially run.

 

You then set the temperature related pause resume values and as long as you are within the increment period the plugin will pause/resume the check based on disk temperatures.    You may also want to have aggressive spin down times on the drives as experience has shown that simply keeping them spinning even if no I/O is taking place significantly extends the cool down time.

Hello! Am I understanding this correctly? The plugin will pause the parity operation when the disks reach the temperature threshold and wait until the temps fall below the temperature threshold value -- then the script immediately resume the parity operations? or will it only attempt to resume after the 'Increment resume time' schedule?

Link to comment
11 minutes ago, rsbuc said:

Am I understanding this correctly? The plugin will pause the parity operation when the disks reach the temperature threshold and wait until the temps fall below the temperature threshold value -- then the script immediately resume the parity operations?

This is the basic behaviour as long as the time is within the overall time slot set for an increment.   How long a temperature related pause will last depends on how quickly your drives cool down to reach the resume temperature threshold.  The plugin will take into account if you have set specific temperature threshold settings at the Unraid level on a drive over-riding the global ones.  You may find the Debug logging level helps with a basic understanding of what the plugin is doing without having to know too much detail of the underlying mechanisms being used.

 

Once you get outside the time slot for the overall increment then the plugin will pause the check and the temperature related pause/resume will stop happening (until the time comes around to start the next increment).

 

If I can provide any further clarification then please ask.  As a new user if you can think of items I could add to the built-in help that would have helped you then please feel free to suggest them.

Link to comment
2 hours ago, itimpi said:

This is the basic behaviour as long as the time is within the overall time slot set for an increment.   How long a temperature related pause will last depends on how quickly your drives cool down to reach the resume temperature threshold.  The plugin will take into account if you have set specific temperature threshold settings at the Unraid level on a drive over-riding the global ones.  You may find the Debug logging level helps with a basic understanding of what the plugin is doing without having to know too much detail of the underlying mechanisms being used.

 

Once you get outside the time slot for the overall increment then the plugin will pause the check and the temperature related pause/resume will stop happening (until the time comes around to start the next increment).

 

If I can provide any further clarification then please ask.  As a new user if you can think of items I could add to the built-in help that would have helped you then please feel free to suggest them.

Interesting, I've enabled Debug logging, and that totally demystifies a lot of what the plugin is doing (Thanks for that). Here is what I'm seeing (I'm sure I have a bad setting or something) -- I start the parity check, it runs for an hour or so, then the hard drives hit their temperature limit, and the parity check pauses. The drives spin down, and the drives cool off, but the plugin doesn't seem to resume the parity operations.

 

If I "Spin up all disks" it will detect the drive temperatures as being cool again and resume the parity check.

 

are there special disk settings that I need to enable for this to work properly?

(also, thanks again for trying to helping me out!)

Screenshot 2022-12-15 152350.png

Screenshot 2022-12-15 152505.png

Link to comment
16 hours ago, rsbuc said:

Interesting, I've enabled Debug logging, and that totally demystifies a lot of what the plugin is doing (Thanks for that). Here is what I'm seeing (I'm sure I have a bad setting or something) -- I start the parity check, it runs for an hour or so, then the hard drives hit their temperature limit, and the parity check pauses. The drives spin down, and the drives cool off, but the plugin doesn't seem to resume the parity operations.

 

If I "Spin up all disks" it will detect the drive temperatures as being cool again and resume the parity check.

 

are there special disk settings that I need to enable for this to work properly?

(also, thanks again for trying to helping me out!)

Screenshot 2022-12-15 152350.png

Screenshot 2022-12-15 152505.png

If you think the plugin is not correctly resuming when drives cool down, then perhaps you can try turning on the Testing level of logging in the plugin and sending me the resulting logs as that will allow me to see the fine detail of what the plugin is doing under the covers.   Testing the temperature related stuff is extremely tricky as my systems do not suffer from heat issues so I have to artificially try to set up tests to simulate temperature issues.

Link to comment
On 12/16/2022 at 8:16 AM, itimpi said:

Testing the temperature related stuff is extremely tricky as my systems do not suffer from heat issues

Nothing a blanket tossed over the server can't solve. 🤣

 

Seriously though, maybe set up an old rig with a couple small drives, and cover up the intakes with a piece of cardboard, you know, for science.

Link to comment
17 minutes ago, JonathanM said:

Seriously though, maybe set up an old rig with a couple small drives, and cover up the intakes with a piece of cardboard, you know, for science

I don't have an old rig to setup any more! :)   I moved a few months ago and anything superfluous to requirements was left behind/scrapped/abandoned.

Link to comment

I have both Mover Tuning and Parity Tuning installed. I have Monthly Parity Checks and Mover scheduled to run at night, but I’d like to go a step further and, when both Mover is running, the Monthly Parity Check pauses… I just can’t get my head around the right combination of settings to make it happen

 

help would be greatly appreciated 

Link to comment
24 minutes ago, newoski said:

I have both Mover Tuning and Parity Tuning installed. I have Monthly Parity Checks and Mover scheduled to run at night, but I’d like to go a step further and, when both Mover is running, the Monthly Parity Check pauses… I just can’t get my head around the right combination of settings to make it happen

 

help would be greatly appreciated 

There is an option in the Parity Check Tuning settings to pause a parity check while mover is running.   Is this not what you want?

 

thinking about it I am not certain I have tested it against mover being started by the Mover Tuning plugin, but if that is not working I would suspect it would be a relatively trivial fix.

Link to comment
2 hours ago, itimpi said:

There is an option in the Parity Check Tuning settings to pause a parity check while mover is running.   Is this not what you want?

 

thinking about it I am not certain I have tested it against mover being started by the Mover Tuning plugin, but if that is not working I would suspect it would be a relatively trivial fix.

 

BINGO! That's it. I found it by looking into the descriptions. Forgot about the Help button. Thanks!

"Pause an array operation (Parity Check, Parity-Sync/Disk Rebuild, disk Clear) if mover is running. These two types of activity adversely affect each other's performance due to contention on the disk drives so it is more efficient to not have them running at the same time."

Link to comment

Attempting to get the "Parity Check Tuning" plugin working on my unRAID installation.  When I click on the "History" button in the "Parity Check Tuning" plugin page a new dialog window opens but it says "File not found."  I tried uninstalling and reinstalling the plugin but still see the same "File not found." message.  I'm on unRAID v6.11.5 and the version of the plugin is 2022.12.05.

Link to comment
10 hours ago, ecuster said:

When I click on the "History" button in the "Parity Check Tuning" plugin page a new dialog window opens but it says "File not found."

I assume that file must come from flash since Unraid doesn't install any OS files anywhere else.

 

What do you get from command line with this?

ls -lah /boot

 

Link to comment
18 hours ago, ecuster said:

Attempting to get the "Parity Check Tuning" plugin working on my unRAID installation.  When I click on the "History" button in the "Parity Check Tuning" plugin page a new dialog window opens but it says "File not found."  I tried uninstalling and reinstalling the plugin but still see the same "File not found." message.  I'm on unRAID v6.11.5 and the version of the plugin is 2022.12.05.

That “should’ give exactly the same results as clicking on the history button on the Main screen.   I only added it there as a convenience to avoid having to navigate back to the Main screen to get at the history, bit I’ll check that does still work on my system.    

 

Edit:  I can confirm that pressing the History button from the plugin settings page no longer appears to work.  I will get this fixed for the next update.

Link to comment
  • 2 weeks later...
On 12/16/2022 at 8:16 AM, itimpi said:

If you think the plugin is not correctly resuming when drives cool down, then perhaps you can try turning on the Testing level of logging in the plugin and sending me the resulting logs as that will allow me to see the fine detail of what the plugin is doing under the covers.   Testing the temperature related stuff is extremely tricky as my systems do not suffer from heat issues so I have to artificially try to set up tests to simulate temperature issues.

I've finally had a few mins to test this out with the TESTING log mode enabled. I think you were hinting at what I've seen.

 

When the array goes into 'overheat mode' and the parity check pauses, the disks eventually spin down and the temperature value in the log goes to "Temp=*" instead of showing an actual Temperature value, so the Parity Check Tuning script doesn't see a valid numerical temperature value to resume the parity check process.

 

after waiting ~12minutes, I manually clicked 'spin up disks' and then 6minutes later the parity check process resumed as it was able to see the temperature values when the disks were spun up.

 

I'm attaching my syslog.

syslog.txt

Link to comment
14 hours ago, rsbuc said:

When the array goes into 'overheat mode' and the parity check pauses, the disks eventually spin down and the temperature value in the log goes to "Temp=*" instead of showing an actual Temperature value, so the Parity Check Tuning script doesn't see a valid numerical temperature value to resume the parity check process.

The plugin always treats the case where the temperature is returned as '*' due to spindown as 'cool' so there needs to be something else going on.   I will see if I can work out what it is from the syslog you provided.

Link to comment
2 hours ago, itimpi said:

The plugin always treats the case where the temperature is returned as '*' due to spindown as 'cool' so there needs to be something else going on.   I will see if I can work out what it is from the syslog you provided.

No worries, I appreciate the effort, if you'd like more info let me know.

Link to comment
22 hours ago, rsbuc said:

No worries, I appreciate the effort, if you'd like more info let me know.

Having examined the logs you provided I can see the point at which the plugin tries to pin up the drives to get the temperature of those currently reporting ‘*’ because they are spun down (after all drives not spun down have cooled sufficiently to make a resume possible).   For some reason it appears that spinup is not working.   I will see if I can work out why.

Link to comment
  • 4 weeks later...

Just pushed a release with quite a few small fixes:

 - New: Remove support for Unraid versions prior to 6.9.2
 - New: Add warning about incompatibility with Unraid Cumulative Parity Check option.
 - Fix: History button on Settings page not working for Unraid 6.11 and 6.12
 - Fix: Extended Parity History on not being displayed from Main screen
 - Fix: After restart was not pausing if outside increment time slot.
 - Fix: If increment type not being monitored and paused before reboot was not re-instating pause on restart
 - Fix: Updates to make master translation file agree with current plugin.
 - Fix: Improve some of the code to better handle array not yet started.
 - Fix: Remove code that failed to spin up drives and leave it to Unraid to do this.

 

Hopefully I have not broken any existing working functionality :)

 

If anyone spots any anomalies then please let me know.

Link to comment

After updating, my passwordless ssh broke and I see this in syslog

Authentication refused: bad ownership or modes for directory /

 

This happened on an earlier update of this plugin as well. I read the discussion of this in another thread and it has something to do with /usr permissions of this plugin?

Edited by csrihari
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.