[Plugin] Parity Check Tuning


379 posts in this topic Last Reply

Recommended Posts

sorrry i am just curious, so what is the difference in having a parity drive and not having parity drive in case of a unclean shutdown?

without parity drive i lose the unwritten data from the cache, i reboot and nothing i can do about the lost data
with parity drive i lose the same data cannot do anything about the lost data as in the previous case but have to run parity check. so where does the parity get corrupted then? why is it considered not clean?

Edited by theruck
Link to post
  • Replies 378
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Parity Check Tuning plugin   The Parity Check Tuning plugin is  primarily designed to allow you to split a parity check into increments and then specify when those increments should be run.

i have been working through all cases where this can happen in the code and I think I now have them all fixed in the version running on my test server.  There was a number of places in the code where

I am currently working on the code to allow array operations to be restarted (resumed) from where they were as long as: the array was shutdown cleanly there have been no changes to the

Posted Images

18 minutes ago, theruck said:

sorrry i am just curious, so what is the difference in having a parity drive and not having parity drive in case of a unclean shutdown?

without parity drive i lose the unwritten data from the cache, i reboot and nothing i can do about the lost data
with parity drive i lose the same data cannot do anything about the lost data as in the previous case but have to run parity check. so where does the parity get corrupted then? why is it considered not clean?

Thats why we need different protection. In deep, no protection can protect everything.

Link to post
On 3/1/2021 at 1:15 PM, S80_UK said:

I shall test accordingly and let you have my feedback in the coming days.  I will start by uninstalling and clearing out any old files so that the new plug-in has a fresh start.

Hello @itimpi, I have a minor update for you - some mixed results so far.  A scheduled check with pause worked correctly.  I love the detail in the final notification and email regarding average speed, etc. That's a nice addition.

 

A manual check last night which should not have paused still did so, just after 9am this morning.  I am now letting it complete, but I shall run it again and with some logging enabled this time.   Please let me know if there are any specifics I should look out for. 

 

Just to be clear - I am not too concerned, but I just want to help you get this to where you want it.  (I don't want to be seen as pushing...)  

Cheers, Les.

Link to post

thanks for the feedback.   This issue is proving a little more intractable than I had hoped :( 


The ideal information for me would be to enable the Testing level of logging in the plugin; recreate the problem; and then let me have the syslog (or system diagnostics) so I can see exactly where the plug-in got confused.    In my tests Manual checks were handled correctly so I must be missing something in the way I set up the test.  The testing mode logging will generate a lot of information in the syslog so you do not want to leave it enabled unnecessarily.

 

in addition if you look in the plugins folder on the flash drive I expect you will see a ‘parity.check.tuning.scheduled’ file (it should be a ‘parity.check.tuning.manual’ file for a manual check).   The contents of that file will be a date/time that I can then tie back to the exact point in the syslog where the detection of the check type went wrong so giving me that file as well will help.    The ‘parity.check.tuning.progress’ type files will also give me information on the progress of the check.

 

Link to post

Thanks for the pointers.  While the check was running I had a parity.check.tuning.manual file, but it was of zero length and the date stamp of the file was the point at which I started the manual check.  It has now gone, presumably since the check finished after I resumed it. 

 

I am attaching the parity.check.tuning.progress.save file for the check just completed. It was started at 00:56 this morning, paused automatically just after 09:00 and resumed manually just after 16:00 this afternoon.  Also the other files in case they're useful.parity.check.tuning.cronparity.check.tuning.cfgparity-checks.logparity.check.tuning.progress.saveparity.check.tuning.progress

Link to post

Interesting - that suggests that the code for detecting the type of check was correct but there is an error somewhere in making use of this information.     I think that may mean I can track down the problem without you needing to gather the Testing mode syslog for me.

Link to post

@S80_UK Just thought I would let you know that your last set of feedback has allowed me to identify a point where I do not handle pause correctly for manual checks.  It appears that the  last update fixed the problem about the plugin getting confused about how the check was started but left behind a bug on acting correctly on this information for manual checks under some conditions.

Link to post

Hi @itimpi - thank you for the update.  Just let me know when you'd like me to test.

 

BTW, I am retired, so free time is not an issue.  But when I was working, one of my tasks was usability testing of embedded software.  I have a reputation for breaking things that I need to maintain...  ;)

 

Edited by S80_UK
Link to post
  • 3 weeks later...

I remeber when I ran this plugin earlier (up to this month) that the "Last Check Incomplete" status would read "Error code:paused" and not "aborted."  Is my memory faulty?  Or do I have an error here?

 

image.png.fbc301209a77c1cf0d05a5025b0ec62a.png

 

Just to be sure, I checked the status here:
image.png.be34fc305333776b77625bfef5226a49.png

 

And I the parity check has stretched over two days so far as expected.  I figure it wil take one more night to complete.
My history doesnt show anything for the parity check that started a couple of days ago.
image.png.c71e312fb0e5508011cf78056f384c1b.png

Link to post

I ‘think’ that the messages you are mentioning are from UnRaid, and not the plugin.   The plugin is not able to change many of the standard unRaid messages - merely try and supplement them.   The standard unRaid messages are not ‘increment’ aware which can sometimes be confusing.

 

nothing is written to the History file until the parity check completes (or is cancelled).   Also when the check finally completes the parity history entry is initially written by the standard unRaid code with just the details of the last increment.    The plugin will later alter this entry to be correct for the whole run, but there could be a delay of minutes before this happens.    You can tell if the information is written by unRaid or the plugin by whether there are values in the last 3 columns of the history as standard unRaid does not capture this information.

 

Having said all that it is always possible that there is still some small anomalies on messaging that could be tidied up.   It is something I am continually looking at refining for each release.   If you ever want me to check something out the progress.save file from the plugins and the history.log files from the plugins folder on the flash can give me something to work with.

Link to post

I just walked into my office to find my server having rebooted. I download the diagnostics, and find this in the log (jabba is the server name).

 

Mar 22 13:41:51 jabba Parity Check Tuning: Unclean shutdown detected
Mar 22 13:41:51 jabba emhttpd: Warning: Division by zero in /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.php on line 947

 

Link to post
14 hours ago, MoebiusStreet said:

I just walked into my office to find my server having rebooted. I download the diagnostics, and find this in the log (jabba is the server name).

 



Mar 22 13:41:51 jabba Parity Check Tuning: Unclean shutdown detected
Mar 22 13:41:51 jabba emhttpd: Warning: Division by zero in /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.php on line 947

 

That looks like an error trying to analyse the results of an earlier array operation that was presumably in progress at the time the server rebooted and could be ignored.  Do you have an idea why the reboot might have happened?

 

Having said that the particular set of conditions that triggered that message in theory  should not occur.  I would be grateful if you could post the .progress and/or .progress.save and the parity-checks.log files from the plugin's folder on the flash drive so I can see if I can work out how it happened and avoid it happening in the future even after an unexpected shutdown.

 

Link to post
4 hours ago, itimpi said:

I would be grateful if you could post the .progress and/or .progress.save and the parity-checks.log files from the plugin's folder on the flash drive so I can see if I can work out how it happened and avoid it happening in the future even after an unexpected shutdown.

I don't know where to find that folder. But I suspect it's not there anyway - to avoid further failures, I already uninstalled the plugin. Sorry.

Link to post
2 hours ago, MoebiusStreet said:

I don't know where to find that folder. But I suspect it's not there anyway - to avoid further failures, I already uninstalled the plugin. Sorry.

All plugins are installed on the flash drive under a path of the form:

/config/plugins/plugin-name

 

That particular error can safely be ignored and is almost certainly a by-product of the fact that the server was rebooted unexpectedly.   Without more information there is not much I can do to guarantee it will not occur again under similar circumstances

Link to post

Just pushed an update in time for the fact that many users run their scheduled parity check on 1st of the month.

 

One new feature is the ability to set automatic parity checks (that run after an unclean shutdown) to be run in increments.  I will be interested to get feedback on whether users will find this useful, or whether they think it is a bad idea.

 

There was a lot of internal code re-organization relative to the last release so I may have inadvertently broken something (although I did not spot it in my tests).   Please report any such issues, anomalies you spot, or suggestions for improvement.

 

Please note that the translations file has not yet been updated for multi-language support with the new and revised text strings to match this release.  I will be doing this in the coming week or so.

Link to post

On this morning's release there appears to be a typo in parity.check.tuning.php. Line 1396 the switch statement has a curly brace on the conditional rather than a paren.

Link to post
24 minutes ago, mrMTB said:

On this morning's release there appears to be a typo in parity.check.tuning.php. Line 1396 the switch statement has a curly brace on the conditional rather than a paren.

 

Looking at the release files you are correct  :(   The files on my own server do not have this error as I always run a php -l type syntax check there to pick up obvious errors.

 

The strange thing is that according to my Dropbox history (where I have the source) I corrected that hours before making the release!  Even more frustratingly it is in a function that is currently not even used in this release and is intended for future use.  However I also see that Dropbox has a 'conflicted' copy of that file which suggests there was a problem syncing a change which might be the root cause.

 

Link to post
2 hours ago, dada051 said:

Same bug with my update. Maybe you should host gitlab on your unraid to manage your code :). Very strange nevertheless


Dropbox suits my style of working because of its ability to sync between multiple machines meaning I can edit on whatever one I am currently using and the changes get synced to everywhere accessing that Dropbox location without me having to take explicit action.

 

Pushed a release that should resolve the syntax errors.

 

Link to post
1 hour ago, dada051 said:

It was more a joke than anything else. I understand your choice. Thanks for the fix

 

I realized that after posting the comment, but since the comment was still valid I decided to leave it :) 

 

One good side-effect of this last release issue is that it has prompted me to add syntax checking of the plugin's files to the scripts I use for automating deploying files and building the package so hopefully any such issue is less likely to make it into a release in the future.

Link to post

Hi, this lovely plugin is central to my build not overheating, but it just did.

My parity check was not paused, so my parity drive heated up to 66°C...

 

According to the logs and web UI there is a syntax error in the current version:

grafik.png.d77ca92c494301196e415c7c06b0f4ec.png

grafik.thumb.png.a24654f47613127eab7731da9ad44e7a.png

grafik.png.7a42403e97f27587b3e155a13662829d.png

 

Also this option always reverts to "No" ("Nein" as translated) - no notifications can be enabled..

 

grafik.png.ae5da5076a04808538d9a72fac47e8cd.png

 

Please fix ;)

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.