Jump to content
itimpi

[Plugin] Parity Check Tuning

92 posts in this topic Last Reply

Recommended Posts

Thanks for creating this plugin.  I have set up successfully and I'm currently part way through a check that I'm spreading across three nights.  Looks good so far. :)

 

Share this post


Link to post

I have released a new version of the plugin with a significant number of improvements and changes:

  • Added new CA plugin update feature
  • Improved GUI on Settings page with more extensive help and better checks on settings
  • Added option to Pause/Resume array operations if disks overheat
  • Added option to Pause/Resume of Parity-Sync/Disk Rebuild and disk Clear operations
  • Added option to send notifications on Pause/Resume
  • Added sending a notification if its detected the array was stopped while an array operation was in progress
  • Added a Paypal donate option as a way of gauging interest in the plugin (be interesting to see if any actually happen!) .

I will be particularly interested to see how the option for Pause /Resume on temperature thresholds works out in practice for those who have such a problem.   I do not have such a problem and was only able to test this feature by setting my trigger temperatures to be much lower than required as in normal use my own Unraid systems do not suffer from the disks over-heating.

 

I would also be interested in any feedback on how useful people are finding the plugins prime purpose of being able to run scheduled parity checks in increments.   I have been assuming that 'no news' is good news in that users are not encountering any problems but some positive feedback would be encouraging.

Share this post


Link to post
Posted (edited)

Crazy. Thanks.

 

But i wonder how to use? I cant find any new buttons or anything under scheduler? (liek for temps)

 

I just need fans for my hdds because of parity check, so thats again like 10 watt less power draw (and not so loud). So this would be usefull.

 

Or do i need to wait till i see the update? But the donate button is there, so its the newest version, right!?

Edited by nuhll

Share this post


Link to post
Posted (edited)
17 minutes ago, nuhll said:

 (unless I messed something up while uploading the update)Crazy. Thanks.

 

But i wonder how to use? I cant find any new buttons or anything under scheduler? (liek for temps)

 

I just need fans for my hdds because of parity check, so thats again like 10 watt less power draw (and not so loud). So this would be usefull.

 

Or do i need to wait till i see the update? But the donate button is there, so its the newest version, right!?

The version I have uploaded today is 2019.03.28.     If you have it should look like the screenshot in the first post.     My guess is that we have to wait for CA to see the update (unless I messed something up while uploading the update).   What version does CA show as being available?

Edited by itimpi

Share this post


Link to post
The version I have uploaded today is 2019.03.28.     If you have it should look like the screenshot in the first post.     My guess is that we have to wait for CA to see the update (unless I messed something up while uploading the update).   What version does CA show as being available?
The XML for ca has couple fatal errors. You've got a couple ampersand that need to be escaped and the opening and closing tags for iconfa dont match each other.

The plug file also has errors that prevent unraid from installing the updates quick look and I'd say it's the double quoted icon

Sent via telekinesis

Share this post


Link to post

Did the Update and I see lots of new changes. 😀

 

What I don't see working:

  • When I click on the support under your installed plugin in list of installed Plugins. it takes me to. https://forums.unraid.net/topic/70783-plugin-mover-tuning/ @Squid you trying to mooch on his gig? lol
  • I don't see a PayPal Button yes I looked. I'm using FireFox unless Its a Browser thing?
  • Also I have a Broken Image for your Plugin in the Plugin list.

 

I also noticed after the update it disabled the Plugin. Unless that is a supposed to happen thing?

 

Share this post


Link to post
1 hour ago, kizer said:

Did the Update and I see lots of new changes.

That is at least progress :)    Hopefully you will like some of the changes.    

 

Please provide feedback on how well they work for you as I only have a limited testing environment so could easily have missed some edge cases.   The saving grace is that if I have messed anything up it is unlikely to cause to much of an issue other than missing a Pause or getting an unexpected Resume.    The plugin never installs any new system libraries or touches any user data so should not be dangerous even in a failure mode in the event one occurs.

1 hour ago, kizer said:

 

 

I get the same behaviour :(   What is strange is that my browser (Microsoft Edge) clearly shows it pointing to the correct support thread.    I cannot find any reference to Mover Tuning in any of my code or the templates that control installation.  Also clicking on the Support link from the Apps tab ends up at the correct location.   Not sure, therefore, where the incorrect link is coming from but as I can reproduce this I can at least work on tracking it down.

1 hour ago, kizer said:

 

  • I don't see a PayPal Button yes I looked. I'm using FireFox unless Its a Browser thing?

I need to look at that further.   The button seems to be appearing on the Apps tab but is then going to the wrong Donate page so something definitely needs looking at.   Again I am not sure why.   I thought the Donate button was auto-carried over from the Apps tab but I could be wrong about that and I need to do something explicit to get onto the Plugins page.

 

1 hour ago, kizer said:

 

  • Also I have a Broken Image for your Plugin in the Plugin list.

 

Fixed for the next release!   The problem was I was trying to specify an icon I though was available on UnRAID but turns out not to be the case.  When testing the code I was by-passing the plugins tab so managed to miss that the icon was not correct.

 

1 hour ago, kizer said:

 

I also noticed after the update it disabled the Plugin. Unless that is a supposed to happen thing?

 

That will be a by-product of introducing the new features and changing the names of some of the variables used internally for storing settings so you reverted to the defaults.  Should not happen again going forward.   Trying to safely migrate the old settings looked a little error prone and so I thought that with this release users were likely to revisit the settings page and I could thus safely let affected settings revert to defaults.

Share this post


Link to post

I've pretty much implemented everything you've added to try it. I'm not concerned about the cooling at all, but I know some are and I'll tinker with that sometime. The Way my CoolerMaster 590 is setup with Fans I'd be very reluctant to think it would ever heat up. 

 

I think this Plugin you put together is pretty awesome honestly. I'm sure a lot of people will get a lot of use out of it as Drives get larger and are Hoarding keeps happening. :D I only see a 13-14 Hour Parity Check on my end, but I'm sure people probably go days. 

 

I'll let you know after the first couple days of next month to see what I see. Notifications and all. I'm letting this go Automatic. 

 

Share this post


Link to post

Great!    I hope to push out an update over the weekend to clear up the little points you noted.

 

I had some personal motivation for getting this plugin working :) .   I have a slow server and a 12TB parity drive and my check can take up to 30 hours so running the check in increments is of great personal benefit.   I always think that implementing something you are going to be using yourself tends to lead to a goid final result.

Share this post


Link to post
5 hours ago, itimpi said:

What is strange is that my browser (Microsoft Edge) clearly shows it pointing to the correct support thread.

It's not.  The .plg has the wrong URL and that URL has the "thread" number for mover tuning and not parity tuning

 

6 hours ago, itimpi said:

I thought the Donate button was auto-carried over from the Apps tab but I could be wrong about that and I need to do something explicit to get onto the Plugins page.

It's never done that.  You have to add in that button if you want onto the .page

Share this post


Link to post
26 minutes ago, Squid said:

It's not.  The .plg has the wrong URL and that URL has the "thread" number for mover tuning and not parity tuning

That is strange - the URL I have is https://lime-technology.com/forums/topic/70783-parity-check-tuning/.  Now that you mention it I agree that seems to be the wrong thread number, but not sure how that happened as I thought the URL was originally set up as a copy/paste.    However looking at the gitHub commits it was always the value it currently is now in the PLG file.   Maybe I am getting things confused with the change to the CA template (which has the correct URL).  I tend to go via the Apps tab as it much faster to load than the Plugins one.   I am actually surprised the URL went anywhere useful as the name part after the thread number is wrong but that is probably the forum software trying to be helpful in some way.

 

35 minutes ago, Squid said:

It's never done that.  You have to add in that button if you want onto the .page

Fair enough.   Not sure if I want to bother but it is nice to know that is what is required (and I can add it to my development notes for future reference).  I was more interested in the Donate button as a way to get feedback on interest in the plugin rather than any expectation of a significant amount being paid.

Share this post


Link to post

Looks like last night Parity started and as expected at 6AM I received a notification saying Parity has been paused at 42%.

 

Could there be an indicator or something you can click that says "Current Parity Completion %." on the Plugin?

Just wanted something visual just encase I wasn't running Send Notifications.


Ah I totally forgot you can click on Array Operation if your running Tab'd View and see the current Completion.

Share this post


Link to post
Posted (edited)

Glad to here that it appears to be working as expected. :)

 

i have raised a feature request to suggest that when a parity check is paused this is indicated in the Status line (I.e. the same message as when running but with ‘(paused)’ added on the end).   That seems the right/best way to handle this and keeps the fact the operation is incomplete easily visible.    I could look into doing it myself if nothing materialises but it might involve amending one of the main built-in Unraid scripts which I would rather not do as it might be prone to breaking things on an Unraid update which would be very bad news.    If you think it would add value to add it to the Settings page that would be quite trivial to do.   In fact in the early stages of the development of the plugin when I was trying to work out how to detect array status I actually had it there but thought it did not add much value so removed it to keep things cleaner.

Edited by itimpi

Share this post


Link to post

Honestly I’m not 100% sure what I’d recommend. I almost did a reboot and if it hadn’t been I just happened to check my phone because I use Prowl to tell me things I wouldn’t of caught the Parity Check paused 42% mention. Just slipped my mind. Normally my Check begins at 12AM and finishes around 2PM. So I automatically know not to do anything until around 3PM on the 1st. 

 

What you gave us already solves a pretty big problem as it is and honestly I hardly ever reboot unless I’m making a system change and I normally reserve those for end of Month so they Parity Check can catch things. 

 

I guess the worst maybe a check if an accidental reboot happens if that’s even possible or something or maybe nothing at all. We cant always have our cake eat it too. Lol 

Share this post


Link to post
Posted (edited)

The change turned out to be trivial so I have sent a pull request on the Unraid webgui GitHub repository that will display something like the following when an operation is paused.    I think this is a good solution that should work well?

 

 

Untitled.png

Edited by itimpi

Share this post


Link to post

I think it would. As long as you said it doesn’t affect any future updates and get wiped out. 

Share this post


Link to post
Posted (edited)
2 hours ago, kizer said:

I think it would. As long as you said it doesn’t affect any future updates and get wiped out. 

That is why I submitted as a ‘pull’ request to the Limetech wegui github repository rather than getting the plugin to try and patch the relevant file.    Doing it that way as long as the ‘pull’ request is accepted by Limetech it will automatically be incorporated into the next Unraid build.

Edited by itimpi

Share this post


Link to post
2 hours ago, itimpi said:

That is why I submitted as a ‘pull’ request to the Limetech wegui github repository rather than getting the plugin to try and patch the relevant file.    Doing it that was as long as ‘pull’ request is accepted by Limetech it will automatically be incorporated into the next Unraid build.

 

Sorry I knew what you meant and should of responded as such. I personally agree with your thinking. 

 

 

Day 2 and now I’m getting a 87% Parity up from yesterday’s 42% Completion. Things seem to be working just fine with the Monthly Parity Schedule and the Tuner installed. 

Share this post


Link to post

I just logged in and saw this on my Browser. If this isn’t an in your Face Heads up I don’t know what is. Lol. 

 

I think with your suggested pull and/or just this I can’t think of anything that wouldn’t be satisfying to a user. I did as well get a Prowl and email notification too. I am testing every option excluding the Temp on the drives. 

D92F0788-98D5-4588-A39E-9B0238CA14B4.png

Share this post


Link to post

100% completion. Funny thing it says at 694MB/sec lol. I normally average around 80-90. 

 

However no errors and I would like to call this another successful operation. 😀

Share this post


Link to post
Posted (edited)
29 minutes ago, kizer said:

100% completion. Funny thing it says at 694MB/sec lol. I normally average around 80-90. 

 

However no errors and I would like to call this another successful operation. 😀

UnRAID only takes into account the last increment when calculating speed.   This can be widely misleading as far as speed is concerned as often it is just reading the parity disk (or fastest array disks) at that point.  Also I have a feeling it thinks it checked the whole of parity in the time it took to run the last increment.    In other words it does not take into account that the check has been paused and resumed a number of times.  The one feature for the plugin that I still need to complete is for the plugin to provide its own history which will include tracking both running and paused time.    This will allow for a more realistic calculation of average speed and will also tell you about total elapsed time.

Edited by itimpi

Share this post


Link to post

Yep totally figured it was that. 4TB in 1.65 hours or whatever the true time was is insane lol. 

 

- Notifications for start/stop - check

- Over Heat Check - check

- Appears to work with Mover Tuner - check

- Configurable Start/Stop - check

 

Possible ideas maybe and I’m totally just chucking them out there:

- CPU idle begin check between specific times/dates

- History Logs like you mentioned 

- Pause/Resume if a specific Process is triggered/stopped. Handbrake or whatever 

- Stop/Resume Dockers while running 

 

Ramble Ramle I think anything you add is Icing on the Cake and I’m a completely satisfied user. 

 

Share this post


Link to post

hi, is this still available? CA finds nothing after various searches. nothing for "parity", and only returns the Mover Tuning when searching for "tuning". thanks.

Share this post


Link to post
5 hours ago, gellux said:

hi, is this still available? CA finds nothing after various searches. nothing for "parity", and only returns the Mover Tuning when searching for "tuning". thanks.

Are you using a version of unraid that supports pausing the parity check?

Share this post


Link to post
5 hours ago, gellux said:

hi, is this still available? CA finds nothing

 

What version Unraid? See requirements in 1st post.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.