Ability to Pause Parity Check


spl147

Recommended Posts

It's funny I just came here to make the same request.

 

When doing a parity check sometimes my server becomes a bit unresponsive since I/O volume is high. Since HDD sizes continue to increase, parity keeps taking longer and longer. Maybe it wasn't a problem when unraid first came out, but with 10TB HDDs having your system not fully operational for 24 hours is less than ideal.

 

Apps like Sonarr become unresponsive at some points and you just need it to do one little thing, but having to scrap all the hours you've already spent doing parity is also painful.

 

Alternatively, make parity super low priority. I wouldn't care if it took twice as long if it didn't impact my day to day usage in any relevant way.

Link to comment

There have already been quite a few previous requests around this type of change.

 

The one I would find most useful is to be able to specify a time period between which a parity check could run.   It would then pause until the time period comes around again after which the parity check would resume from where it was.   This would allow the parity check to be spread over a number of days and by only running outside the main hours of use of the server would cause minimal inconvenience.  As disk sizes get larger the value of such a  feature to typical users is steadily increasing.

 

A manual pause/resume feature would be a useful addition, but not as useful as the automated one I envisaged in the above paragraph.  Ideally both would be available.

Link to comment
  • 1 month later...

Or a load based parity check.  As an example, I have a ZFS mirror - when it does a scrub it hammers the disk but when I use the file system, the scrub seems seem to have very little impact and the scrubbing picks up again after the transaction.

 

unraid parity check seems to run at max priority, probably to minimum check time.  As mentioned, with huge drives taking more than a day to check.. some kind of load based scheduling option would help parity checks from hogging IO at the expense of completion time.

 

Link to comment

I'd be happy with an incremental parity check: for example, if I have an 8TB parity disk, I would be able to check 2TB every night for 4 nights (not necessarily consecutive nights). Alternatively, run parity check for x hours at a time. In both cases, resume where the last check ended.

 

This seems like the simplest parity check feature to add, since no monitoring of the system is required for throttling. It's like an abbreviated version of the current parity check.

Link to comment

If this is not a feature that is likely to be added to the main Unraid release any time in the near future perhaps a starting point would be to add something at the driver level that allows a parity check to be started at a specified offset.   I believe that just adding that change would allow for development of a plugin that could provide many of the features requested here as it is already possible to monitor the progress of a check and to stop a check.   I know I would be interested in working on such a plugin.

Edited by itimpi
  • Like 1
Link to comment

I can add start and end range for parity check, and already upon user cancel, the parity operation will stop on a proper boundary.  Theoretically then one could break up a parity check/sync/clear to execute over a specified range or specified time period.  If someone wants to take a stab at the UI or try and talk @bonienl into it, I will add the API... any takers?

  • Like 3
  • Upvote 1
Link to comment
2 hours ago, limetech said:

I can add start and end range for parity check, and already upon user cancel, the parity operation will stop on a proper boundary.  Theoretically then one could break up a parity check/sync/clear to execute over a specified range or specified time period.  If someone wants to take a stab at the UI or try and talk @bonienl into it, I will add the API... any takers?

You should already have had an email from me volunteering to work on this a few days ago!

Edited by itimpi
  • Like 1
Link to comment
32 minutes ago, itimpi said:

I am happy to let you have a go if you are interested as you have much better experience than me at doing this sort of thing!   However I thought you might have a lot of other items taking up your time?

If you have already some ideas then I am happy to assist and make sure it fits in the current GUI design.

Link to comment
8 minutes ago, Squid said:

Who uses email anymore?

Actually thinking about it technically I did not use email but I did ask for exactly what Limetech have suggested might be done :)    What I actually used was the contact form on the web site, but on that site they ask for your email address!

 

Since I did that a few days ago I wondered initially if the Limetech post was a reaction to that.

Link to comment
1 minute ago, bonienl said:

If you have already some ideas then I am happy to assist and make sure it fits in the current GUI design.

I did initially draft out some ideas a while ago to try and work on this and started on a shell script but found that there was currently no way to start a parity check at a specified offset from the start and that proved a show-stopper.

 

Happy to do the donkey work of getting a basic mechanism working and then co-operating on getting it to fit into the current GUI.

Link to comment
1 hour ago, jonathanm said:

I would like to submit a feature request to this plugin in progress.

Respectfully, this is known as feature creep.  The initial plugin must be simple and do one thing well, that is: manual pause and resume of a sync operation in progress.  If author wants to include an automatic pause/resume based on time or a schedule, fine.  Anything beyond that adds needless complexity.  Once reviewed and tested, we will incorporate into webGUI itself.  After this is done and released, then we can consider what additional functionality would be valuable.

Link to comment
48 minutes ago, limetech said:

Respectfully, this is known as feature creep.  The initial plugin must be simple and do one thing well, that is: manual pause and resume of a sync operation in progress.  If author wants to include an automatic pause/resume based on time or a schedule, fine.  Anything beyond that adds needless complexity.  Once reviewed and tested, we will incorporate into webGUI itself.  After this is done and released, then we can consider what additional functionality would be valuable.

Sorry. I just got excited about all the new possibilities that can be explored once the base functionality is solid.

Link to comment
  • 4 weeks later...

It looks like this feature has been added in RC3.

 

Version 6.7.0-rc3 2019-02-09

Base distro:

jq: version 1.6

oniguruma: version 5.9.6_p1

php: version 7.2.14

Linux kernel:

version: 4.19.20

md/unraid: version 2.9.6 (support sync pause/resume)

patch: PCI: Quirk Silicon Motion SM2262/SM2263 NVMe controller reset: device 0x126f/0x2263

Management:

emhttp: use mkfs.btrfs defaults for metadata and SSD support

emhttp: properly dismiss "Restarting services" message

firmware:

added BCM20702A0-0a5c-21e8.hcd

added BCM20702A1-0a5c-21e8.hcd

vfio-pci script: bug fixes

webgui: telegram notification agent bug fixes

webgui: VM page: allow long VM names

webgui: Dashboard: create more space for Dokcer/VM names (3 columns)

webgui: Dashboard: include links to settings

webgui: Dashboard: fix color consistency

webgui: Syslinux config: replace checkbox with radio button

webgui: Docker page: single column for CPU/Memory load

webgui: Docker: usage memory usage in advanced view

webgui: Dashboard: fixed wrong display of memory size

webgui: Dashboard: fix incorrect memory type

webgui: Plugin manager: align icon size with rest of the GUI

webgui: Plugin manager: enlarge readmore height

webgui: Plugin manager: add .png option to Icon tag

webgui: Plugin manager: table style update

webgui: Added syslog server functionality

webgui: syslog icon update

webgui: Main: make disk identification mono-spaced font

webgui: Added parity pause/resume button

webgui: Permit configuration of parity device(s) spinup group.

Edited by pyrater
format
Link to comment

Yes.   I am currently testing a ‘Parity Check Tuning’ plug-in I am developing that utilises this feature to allow the parity check to be run in increments spread over several days (only running at defined times).   Not too fancy at the moment but still looks like being very useful for those who have large drives and do not want the parity check to be bogging down performance by having it running during prime time. 

  • Like 1
Link to comment

@itimpi

May I add my 2 ct. for consideration as you're still in development.

Several people are using the S3 standby plugin.
The plugin performs some checks to evaluate if it can issue the S3 standby command.
Do you think you can use that work to implement the "pause parity" feature in such a way,
as to run automatically when the server is running idle?
e.g. if there is "low activity" on the NIC or
if the "running hours" is the only condition that is blocking the S3 sleep.
(Probably need both to be able to stop the check when NIC load goes up again)
This would be a very useful feature.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.