Improved (more precise) Incremental Parity Check

[Jeff Adams]

I've been running Unraid for a few years now, but haven't been running scheduled parity checks because of the time it would take to complete them.  Then I found out about the tuning plugin which allows it to run for certain times of the day and then suspend the process.  This sounded great and indeed I have started using it. 

 

However its not ideal.  It checks however much of the disk it can check in the time window allowed.  If it doesn't finish the whole check by the next time it is scheduled to run it starts over from the beginning so the end of the array may not be checked.  If it finishes before there may be days it doesn't check anything.  Also if the system is shut down it won't start the check again for awhile (unless I change the start date to the next day, which means it would start the check over from the start).

 

I would like to propose a more robust/precise solution to incremental parity checks.  Something where I could set the parity check to run daily.  Then I could specify the number of chunks to spread the check over.  One of the chunks would be checked every day.  Could be modulus of the days since epoch.  So for example.  Lets say I wanted to check the entire array every 2 weeks.  I schedule start to daily.  Set my start time.  Set my number of chunks to 14.  There are currently 738900 days since epoch.  738900 mod 14 = 8 so chunk #8 would be checked today.  Tomorrow it would do chunk 9.

 

I do realize the parity checker itself would have to be changed to be told what part of the array to check whereas the tuning plugin uses the existing one so this suggestion would be more difficult to implement.  I just like the idea of a precise equal part of the array being checked everyday and it can continue the check the next day without starting again from the beginning and not wait til the next weekly/monthly start if the computer happens to be shut down for a day.

[trurl]

Why would you want daily parity checks? Parity is maintained realtime, parity check is just to make sure all is well. Most only check parity monthly.

[Jeff Adams]

I would rather it spend 30 minutes daily checking the array in the middle of the night, than 12+ hours once a month and impact anything I'm trying to do on that one day.  Personal preference I guess.

[itimpi]

19 minutes ago, Jeff Adams said:

However its not ideal.  It checks however much of the disk it can check in the time window allowed.  If it doesn't finish the whole check by the next time it is scheduled to run it starts over from the beginning so the end of the array may not be checked.  If it finishes before there may be days it doesn't check anything.  Also if the system is shut down it won't start the check again for awhile (unless I change the start date to the next day, which means it would start the check over from the start).

Are you sure you are talking about the Parity Check Tuning plugin because:

• The plugin does not restart from the beginning for each increment.   It continues from the point already reached when starting the next increment.  This is repeated until the whole parity drive has bedn checked.   
• The plugin has an option to restart from the point last reached after a reboot (as long as the shutdown as a tidy one).   This caters for those who want to shutdown their server overnight or those who have regular power cuts (as long as they have a UPS so that the clean shutdown can be achieved).

iI looked into using size increments instead of time ones but although technically possible this proved not that useful in practice since the speed can be affected by many factors so the time for a given percentage of the parity drive could vary significantly.  Time based increments proved far more practical and it is easy enough for users to work out how many increments a full check takes if they run a similar length increment each day.

[trurl]

29 minutes ago, Jeff Adams said:

30 minutes daily checking the array in the middle of the night

Your nights are only 30 minutes long?

 

Even without the plugin, latest releases do incremental parity checks. Mine run monthly, 12am-6am, and is finished partway through the 3rd night of the month..

[Kilrah]

3 hours ago, itimpi said:

The plugin does not restart from the beginning for each increment.

The issue is OP is doing way too frequent checks so there's a new one scheduled before the previous is done.

 

4 hours ago, Jeff Adams said:

I would rather it spend 30 minutes daily checking the array in the middle of the night, than 12+ hours once a month and impact anything I'm trying to do on that one day.

If you schedule a check every 1-2 months like is usual then even with 30 mins a night you'll have ample time for it to finish before a new one is scheduled. 

There is no point constantly checking, you're just adding wear to your drives for nothing.

Edited January 15 by Kilrah