How does dual parity work, actually?


Recommended Posts

10 hours ago, trurl said:

Unraid monitors these things for you but you should set up Notifications to alert you immediately by email or other agent so you will know without even opening the webUI.

Thank you, and yes, I have had notification setup for years :-)

Link to comment
  • 1 year later...

I know it's an old topic but is there something that detects bitrot (of any type, including bad sectors), and fixes the lost data be rescuing it using the parity, on a regular basis on unRAID? I read there is something like this in synology so I'm hoping to not have to buy synology but instead self-build.

Link to comment
1 hour ago, Bob2 said:

I know it's an old topic but is there something that detects bitrot (of any type, including bad sectors), and fixes the lost data be rescuing it using the parity, on a regular basis on unRAID? I read there is something like this in synology so I'm hoping to not have to buy synology but instead self-build.

When a drive fails a read request, all the rest of the drives are read, the failed read request is fulfilled from the resulting parity calculation, and that data is written back to the drive that failed the read request. If that write fails, then the drive is marked as invalid (red X) and the parity calculations are used for all further I/O to that drive slot, the physical drive is no longer accessed.

 

That is the total extent of what Unraid does by default, anything beyond that would need to be handled by recovery from backups.

Link to comment
  • 6 months later...

Circling back on this....

Current setup:
1x Parity = 14TB 
10x disk (largest being 10TB in the set)

I have a spare 10TB and 12TB on hand.

 

If I were to add the 12TB to the second parity, the latest data disk in the 'disk' set moving forward could only be 12TB....
I think I may wait until I can track down another 14TB for the second parity disk to match the first.

Thoughts?

Thanks.

Link to comment
12 minutes ago, bombz said:

If I were to add the 12TB to the second parity, the latest data disk in the 'disk' set moving forward could only be 12TB....

Yes - no data drive can be larger than any of the parity drives.   Of course you could later put a larger drive into the parity2 site.

  • Like 1
Link to comment
2 minutes ago, itimpi said:

Yes - no data drive can be larger than any of the parity drives.   Of course you could later put a larger drive into the parity2 site.

Hey,

Appreciate the prompt response.
Not a bad thought if I come across a larger disk than 14TB, and not a bad idea either!

Thank you kindly :-) 

Link to comment
On 2/27/2022 at 2:08 PM, itimpi said:

Yes - no data drive can be larger than any of the parity drives.   Of course you could later put a larger drive into the parity2 site.

Hello again,
Managed to find a 18TB to add to second parity... sigh so much disk to toss at parity!
So this will be added to parity 2.

As I move forward I can upgrade each parity in a 'step' format from the sounds of it?
For example: In years to come Parity 1 could go to 20TB, then party 2 to 24TB, and so forth?

Link to comment
1 hour ago, bombz said:

Hello again,
Managed to find a 18TB to add to second parity... sigh so much disk to toss at parity!
So this will be added to parity 2.

As I move forward I can upgrade each parity in a 'step' format from the sounds of it?
For example: In years to come Parity 1 could go to 20TB, then party 2 to 24TB, and so forth?

Yes, and of course, you can reuse parity disk as data when you upsize parity. Just make sure you always keep a current backup of flash.

  • Like 1
Link to comment
10 hours ago, trurl said:

Yes, and of course, you can reuse parity disk as data when you upsize parity. Just make sure you always keep a current backup of flash.

Hello, 
Good stuff.
Yes I do have CA Backup running and do a manual backup every so often.
Appreciate all the help!

Link to comment

Hello again,
Forgot to ask, what's your thoughts on preclearning new OEM disk? I have done it in the past... however, there has been times have skipped the preclear process completely and installed new disk into the array.

Is it best practice to preclear all disk before add to array, or better to preclear older disk, or 'used' disk?

Thanks. 

Edited by bombz
Link to comment
37 minutes ago, bombz said:

Forgot to ask, what's your thoughts on preclearning new OEM disk? I have done it in the past... however, there has been times have skipped the preclear process completely and installed new disk into the array.

 

Either way is fine.  However, I always preclear any new disk because I want to detect any disk that may be DOA or will fail from infant morality.  (Google 'bathtub curve' for what I am talking about.)  In fact, it makes more sense to skip doing it on an older drive if you look at the failure curve for hard drives...)

 

Back in the olden days of Unraid, the array was unavailable while the built-in preclear operation was being done.  With the typical size drive back then that was a hour or so.  As drive capacities increased, so did the time required.  The preclear script was written so that the preclear operation could be done prior to adding the disk to the array.  Doing so reduced the offline time to a very few minutes as only formatting was required to add a new disk to the array.

Link to comment
11 minutes ago, Frank1940 said:

 

Either way is fine.  However, I always preclear any new disk because I want to detect any disk that may be DOA or will fail from infant morality.  (Google 'bathtub curve' for what I am talking about.)  In fact, it makes more sense to skip doing it on an older drive if you look at the failure curve for hard drives...)

 

Back in the olden days of Unraid, the array was unavailable while the built-in preclear operation was being done.  With the typical size drive back then that was a hour or so.  As drive capacities increased, so did the time required.  The preclear script was written so that the preclear operation could be done prior to adding the disk to the array.  Doing so reduced the offline time to a very few minutes as only formatting was required to add a new disk to the array.

Hello, 

Awesome. I always felt preclear was recommended, and was getting into bad habits tossing disk in, without preclearing.
I generally boot up another standalone workstation with the preclear_dish.sh 1.15 script on the USB flash and run it on that system rather then within the array itself, as large disk (last I recall 10TB took 72ish hours) to preclear.

Is my method acceptable using the following preclear.zip?


Once completed it pop the precleared disk into the array and letter rip :-)

Thanks for your feedback, I will read up on bathtub curve
 

Edited by bombz
Link to comment
3 minutes ago, Frank1940 said:

@Joe L.'s preclear script has not been maintained for years.   You should be using either the Plugin or the Docker.  You can find them in the Unraid App store--- the Apps tab of the GUI.  (Personally, I use the preclear Docker BUT I am not afraid of the command line.)

Ah I see
Me and my old school methods (haha)
I see there is an unassigned devices preclear too, that's pretty cool, I could toss the disk in my dock and preclear that way... hm may give that a try?

image.thumb.png.fc9b150b85f6d6118bd546b1ccfe1ee4.png

Link to comment
  • 1 month later...
On 2/27/2022 at 2:08 PM, itimpi said:

Yes - no data drive can be larger than any of the parity drives.   Of course you could later put a larger drive into the parity2 site.

Hello again, 

I misunderstood, in a (2x) party set.
I have (2x) parity implemented right now (14TB & 18TB).
My thought's because the 18TB was there, I could add a 16TB into the data set. However, that's not the case.

Let the swapping begin!

Link to comment

Follow up,

When disks have been previously precleared (Parity or Data) and when swapping them within the array, is best practice to run another preclear on the disk to zero the disk before swapping them to their new location within the same array?

 

image.png.318ca54ad9734f3b96b5ba0d2f3bc0f3.png

Edited by bombz
Link to comment

No.  But I (personally) would always run a non-correcting parity check first before doing any non-essential array non-essential operations changes.  That way if any other disk in the array has any issues that might give a problem, I can address it before I start making any changes to the current array.

Edited by Frank1940
Link to comment
15 minutes ago, Frank1940 said:

No.  But I (personally) would always run a non-correcting parity check first before doing any non-essential array non-essential operations changes.  That way if any other disk in the array has any issues that might give a problem, I can address it before I start making any changes to the current array.

Hello,

OK. I was asking because the original 16TB i was going to put to data, is going to Party1 slot now.
Then that disk from party1 is going to data.
So I should be fine doing the direct swap.

I thought of another thing while I was doing all of this.... What if I wanted to go back to (1X) parity disk rather than (2X) is that a 'simple' process to do?

Link to comment
13 minutes ago, bombz said:

I thought of another thing while I was doing all of this.... What if I wanted to go back to (1X) parity disk rather than (2X) is that a 'simple' process to do?

Easy.   Stop array; unassigned unwanted parity drive: start array

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.