Request: Broad explanation of the Mathematics behind dual parity

[Anon]

Hello,

 

TLDR: I want to know (in broad terms) how exactly Unraids implementation of Parity2 works.

 

the way single parity works is very easy to understand.

When it comes to dual parity there are a lot of forum posts, reddit posts, youtube videos, the Unraid Wiki itself talking about "reed Solomon code" or "p+q parity" etc. I understand the practical effects of the second parity that Unraid provides and I generally understand the mentioned parity algorithms. But those concepts don't really work with how Unraid is actually implementing them.

 

Unraid keeps "Parity 1" the same so it still calculates the Odd / Even situation. Parity 2 is somehow able to recreate one disk by itself (if the other parity and one drive fails) and also recreate 2 drives together with  the information parity 1 can provide.

Is there some sort of Search term for the algorithm used that I can use to find out more? All the "normal" dual parity mathematics always assume both parity drives to work the same with their algorithm and do not work with 2 different types of parity. Or spread the parity data between the drives (Like Raid 6) which is also not what unraid does.

 

Is anyone well versed enough to point me in the right direction? Or is this a totally new algorithm Unraid developed (modified) and is keeping as a "Trade secret"? I am currently finishing up my master degree in this type of topic so I guess its mainly personal interest that is driving me to actually understand how this works mathematically.

 

In a dual failure scenario parity 1 can say if the 2 values of the added disks together should be odd or even. (if even either 0,0 or 1,1, if odd either 0,1 or 1,0). The second parity's "Magic" needs to find out which of those 2 options it is based on what it has stored itself.

 

(I hope this category is correct. Its not really "Support" and more a question about understanding but I just copied the category from this post that also touched upon dual parity (sadly they never explain how its actually done):

[itimpi]

Not quite sure what the confusion is?

 

Parity1 can calculate the correct bits for 1 failed disk.  Parity2 can do the same for a second drive using a different algorithm (ignoring parity1).   As such they are independent of each other.

 

if you want the exact details of the parity2 algorithm then I have no idea other than the fact it includes the drive slot as one of the elements and that the algorithm is such that a 0 byte on a data drive does not change the calculated result

[trurl]

3 minutes ago, itimpi said:

Parity1 can calculate the correct bits for 2 1 failed disk

FTFY

[Anon]

@itimpi -> The confusion is on how exactly it mathematically works.

There is no confusion in how it works from a outside perspective and what it means for operating Unraid.

For a normal End User this information is completely irrelevant but I personally would like to dive deeper into how it works on a operation by operation level.

I don't just want to know that it works I wanna know why and how exactly it works.

 

I spent a few hours trying to find this info but it seems like nobody has ever actually answered in detail and just referred to "generic" algorithms on how a dual parity works in other systems.

 

I'd be willing to put up a small bounty if someone can actually answer me this. (As long as its allowed to offer such a bounty in this forum)

 

Edited February 21, 2023 by Anon

[itimpi]

Have you tried looking at the source code for the Unraid md driver as it is included with Unraid. I  would not be surprised if the calculations are in there somewhere (although I admit I have not looked)

[trurl]

https://en.wikipedia.org/wiki/Standard_RAID_levels#General_parity_system

[Anon]

Hi, thanks for the link @trurl this explains it perfectly (at least at a first quick skim). And somehow I didn't find that when searching earlier. Feel stupid now. Especially since it's a easy to find Wikipedia Link...

 

Thanks a ton!

 

Edited February 21, 2023 by Anon