Dual parity protection

[gfjardim]

Forked the Add support for 24 array drives thread:

 

 

My point is that it's more than just changing a constant from "21" to "24" to make this change, and to go beyond 24 is way more changes.

 

Tom, did you investigate the possibility of increase the parity protection to two or maybe more drives? I know this require a major driver rewrite, but since drives become larger and rebuild time increases, this can be as important as increasing the drive limit.

 

This is just my humble opinion.

[Just Me]

I agree. If you really want to use that many hard drives it would be good if unraid could compensate the loss of two disks.

[abs0lut.zer0]

Forked the Add support for 24 array drives thread:

 

 

My point is that it's more than just changing a constant from "21" to "24" to make this change, and to go beyond 24 is way more changes.

 

Tom, did you investigate the possibility of increase the parity protection to two or maybe more drives? I know this require a major driver rewrite, but since drives become larger and rebuild time increases, this can be as important as increasing the drive limit.

 

This is just my humble opinion.

 

+1

 

may i also say please do not let this be a feature only be in the pro version only, i would also like to access this feature in the plus version (obviously it would not count as a data disk so there would still be 5?)

[Isolgrac]

Tom, did you investigate the possibility of increase the parity protection to two or maybe more drives? I know this require a major driver rewrite, but since drives become larger and rebuild time increases, this can be as important as increasing the drive limit.

 

This is just my humble opinion.

 

+1 from me too, if anyone is counting 

[jumperalex]

and if not actual q + p dual parity, then perhaps dual arrays with a fused user share.  Might actually be easier with less resource overhead.  It still wouldn't protect from dual drive failures, but it would turn a single 21 drive array into maybe 2x11 drive arrays in a single box with a single fused file system.  For me, I get nervous with the idea of 20 drives all being protected by a single parity and then hoping nothing goes wrong with the remaining drives during a 3TB or 4TB parity rebuild

 

Change the above numbers to whatever makes the most sense with the "sdX" limits you discussed in the other thread

[ftp222]

Tom, did you investigate the possibility of increase the parity protection to two or maybe more drives? I know this require a major driver rewrite, but since drives become larger and rebuild time increases, this can be as important as increasing the drive limit.

 

This is just my humble opinion.

 

+1 from me on this one too.  Currently at 14 Data drives and a bit nervous to add more without extra protection.  Would LOVE to see this feature added in a future release.

 

 

[Pauven]

+1 me too.  Willing to pay extra for a ParityPlus license option. 

 

Thank you for everything Tom, I'm very happy with 4.7.  Just getting nervous as my 2 yr old array (just checked, 3 years this month!) approaches 20 drives.  In my experience when drives start dropping they often like to go in groups. They're clicky like that...

[chuck23322]

Thanks for the info on that limetech. On that basis, I would personally like to see this come later on after an initial stable 5 release.

 

+1

 

Thanks for your reply Tom. I hope this makes it clear about what is involved in this change and it sounds like it won't be a 5.0/5.1 feature. I do think that long term, support for 4 character identifier strings would mean far larger servers and far more expansion possibilities but is it worth the time involved? Who's to say a server with 25, 30, 35+ drives would even run very well? Parity checks would take forever and the chances of an URE or similar causing parity/corruption issues increase drastically and there would be a need for additional protection, such as double parity.

 

 

My 2 cents.  P+Q parity is needed before we build such huge systems with multiple points of failure.

I agree.

 

22 data + cache + 2 parity + flash = 26 devices.  (and no need to re-write code to handle 4 char device names)

Although I think diagonal parity will probably work better, as I don't think  P+Q would allow any disks to spin down.

 

My reply

 

 

Yea, I don't care about the methodology....  I just remember reading about P+Q here being a "maybe later" feature -- doesn't matter to me if it's diagonal, horizontal, virtual, or pluto versions ------ what I just mean to say is -- some new form of parity that can tolerate TWO drives failing in a system.

 

And my 2 cents is that is needed before we start building 24+ drive systems -- (heck, I think it's needed now for even smaller systems, but that's one guys opinion after being burned in my 18 drive system which I have reduced to 10 drives of higher capacity)

 

 

I want to add one more thing.  Because of my experience trying to narrow down a data corruption issue,  one of my "expectations" in a P+Q (or we'll just call it Dual Parity) is that not only should Unraid tell me that "you have a parity problem" -- but I am expecting it to tell me "*THAT* drive is the cause of the problem".

 

Today, if I get a parity issue -- all I can do is tell it to update parity.  I can't be sure whether or not it was the parity drive, or any one of my data drives at fault.

 

That's an expectation I have

 

...Chuck

[jumperalex]

Is that actually viable even with P + Q parity?  Does the logic actually support knowing if the error resides on P, Q, or D1...Dn?  Or worse, more than one of those?

[chuck23322]

Is that actually viable even with P + Q parity?  Does the logic actually support knowing if the error resides on P, Q, or D1...Dn?  Or worse, more than one of those?

 

I can make up expectations, it's up to the smarter people to tell me "NFW"

 

But I did think the point of P+Q was with a SINGLE drive failure {or being flaky} it COULD tell me which drive is giving me parity errors. 

 

My limited understanding is -- say a 4 drive system -- P+Q + Data1 + Data2 -- when doing a parity check, you have sufficient info (P + Q + Data1, P + Q + Data2, P + Data1 + Data2, etc) -- to compare the various parities against the 4th drive and tell where the failure is. 

 

With a DOUBLE drive failure, I understand why it couldn't (maybe) but still survive a DUAL failure.

[jumperalex]

I suppose i can envision a way to pinpoint the offender using a vertical + diagonal parity.  But I think it would require a full parity scan after an error is detected and I'd still have to think about the logic to see if there are the necceesary and sufficient conditions to say that you KNOW the error resides in the data drive bit or the parity bit.

 

anyway, it is friday at 4pm before a 3day weekend and I need to secure the building.  Have a great weekend