unRAID 6.2 - Screenshot Dual Parity support

[CHBMB]

Sorry, I should have made it clear.

 

What I meant hard drive is getting bigger overtime, 8TB Disk for example and in next few years there will be something like 15-20TB disk.

 

Parity will take forever to complete as disk size getting bigger overtime.

 

Real time parity when you copy the file to the array won't take any longer, assuming the same file size, agreed parity checks will take longer.  But I still don't see how that removes the need for some sort of redundancy...

[BRiT]

Sorry, I should have made it clear.

 

What I meant hard drive is getting bigger overtime, 8TB Disk for example and in next few years there will be something like 15-20TB disk.

 

Parity will take forever to complete as disk size getting bigger overtime.

 

And the loss of a single disk unprotected by dual parity system will be even more catastrophic.

[cybrnook]

Do we have the knowledge already to answer the question of what it will take to enable dual parity? Currently I am in 6.1.6 and will move to 6.2.0 when it comes out just for this feature alone.

 

My question is, I have a 6 x 4TB array using single parity today. I already have my 2nd 4TB blank sitting in it's cage but currently unassigned to the array. When we upgrade to 6.2.0 (assuming through the web interface as I have done on the 6.1 incremental's), will enabling dual parity be as easy as just assigning my spare to the 2nd party location and hitting okay and letting it rebuild. Or is this something that will require a rebuild of the entire array, having to be enabled prior to copying data?

[archedraft]

To my knowledge, there has been no details of how dual parity currently works. I would assume it would be as easy as assigning a new disk to the parity slot.

[garycase]

I'm reasonably confident that all you'll need to do is stop the array; assign the 2nd parity drive; and then Start the array and let it do the new parity sync.    As with adding any other drive to the array, it's a good idea to do a parity check BEFORE you do that, as you want to be sure you don't have any issues before making a change to the array.

 

 

[cybrnook]

Absolutely, I have Party checks scheduled weekly. (Maybe the parity log tracking will be fixed in the next release too, just to throw that out there :-) instead of having to apply hotfixes )

 

But yes, I will have the array in a static state. Meaning no pending changes from cache, no active users, and directly after a full parity scan. (Assuming that we are correct that it just needs to be flipped on)

[HellDiverUK]

Looking forward to dual parity.  Have been for about a year. 

 

Of course, going dual parity means my Fujitsu server is going to be too small - I'll probably migrate back to my Define R4/AMD based machine as it has 8 HDD slots (double the number in the Fujitsu).

[abs0lut.zer0]

if this has been answered i apologize up front.

will the second parity disk count towards the device limit?

 

if you can only afford a basic license then that means only 4 disks for data

[bigjme]

All devices count towards the limit. If there parity, storage, or cache

[SSD]

It it's interesting to note that dual parity's main value is perceived by protecting against dual drive failures, but these have been quite rare. Even a real drive failure in the midst of recovering from another failure is exceedingly uncommon.

 

The more common failure scenarios are with knocking things loose exchanging or adding new drives. And users getting in over their heads trying to recover. Dual parity could provide such users a lot of protection against their own worst enemy.

 

And it doesn't take a loose cable to send an array to the brink of data loss.  New users often shoot themselves in the foot in imaginative ways requiring extraordinary effort to salvage data. Dual parity gives the experts and moderators extra options to help them.

 

Also of note is that single parity detects errors but does not point to the cause. For example, imagine a single parity error. UnRaid assumes the problem is with parity, a good guess but not universally true. A second parity may enable triangulation to point at the disk that is the true cause. Use cases like this are particularly interesting to people involved in the dual parity value discussions in the past.We're not sure how much if any of these types of enhancements would be in 6.2, but dual parity may very well open the door to these options.

 

Based on this, I believe every new user and anybody that uses unRaid as an appliance should install dual parity, even for arrays of 3 or 4 drives. Only users that really understand what's happening under the hood and have the ability to diagnose and do no harm in real world scenarios should even consider single parity. For those classes of users, the number of drives in the array may come into play in making such decisions. But once you get over 6 or 7 drives, everyone should be on dual parity IMO.

 

Before Gary says it, I'll add a special exclusion to those that keep meticulous backups. For such users, dual parity is much less important.

[garycase]

if this has been answered i apologize up front.

will the second parity disk count towards the device limit?

 

if you can only afford a basic license then that means only 4 disks for data

 

As noted, the total device count is 6 for Basic => so yes, if you assign 2 parity drives, you could only have either 4 data drives or 3 data drives and a cache drive.

 

It's hard to imagine, however, that if you can afford to build a fault tolerant server, and to acquire the media to populate it with, that an extra $30 for the Plus key is really an affordability issue.

 

[garycase]

... I believe every new user and anybody that uses unRaid as an appliance should install dual parity, even for arrays of 3 or 4 drives.

 

Agree.  The HUGE improvement in reliability is easily worth the cost of a single extra drive ... and (assuming LT implements the necessary code) it will allow the exact location of a parity error to be known and (on the next check) corrected, without having to always assume it's on the parity disk.

 

 

... Before Gary says it ...

 

Too late 

Agree with Jim => the need for dual parity is far less critical if you have complete backups (as I do).    I'll likely add a 2nd parity drive anyway "just because" -- but it's FAR more important for those who don't have backups to do this.

[trurl]

Since I am not at all superstitious, I will go ahead and say this. I have never had a disk failure in unRAID.

 

The many drive rebuilds I have done have been for upsizing. And I only have 5 data disks. And while I don't have complete backups, I do have backups for everything I consider important.

 

I have a 2nd parity drive ready to go just because I want the experience.

[CHBMB]

I've got a 2nd parity drive ready....

 

I've had drive failures before now, but not in Unraid....

 

Although have replaced "dodgy" but not dead drives...

 

I originally thought we'd seen this early in the New Year, I am now revising my estimate to after CES....

 

[glow=red,2,300]Soon^TM[/glow]

[unevent]

Sorry, I should have made it clear.

 

What I meant hard drive is getting bigger overtime, 8TB Disk for example and in next few years there will be something like 15-20TB disk.

 

Parity will take forever to complete as disk size getting bigger overtime.

 

Real time parity when you copy the file to the array won't take any longer, assuming the same file size, agreed parity checks will take longer.  But I still don't see how that removes the need for some sort of redundancy...

 

Dual parity on one machine or essentially dual parity with two unraid servers (backup) at double the hardware and double the cost....dual parity in one system is rather a no-brainer in my opinion.

[archedraft]

I originally thought we'd seen this early in the New Year, I am now revising my estimate to after CES....

[glow=red,2,300]Soon^TM[/glow]

I'm guessing mid 2016 and 2 (maybe 3) unRAID forum uprisings...

[CHBMB]

I originally thought we'd seen this early in the New Year, I am now revising my estimate to after CES....

[glow=red,2,300]Soon^TM[/glow]

I'm guessing mid 2016 and 2 (maybe 3) unRAID forum uprisings...

 

That actually has made me laugh out loud..... 

[archedraft]

 

I originally thought we'd seen this early in the New Year, I am now revising my estimate to after CES....

[glow=red,2,300]Soon^TM[/glow]

I'm guessing mid 2016 and 2 (maybe 3) unRAID forum uprisings...

 

That actually has made me laugh out loud..... 

 

Haha, that's awesome.

[cybrnook]

The triangulation of potential corruption I think is awesome. As for someone, like myself, who has multiple Servers in the house, my secondary server runs just a basic replication of my unRAID's shares on a weekly basis. So, as of today, the potential for actual file corruption, thus an adjusted parity correction to commit the corruption, to traverse it's way into my secondary backup is always a thought in the back of my mind.

 

I look forward to not having to worry about that.

 

 

As well, I have had a secondary drive failure in the past when rebuilding a RAID5 array from a single disk failure and experienced complete data loss. I do not want to repeat this again. (Was not on unRAID, I discovered unRAID many years after with 4.7)

 

So again, dual disk failure tolerance is greeted with open arms.

[garycase]

Dual parity on one machine or essentially dual parity with two unraid servers (backup) at double the hardware and double the cost....dual parity in one system is rather a no-brainer in my opinion.

 

Two entirely different things.    A 2nd server for backups protects against inadvertent deletions; catastrophic hardware failures on your main server;  a bad virus or malware attack that corrupts the data on your primary server;  etc.

 

Dual parity simply provides a 2nd level of fault tolerance in the event of a failure AND will also allow any parity error to be isolated and corrected more accurately than with the current single parity system -- which always assumes the error is on the parity disk.

 

Dual parity does NOT eliminate the need for backups ... although I concede it reduces the likelihood of loss on the primary server; so if you're among those who don't bother to backup, the reduced risk of data loss is certainly worthwhile.

 

[pfp]

How will dual parity affect the need to spin up disks?  One aspect of unRAID that I really like is that even with a a 24 disk array only 2 disks need to be spinning when I write data to any drive (the drive where the data is saved and parity) and the other 22 disks can just sit there powered down.  Will dual parity add more than 1 additional disk (the second parity) that must be spinning for writes?

 

I'm amusing nothing would change for reads.

[unevent]

Dual parity on one machine or essentially dual parity with two unraid servers (backup) at double the hardware and double the cost....dual parity in one system is rather a no-brainer in my opinion.

 

Two entirely different things.    A 2nd server for backups protects against inadvertent deletions; catastrophic hardware failures on your main server;  a bad virus or malware attack that corrupts the data on your primary server;  etc.

 

Dual parity simply provides a 2nd level of fault tolerance in the event of a failure AND will also allow any parity error to be isolated and corrected more accurately than with the current single parity system -- which always assumes the error is on the parity disk.

 

Dual parity does NOT eliminate the need for backups ... although I concede it reduces the likelihood of loss on the primary server; so if you're among those who don't bother to backup, the reduced risk of data loss is certainly worthwhile.

 

Gary...its not an argument for dual parity replacing the need for backups of critical files...common sense should be at play here for that.  More of a statement of single vs dual and the ability to survive single drive and two drive failures using the fact of two single parity arrays on two machines at potentially 2x the $$cost is the equivalent of dual parity on one for fauly tolerance.  User error of file deletion etc., while should be obvious, is not a feature of dual parity, unless we are including snapshots...then we may have something to talk about.

[garycase]

I'm amusing nothing would change for reads.

 

I presume you mean "assuming"      ... and yes, nothing will change for reads.

 

[danioj]

Just noticed this thread. Given in the screenshot of the unRAID GUI I have a couple of musings ...

 

1) It is clearly in BETA (BETA15-WIP) and not ALPHA. I wonder if we will get it when its in BETA15+n? OR I wonder if its BETA15 (Private) and will be released as BETA1 (Public) OR perhaps there will be no Public BETA. Interesting.

2) Given the second Parity slot is called Parity 2. It seems to be following the same naming convention as the btrfs cache pooling (where we know you can have more than 2 disks). I wonder if that means LT has actually implemented n Parity and not just dual?

 

Anyway, its fun to speculate!

 

[CHBMB]

 

1) It is clearly in BETA (BETA15-WIP) and not ALPHA. I wonder if we will get it when its in BETA15+n? OR I wonder if its BETA15 (Private) and will be released as BETA1 (Public) OR perhaps there will be no Public BETA. Interesting.

 

 

We'll get it soon™