Will there ever be support for multiple parity drives?

[ootuoyetahi]

With my server growing past 15 hard drives, the probability of multiple drive failures is growing as well. While I admit I don't really understand how the technology works, it seems like adding a second parity drive shouldn't be an impossibility. Is there something technical that is preventing unraid from adding a second parity drive or am i looking too far into it and there isnt really a need?

[garycase]

Dual parity is definitely a VERY desirable feature -- but it is far more complex mathematically than a single parity drive.  There are several ways to implement it ... I don't know which one Limetech favors (or even if it's on their near-term enhancement list) -- but it would clearly be an excellent improvement in the reliability of UnRAID.    There have been a LOT of requests for this feature, so it's reasonable to assume that it will be added at some point in the future ... the real question isn't really IF -- but WHEN.

 

The biggest improvement isn't that you can then have two drives fail and not lose any data -- you would still definitely want to replace a drive when ONE failed.  But the really nice feature it would add is that if a second drive was to fail during the rebuild of a failed drive, the rebuild would still successfully complete.

 

As drives get larger, and the # of drives grows, the likelihood of that 2nd failure during a rebuild grows notably larger.  Most modern drives have error rates of 1 in 10^14 bits ... that's one uncorrectable error in 100 trillion bits, or one  error in 12.5TB.    Enterprise drives typically have error rates on the order of 1 in 10^15 bits, so they average one error ever 125TB.    These numbers once seemed so much higher than any storage array you might build that errors seemed very unlikely -- but clearly with modern array sizes that's no longer the case.

 

[itimpi]

I am hoping that when they go down this route they opt for the so-called 'diagonal parity' scheme.  The big advantage of this having read some of the papers on it is that as well as providing extra redundancy it allows the exact drive that has the problem to normally be identified so that it can be corrected on the problem drive 'in flight'.  This is an improvement over the current simpler scheme where a failure can be identified, but not the drive which caused the failure.

 

However, although very desirable I am sure this is a non-trivial enhancement.  It would have to be rock-solid as an errors in the implementation would be very likely to lead to data loss.

[ootuoyetahi]

I guess I should have done a little research on the boards before starting this thread as its been discussed http://lime-technology.com/forum/index.php?topic=34427.15.

 

It seems that this is not something that is going to be implemented anytime soon and has me thinking about possibly switching to FreeNas unfortunately. I also dont like the fact that the argument against dual parity at least in that thread is backups. If backups are the argument against better fault tolerance, why have a parity drive at all? Its like, why lock my house because i have homeowners insurance.

 

I understand backups are important, and my important data is backed up offsite, but it seems crazy to me to argue dual parity isnt a crucial feature because you should be backing up. /rant

[garycase]

A single parity bit can never identify an error location, since it's simply an XOR of all the bits in any given plane.   

 

All dual parity implementations provide a mechanism to identify the specific location of an error, whether diagonal parity, Reed-Solomon, or any of the other techniques.  Doesn't need to be "diagonal parity" to provide this capability.

 

[itimpi]

I do not think that anyone has said that dual parity I not a good idea!  What has been said that backups of important data should be done because regardless of the number of parity disks you have there are things that can go wrong that can lead to data loss.

 

There is also the point that the moment all a parity error does is tell you that something has gone wrong.  It does not tell you which disk it went wrong on, or what file(s) are effected.    That is why if one is going move toward dual parity one does not simply want an additional parity disk using he same algorithm as the current parity disk.  If you look at some of the papers such as this one on the 'diagonal parity' algorithm you see both a discussion of this issue, and how that algorithm helps solve the problem.

[jonp]

I guess I should have done a little research on the boards before starting this thread as its been discussed http://lime-technology.com/forum/index.php?topic=34427.15.

 

It seems that this is not something that is going to be implemented anytime soon and has me thinking about possibly switching to FreeNas unfortunately. I also dont like the fact that the argument against dual parity at least in that thread is backups. If backups are the argument against better fault tolerance, why have a parity drive at all? Its like, why lock my house because i have homeowners insurance.

 

I understand backups are important, and my important data is backed up offsite, but it seems crazy to me to argue dual parity isnt a crucial feature because you should be backing up. /rant

Backups are critical, even with dual parity. How else would you recover data should your system become physically damaged because of fire, flood, or even the more likely scenario of dropping equipment during a move?  What happens if you accidentally delete or overwrite important data?

 

We realize there is a demand for dual parity as a feature to improve fault tolerance, but to consider any level of parity on par with a true backup is a serious oversight.

 

This feature is something we can revisit for discussion to implement at a later time, but it is not as critical of a feature as compared to others that we have implemented in the shorter term that have truly differentiated unRAID from our competitors.

 

I appreciate the passion and if you feel that dual parity will somehow provide you with greater peace of mind that you don't have now, I understand, but with all avenues available to back up data nowadays, I truly hope you will employ one to protect yourself against events that parity protection does not.

[garycase]

...  I also dont like the fact that the argument against dual parity at least in that thread is backups.

 

I don't see anything that says dual parity shouldn't be implemented because of backups.  It's certainly true that NO fault-tolerant system is a substitute for backing up your important data - regardless of how fault-tolerant it is (single parity, dual parity, triple parity, etc.).  Fault-tolerance is to keep your system running ... not to back it up.    And I certainly agree that with the size of modern disks dual parity would be a very nice improvement in the fault tolerance of our UnRAID systems.

 

[ootuoyetahi]

 

Backups are critical, even with dual parity. How else would you recover data should your system become physically damaged because of fire, flood, or even the more likely scenario of dropping equipment during a move?  What happens if you accidentally delete or overwrite important data?

 

My point is that one has nothing to do with the other. They are two different things and backups really shouldnt even be a part of the discussion.

I appreciate the passion and if you feel that dual parity will somehow provide you with greater peace of mind that you don't have now, I understand, but with all avenues available to back up data nowadays, I truly hope you will employ one to protect yourself against events that parity protection does not.

 

As I stated in my original post, I DO back my important data up offsite but that does not mean I dont want greater fault tolerance.

[ootuoyetahi]

...  I also dont like the fact that the argument against dual parity at least in that thread is backups.

 

I don't see anything that says dual parity shouldn't be implemented because of backups.  It's certainly true that NO fault-tolerant system is a substitute for backing up your important data - regardless of how fault-tolerant it is (single parity, dual parity, triple parity, etc.).  Fault-tolerance is to keep your system running ... not to back it up.    And I certainly agree that with the size of modern disks dual parity would be a very nice improvement in the fault tolerance of our UnRAID systems.

 

You're right, no one really comes out and says it but I interpreted all of the do backups talk as its not needed and that really begs the question why are they using unraid if backing up their data is all they need.

[jonp]

 

Backups are critical, even with dual parity. How else would you recover data should your system become physically damaged because of fire, flood, or even the more likely scenario of dropping equipment during a move?  What happens if you accidentally delete or overwrite important data?

 

My point is that one has nothing to do with the other. They are two different things and backups really shouldnt even be a part of the discussion.

The reason they are part of the discussion and always come up is because backups are a solution for those worried about multiple disk failures. Backups don't provide fault tolerance but they do prevent data loss. When so many viable solutions are available and the likelihood of multiple concurrent disk failures is so low, backups are a viable alternative to dual parity for data protection.  The is especially true in atypical unRAID setups where the majority of the data stored is WORM (write once, read many; I'm referring to media files here).

 

The bottom line here is that dual parity is an insurance policy to prevent service interruption from your NAS in the very unlikely even of multiple disk failures occurring before you have had a chance to replace and rebuild the first disk that fails.

 

Edit:  I should add that this isn't something we are writing off as a useless feature, we just have an order of priorities of things to implement and this isn't at the top of the list right now.

[trurl]

Another reason the subject of backups comes up in some of these discussions is to make sure all users understand the importance of backups. I have certainly seen posts from users that have a problem, have no backups, and assume unRAID has their butt covered.

[ootuoyetahi]

 

Edit:  I should add that this isn't something we are writing off as a useless feature, we just have an order of priorities of things to implement and this isn't at the top of the list right now.

 

This is somewhat surprising because of why people use unRaid. One of the first features mentioned on the what is unRAID page is that it creates an array of discs without raid but with parity protection. I would have thought that improving upon the core feature of the OS would be much closer to the top of this list. I will accept your answer although it does surprise me a bit.

 

Also I was/am not trying to create WWIII simply get a little bit more information.

 

 

[ootuoyetahi]

Another reason the subject of backups comes up in some of these discussions is to make sure all users understand the importance of backups. I have certainly seen posts from users that have a problem, have no backups, and assume unRAID has their butt covered.

 

I understand and it is worthy to mention but I dont think it needs to take over the discussion.

[jonp]

 

Edit:  I should add that this isn't something we are writing off as a useless feature, we just have an order of priorities of things to implement and this isn't at the top of the list right now.

 

This is somewhat surprising because of why people use unRaid. One of the first features mentioned on the what is unRAID page is that it creates an array of discs without raid but with parity protection. I would have thought that improving upon the core feature of the OS would be much closer to the top of this list. I will accept your answer although it does surprise me a bit.

 

Also I was/am not trying to create WWIII simply get a little bit more information.

I didn't think this was a WW2 thread at all. You raise good points and voicing your opinion on these forums is the ONLY way we have to measure the desire from our users for certain features.

 

Improving on our core features is what we always do. We have improved support for further filesystems, more applications, more use cases, and we've completely redone our management interface. We've added system notifications, cache pool support, and have upgraded the entire operating system to a 64 bit platform.  Just recently we baked in support for UPS shutdown, an arguably more important feature than dual parity as well.

 

Dual parity is a feature that will take some effort to code into Unraid and make it easy for folks with existing systems to utilize. We could easily have spent the majority of our time on this one feature in exchange for all the others, but it wouldn't have gained us enough in the competitive sense.

 

Hope this explains it!

 

[SSD]

Just wanted to chime in here. I think dual parity is a core NAS feature supported by most competing products / technologies. I think it should be very high on the priority list.

 

I agree that users should back up data they can't afford to lose - normally unique works like family pictures, home movies, and correspondences/financial records. But arrays also contain vast amounts of "recoverable in other ways" data and the cost and diligence to back up every file is not justified to the user. So realistically, users are making decisions about what to back up and what they are willing to lose / recover in other ways, in the event of a calamity. By investing in a NAS like unRaid, users have an economic form of redundancy to avoid the time, effort and possibly cost to re-acquire and assemble their "recoverable in other ways" data for drive failure - a common and foreseeable problem. These do not protect from unforeseeable events like acts of God or fire - things likely having them looking for a new home (for which they have no backup) also. Users are not tolerant of losing data due to insufficient redundancy options in their NAS, which they bought to explicitly protect them from drive failures!

[WeeboTech]

Just wanted to chime in here. I think dual parity is a core NAS feature supported by most competing products / technologies. I think it should be very high on the priority list.

 

 

Agreed, Sometimes I think, why is my NAS becoming a virtualization platform? I need it to protect my data.

 

 

How do people feel about snapraid on top of unRAID as an added level of protection?

Last time I tried, I was able to compile it in a slackware dev environment.

[JonathanM]

How do people feel about snapraid on top of unRAID as an added level of protection?

Last time I tried, I was able to compile it in a slackware dev environment.

I would love to see a proof of concept and some use examples. Would you be able to run it against selected user shares instead of selecting specific drives to protect?

[WeeboTech]

How do people feel about snapraid on top of unRAID as an added level of protection?

Last time I tried, I was able to compile it in a slackware dev environment.

I would love to see a proof of concept and some use examples. Would you be able to run it against selected user shares instead of selecting specific drives to protect?

 

I think it requires the parity drive to be 'slightly' larger then volume you want to protect.

Which means, you cannot fill all your drives up to 100%, drives would need to leave some spare space.

 

It would be an interesting exercise to dump the super.dat file and build a snapraid configuration section of the data drives leaving the only manual configuration being the new parity drive you want to add.

 

From what I remembered reading, snapraid can also function as an undelete until you manually resync the snapraid parity.

I.e. you can restore individual files.  Although, I'm going from some forum reading I did a few months ago so I 'm not sure how it's implemented.

[Squid]

How do people feel about snapraid on top of unRAID as an added level of protection?

Last time I tried, I was able to compile it in a slackware dev environment.

I would love to see a proof of concept and some use examples. Would you be able to run it against selected user shares instead of selecting specific drives to protect?

 

I think it requires the parity drive to be 'slightly' larger then volume you want to protect.

Which means, you cannot fill all your drives up to 100%, drives would need to leave some spare space.

 

It would be an interesting exercise to dump the super.dat file and build a snapraid configuration section of the data drives leaving the only manual configuration being the new parity drive you want to add.

 

From what I remembered reading, snapraid can also function as an undelete until you manually resync the snapraid parity.

I.e. you can restore individual files.  Although, I'm going from some forum reading I did a few months ago so I 'm not sure how it's implemented.

But would using a single parity drive via snapraid actually gain you anything?  Wouldn't it be more or less the same as what we have now?  Inorder to get dual parity in combination with snapraid wouldn't you have to dedicate 2 drives to snap raid, and 1 for unraid (so that you still have the virtual drive in the event of a single failure).

 

If you use snap raid for dual parity without unRaid's parity drive, you might as well forgo unRaid and just do a full *nix install and run snapraid alone.

[WeeboTech]

How do people feel about snapraid on top of unRAID as an added level of protection?

Last time I tried, I was able to compile it in a slackware dev environment.

I would love to see a proof of concept and some use examples. Would you be able to run it against selected user shares instead of selecting specific drives to protect?

 

I think it requires the parity drive to be 'slightly' larger then volume you want to protect.

Which means, you cannot fill all your drives up to 100%, drives would need to leave some spare space.

 

It would be an interesting exercise to dump the super.dat file and build a snapraid configuration section of the data drives leaving the only manual configuration being the new parity drive you want to add.

 

From what I remembered reading, snapraid can also function as an undelete until you manually resync the snapraid parity.

I.e. you can restore individual files.  Although, I'm going from some forum reading I did a few months ago so I 'm not sure how it's implemented.

But would using a single parity drive via snapraid actually gain you anything?  Wouldn't it be more or less the same as what we have now?  Inorder to get dual parity in combination with snapraid wouldn't you have to dedicate 2 drives to snap raid, and 1 for unraid (so that you still have the virtual drive in the event of a single failure).

 

Probably. I haven't really tried it out yet.

Then again, if unRAID is the backup for some people, it might be important enough to put in the extra drives.

 

I was considering the use of a multi drive jbod box as a added layer of protection.

I believe with snapraid you can detect silent corruption of the data (if it has not been synced).  That was my original plan.

 

If you use snap raid for dual parity without unRaid's parity drive, you might as well forgo unRaid and just do a full *nix install and run snapraid alone.

 

I'm seeing this becoming a pattern.

What you loose is the live updating of parity. user shares with split levels, spiffy dynamix interface and virtualization management from gui.

[jonp]

Agreed, Sometimes I think, why is my NAS becoming a virtualization platform? I need it to protect my data.

 

The idea of the NAS becoming a virtualization platform is simple:  it allows folks to extract more value out of their investment in hardware while not sacrificing data protection.  Traditional NAS solutions that purely store, serve, and protect your data are all dead.  All the players have employed methods of supporting 3rd party applications on their system which frankly, even with chroot, are more limiting and add more complexity than any of the virtualization technologies we've employed (Docker/VMs).

 

In addition, many folks nowadays are becoming more and more budget-conscious when it comes to their spend on x86 hardware.  Having a PC, HTPC, NAS, App Server, etc. is just not financially reasonable and if you could consolidate even just some of that into a single appliance, why wouldn't you?  You could have a machine that is more powerful / capable than any of those individual assets and spend less total money than you would to keep those as disparate systems.

 

When > 90% of our user base (per this poll) tells us they are using plugins to install applications on their NAS, solving applications supportability became a priority over adding dual parity.  Why?  Because dual parity, as a feature, doesn't affect 90% of the user base.  The biggest reason folks want it is for larger sized arrays where the risk is greater for multiple disk failures.  That doesn't mean this isn't important, it just means that dual parity, as a feature, is less desirable than better support for applications.

 

For these reasons, we put time, effort, and energy into the area that affects the majority of users and delivers value in ways the differentiate us from our competition (instead of just playing catch-up so we can say, "yeah, we got dual parity as well.")  This is more than just virtualization.  It's multiple file system support, UPS support, notifications, SMART data reporting, etc. 

 

This really isn't a response to "why no dual parity" as much as defending the reasons we haven't prioritized it over other features.  Anyhow, I'm going to leave this thread be for now and as I mentioned before, we will revisit this topic at a later time.

[WeeboTech]

Agreed, Sometimes I think, why is my NAS becoming a virtualization platform? I need it to protect my data.

 

The idea of the NAS becoming a virtualization platform is simple:  it allows folks to extract more value out of their investment in hardware while not sacrificing data protection.  Traditional NAS solutions that purely store, serve, and protect your data are all dead.  All the players have employed methods of supporting 3rd party applications on their system which frankly, even with chroot, are more limiting and add more complexity than any of the virtualization technologies we've employed (Docker/VMs).

 

In addition, many folks nowadays are becoming more and more budget-conscious when it comes to their spend on x86 hardware.  Having a PC, HTPC, NAS, App Server, etc. is just not financially reasonable and if you could consolidate even just some of that into a single appliance, why wouldn't you?  You could have a machine that is more powerful / capable than any of those individual assets and spend less total money than you would to keep those as disparate systems.

 

When > 90% of our user base (per this poll) tells us they are using plugins to install applications on their NAS, solving applications supportability became a priority over adding dual parity.  Why?  Because dual parity, as a feature, doesn't affect 90% of the user base.  The biggest reason folks want it is for larger sized arrays where the risk is greater for multiple disk failures.  That doesn't mean this isn't important, it just means that dual parity, as a feature, is less desirable than better support for applications.

 

For these reasons, we put time, effort, and energy into the area that affects the majority of users and delivers value in ways the differentiate us from our competition (instead of just playing catch-up so we can say, "yeah, we got dual parity as well.")  This is more than just virtualization.  It's multiple file system support, UPS support, notifications, SMART data reporting, etc. 

 

This really isn't a response to "why no dual parity" as much as defending the reasons we haven't prioritized it over other features.  Anyhow, I'm going to leave this thread be for now and as I mentioned before, we will revisit this topic at a later time.

 

I would bet many of those plugins were just to deal with shortcomings in unRAID itself.

No email support, no ups support, no graceful power down support, no monitoring, etc, etc.

People would get denied requests to add basic unix support programs.

 

Really, I don't want to get into a pissing contest about it, but the poll was about plugins and not feature requests.

 

Besides the most basic NAS functionality that should have been built in, the most often requested feature I've seen (even in years before the extra presence) were dual parity.

 

I'm not an opponent to docker or VM's.

 

My comment was one one in passing. 

I have it in my mind every so often when I look at how long it's taken to get protection of data vs running adhoc applications.

 

I think unRAID is moving in a good direction.

Yet, When the execution and management of adhoc applications is a higher priority vs protecting data I question it.

This was one of those times when I voiced it.  So be it. Shoot me. 

[garycase]

I would bet many of those plugins were just to deal with shortcomings in unRAID itself.

No email support, no ups support, no graceful power down support, no monitoring, etc, etc.

People would get denied requests to add basic unix support programs.

 

... and I suspect you'd win that bet 

 

I agree a lot of folks still use UnRAID primarily as a NAS.    I echo your earlier comment:  "... Sometimes I think, why is my NAS becoming a virtualization platform? I need it to protect my data."    While a good VM manager and hypervisor support are indeed attractive features, given a choice between those and dual parity I'd take dual parity in a heartbeat.    There are plenty of other ways to support virtualization -- VMware (either Workstation or a dedicated ESXi box), Hyper-V, etc. 

 

 

Besides the most basic NAS functionality that should have been built in, the most often requested feature ... dual parity. "

 

Ditto.    As a NAS, dual parity is by far the most significant enhancement that could be added.

 

[WeeboTech]

In the meantime, I think we're in the home stretch of the current release, which opens the next door of possibility.

 

So, does anyone think snapraid on unRAID is a worthy option for the time being, if not, I won't go any further with discussing it as a temporary option.