It came up in late 2008, that disk quantity would be increased and a second parity drive similar to RAID-6 was going to be introduced as features to Pro. I'm currently a Plus user but would upgrade in 1 second if it meant getting what was referred to as Q-Parity.

 

Edit: This - http://lime-technology.com/forum/index.php?topic=2634.msg21695#msg21695

Good find!

 

 

Agreed. I'd upgrade from Plus to Pro immediately for an extra layer of redundancy.  (I know it won't replace my backups, but it would still be nice.)

 

Kevin

I also agree that this would be a great addition to unRaid.

 

I was reading up a bit try to see if there is any reason it wouldn't work for unraid, but couldn't find any.  All you would need is two parity drives >= the largest data drive, and a CPU powerful enough to handle the more complex q-parity calculation -- though even some of the slowest CPUs we're currently using would probably be up to the task.

 

Hopefully something we'll see in Unraid 5.

 

Also note that in addition to allowing recovery of up to two drives, Q-parity would add the ability to detect corruption (different from failure) of data on a single disk, making the "check parity" feature all that more useful.

Q-parity would add the ability to detect corruption (different from failure) of data on a single disk, making the "check parity" feature all that more useful.

Now that alone would be a very good reason for me to upgrade!

 

This is something I was just about to inquire about.  I particularly am interested in this for building a very large array.

 

 

Hoping Tom can look into this for the 5.x series. It would be great for bigger arrays to have this option. When dealing with enterprise-grade arrays, I always go the raid6/hot spare route if over 8 drives because you really can never tell. With consumer-grade drives, the possibility of having 2 simultaneous drive failures is significantly increased as you add more and more drives.

 

To be frank, a 19 drive unRAID setup with only 1 parity drive scares the jeebus out of me. Add that to the fact that as of 4.5.3, there's no "official" notification scheme in place in case a drive dies and you have the mixings of a potentially serious issue on your hands.

 

Sure there's unraid_notify and a few other 3rd party scripts developed by our wonderous forum member, but I think these safeguards need to be in place in an official means. Q-Parity would go a long way to helping this situation.. along with official notifications.

 

I concur!

 

This would be a fantastic addition to the product. 19 drives with only 1 parity would scare the hell out of me too. I think as HDD's get larger the possibility of a 2nd drive failure during the parity-sync phase will become increasing likely.

+1

vote for this feature!

Is it possible to calculate q-parity only by one reading from the parity drives and the single data drive being written to (as unraid currently does), or would it require re-reading data from all other drives in the array?

 

Another idea: hardware mirror of the parity drive? Or software mirror for those running full slackware distros? Would this work with unRAID as it is implemented today?

Another idea: hardware mirror of the parity drive? Or software mirror for those running full slackware distros? Would this work with unRAID as it is implemented today?

To help you would need to mirror every drive.

 

The parity drive is no more important than any other disk in your server when you have a drive failure. 

You can mirror the parity drive, and have data disk1 and data disk2 fail.  You've just lost the data on both disk1 and disk2 and cannot rebuild.  The mirrored parity did nothing to help.

 

Joe L.

The parity drive is no more important than any other disk in your server when you have a drive failure. 

You can mirror the parity drive, and have data disk1 and data disk2 fail.  You've just lost the data on both disk1 and disk2 and cannot rebuild.  The mirrored parity did nothing to help.

 

Joe L.

 

What is needed for total protection at the current time is RAID1 on cache, but this can be done with some hardware devices.

 

RAID1 on parity does not add any protection. As mentioned it is only incrementally more important then the other drives. 

 

A cache or ware spare drive equal to parity in the server would be of more use as it would be immediately available to replace a drive should a failure occur.

What is needed for total protection at the current time is RAID1 on cache, but this can be done with some hardware devices.

 

For me, q-parity would be a higher priority than a protected cache drive.  Critical data can always be written directly to the protected array, avoiding cache.  Having q-parity would add an extra layer of comfort for the times when I'm out of town -- but if q-parity requires all drives to be spun up for writes, I may not be that keen on it for my media storage server.  I would have to think that it's possible to re-calculate the q-parity without re-reading data from all the other drives (just as you can with parity), but I can find no example of it.

 

For me, q-parity would be a higher priority than a protected cache drive.  Critical data can always be written directly to the protected array, avoiding cache.  Having q-parity would add an extra layer of....

 

Frankly, I would prefer to see q-parity at a higher priority also. As arrays get larger and larger, the ability to survive the loss of 2 disks is a big plus.

 

My comment is really based on total protection and a point where your data is vulnerable. I.E. if you move allot of data to cache drive and it fails before it is moved off that drive. Again, chances of this occuring are small if you are using recent drives... But it's still a period of time when your data is unprotected.

 

So.... We are in agreement with Q-parity.

Adding my +1 on this as well.

 

To be frank, a 19 drive unRAID setup with only 1 parity drive scares the jeebus out of me.

 

It concerns me too... I also run a parity check at least every month just to sanity check the drives, plus I've actually changed my case to an Antec Twelve Hundred since I wasn't happy with the average drive temperatures (I'm at 8 total disks, including parity).

 

Before I get to 19 drives, I'm more likely to consider a second server based on some kind of risk evaluation, but I'm hoping that Lime's (Tom's) view is not "let them buy a second server" as a solution to this.

Before I get to 19 drives, I'm more likely to consider a second server based on some kind of risk evaluation...

 

Interesting, why this?

To be frank, a 19 drive unRAID setup with only 1 parity drive scares the jeebus out of me.

 

It concerns me too... I also run a parity check at least every month just to sanity check the drives, plus I've actually changed my case to an Antec Twelve Hundred since I wasn't happy with the average drive temperatures (I'm at 8 total disks, including parity).

 

 

Y'all are actually in a better place then others I've seen with massive arrays.

I see some people on the avsforum boasting these massive servers with umpteen drives.

And here you have access to some low level drive information, the ability to run scheduled scans and if you loose a drive, identify and swap another pretty easily.

Worse possible case is loosing two drives, and you still have a huge amount of your data still available.

Y'all in a pretty good position to take care of your array.

To be frank, a 19 drive unRAID setup with only 1 parity drive scares the jeebus out of me.

 

It concerns me too... I also run a parity check at least every month just to sanity check the drives, plus I've actually changed my case to an Antec Twelve Hundred since I wasn't happy with the average drive temperatures (I'm at 8 total disks, including parity).

 

 

Y'all are actually in a better place then others I've seen with massive arrays.

I see some people on the avsforum boasting these massive servers with umpteen drives.

And here you have access to some low level drive information, the ability to run scheduled scans and if you loose a drive, identify and swap another pretty easily.

Worse possible case is loosing two drives, and you still have a huge amount of your data still available.

Y'all in a pretty good position to take care of your array.

And, the more drives you have in your array, the less (percentage wise) you lose if you do have two disks that fail concurrently   ;D  

 

If you only have two disks in your array, you lose 100% of the data  , if you have 20 disks, you only lose 10%.

 

Joe L.

If you only have two disks in your array, you lose 100% of the data  , if you have 20 disks, you only lose 10%.

 

For me it's not so much about the data lost, but the time lost having to deal with sorting it all out -- figuring out what I lost, recovering what's important from other backups, etc.  Q-parity certainly could save some time in the event of a two disk failure.

 

If you only have two disks in your array, you lose 100% of the data  , if you have 20 disks, you only lose 10%.

 

For me it's not so much about the data lost, but the time lost having to deal with sorting it all out -- figuring out what I lost, recovering what's important from other backups, etc.  Q-parity certainly could save some time in the event of a two disk failure.

 

 

I will eventually be working on another addon that will index all of the disks and files and create md5sums of everything.

Sort of like tripwire.

This will allow people to see what is missing, or corrupt.

After that I'll investigate using par2 for verification and/or rebuilding of corrupt files.

"what happened" to maaaaany things...

 

 

I will eventually be working on another addon that will index all of the disks and files and create md5sums of everything.

Sort of like tripwire.

This will allow people to see what is missing, or corrupt.

After that I'll investigate using par2 for verification and/or rebuilding of corrupt files.

 

Wow, WeeboTech, that is really cool!

I will eventually be working on another addon that will index all of the disks and files and create md5sums of everything.

Sort of like tripwire.

This will allow people to see what is missing, or corrupt.

After that I'll investigate using par2 for verification and/or rebuilding of corrupt files.

 

This sounds awesome... and truly paranoid.

I will eventually be working on another addon that will index all of the disks and files and create md5sums of everything.

Sort of like tripwire.

This will allow people to see what is missing, or corrupt.

After that I'll investigate using par2 for verification and/or rebuilding of corrupt files.

 

This sounds awesome... and truly paranoid.

 

I'm not that paranoid. heh... I just decided to gather a few applications I have into one.

 

I use locate/mlocate to index the whole system. Makes it fast to find that file you are looking for.

Using that idea, I can scan the whole array, add in md5sums.

 

Now if a drive dies and you rebuild it, you can compare the md5sums to make sure nothing was corrupt.

For people who "do something wrong" and have to do rebuild-tree. You can do a compare to make sure nothing was corrupt.

 

If this works well. then I may add in a par2 directory structure to add in another layer of protection against corruption. So not only will it detect it, It might be able to repair it.

 

I know raid is not a backup solution. But my unRAID is my backup solution.

Tapes are just not feasible any more.