I've been trying to figure this one out for a while. Can anyone actually explain how the UnRAID parity works. IE some of the typical RAID set-up's you might have 4 drives with 1 being parity for a 25% loss. But the parity could cover multiple drive failures.

 

With UnRAID as I understand it you need a parity for every 11 data drives. So for 12 drives 1 drive is parity. If one fails you're ok. If two fail you "might" be ok. Why might? What happens if your parity is twice the size of the other drives? Are you for sure covered with a two drive failure? What happens when you've got 15 drives like is the current "pre-built" systems? If one drive goes down do you loose data or do you have to have two parity drives.

 

Is there a detailed description somewhere that I've missed?

 

Thanks all.

 

 

A single parity drive works with an unlimited number of data drives, the limitation of Unraid is in the software itself and is *somewhat* arbitrary.  Of course, if the parity drive served, let's say 1,000,000 drives, it wouldn't offer much help because the odds of multiple drives failing would be high.

 

Parity works using binary math: 1+0=1, 0+0=0, 1+1=0 (10, actually, but it lops off all but the last digit).  By looking at the first bit of every drive and adding them together, the answer would be 0 or 1.  Let's say it is 1.  If a drive fails, the logic says, look at the sum of the first bit off of the rest of the drives and compare it to the parity bit.  If they match, the missing bit was a 0, if it doesn't match, it was a 1.

 

This only works for a single bad drive, thus why multiple drive failures are bad.  However, in Unraid, if you lose N drives at once, you only lose N (or N-1 if one of the bad drives was the parity drive) and the rest of the drives are still good.

 

 

Bill

 

Addition: The unintuitive nature of all of this can be summarized in the question,"how can you store 10+ drives of info on a single drive to be redundant?"  The answer is, you can't.  What you can do is use a single additional drive to profile the other drives so you can figure out what is missing if one drops off.

 

Addition2: I realized I did not explicitly answer all of your questions.  If you lose two drives you WILL lose at least one drive worth of data.  The "at least" or "maybe" or "might" is because if one of the bad drives is the parity drive, that causes you no data loss.  In most other methods of RAID where striping is employed (single file spread across multiple drives for the primary reason of enhanced read/write performance), a two drive loss guarantees the entire array is gone UNLESS you go through some painful data reconstruction process - something both expensive and time consuming.  Most of us would rather rerip all of our sources.

 

Understand the math but still quite confused as to exactly how it's working. Though it sounds like no matter how many drives you have you have the same protection provided only one drive fails, GREAT! Very little waste. Likewise though it sounds like again no matter how many drives if two drives fail you're doing to loose both drives, NOT so great?

 

Wouldn't there be a future possible option for better redundancy in case of multiple drive failures. IE either with a larger parity drive or two parity drives? Otherwise couldn't a mirrored systems be done with say two eight-bay external cases and a server configured as a NAS? According to my figures the mirrored system would cost about the same (or slightly more) while using similar hardware.

 

 

 

 

 

 

The problem with schemes that protect from multiple drive failures is that they have a far worse ROI.  The odds that two drives failing is (theoretically) the square of the odds a single drive failing, so protecting against it can be relatively expensive.  Dual parity drives protect (I believe) only the parity drive.

 

If you are paranoid like me, you would employ some sort of mirroring.  I personally use multiple layers of redundancy and backup, from spreading key files across the main hard drives in my house (i.e. some of my wife's key files are backed up (and hidden) on the hard drive of my kids' machine), separate network drives, the Unraid, and Mozy.  No, I don't have true backup for my DVD collection, but then again I have the DVDs.

 

At some point, I will probably build a second Unraid as a simple mirror, using the castaway drives from my main Unraid's upgrade path.

 

 

Bill

Understand the math but still quite confused as to exactly how it's working. Though it sounds like no matter how many drives you have you have the same protection provided only one drive fails, GREAT! Very little waste. Likewise though it sounds like again no matter how many drives if two drives fail you're doing to loose both drives, NOT so great?

 

Wouldn't there be a future possible option for better redundancy in case of multiple drive failures. IE either with a larger parity drive or two parity drives? Otherwise couldn't a mirrored systems be done with say two eight-bay external cases and a server configured as a NAS? According to my figures the mirrored system would cost about the same (or slightly more) while using similar hardware.

One point you have possibly not considered is that when a drive fails (and eventually one will) it can be replaced with a working drive.  Once that is done, the original contents are recreated from the parity drive in combination with the data on all the remaining drives, and then written to the replacement drive.  Once that recovery process completes, you have no drive failures and are back running normally. Data from another drive failing subsequently can again be easily recovered.

 

With commercial arrays you will see mirroring of data (multiple exact copies) in combination with parity.  This is overkill for a home media server as already mentioned.  An unraid system with two drives (one parity and one data) is equivalent to a mirrored drive server.  In fact, in that case, since even parity is stored, the parity drive will hold an exact copy of the data drive.

 

In a true mirrored system, to have 11 data drives you would have 11 more drives mirroring them. (22 drives) in a raid array, only one drive is needed to store the parity calculation bits. The cost is far lower with half of the disk drives.  Additionally, with more drives spinning, there are more drives to eventually fail. Mirror based arrays have a higher disk failure rate, simply because they have more disks.

 

For critical data, write your files to DVDs, store them off site.  Or, write them to a spare hard disk, store it off site.  A RAID array is not a substitute for backup copies of your data.  It does not protect you from your own stupidity. If you overwrite a critical file, odds are it is gone. 

 

A raid array is a way to protect yourself from disk drive failures, nothing more, and only if you replace the failing drives as soon as possible after they fail.  The whole idea is to not allow the hardware to degrade to where two drives have failed at the same time. If a single drive fails, replace it as soon as possible. You might not get the best sale price, but at that point in time, your data is more important than a few dollars savings.

 

Joe L.

Actually I am aware that as long as two drives don't fail at the same time you're good as gold and that they can be swapped out as they fail. I was more concerned about two drives failing at the same time. I do understand that the odds might be low so as to be a none issue. But prefer to err on the side of more protection. Of course I'm trying to get best protection for the dollar like are many which is why UnRAID is such a good option.

 

The point of looking at the mirrored set-up was that while extra drives IE say 20 (2 sets of 10 drives -mirrored) plus cases, server, all parts seems to compare in price to a 11 drive (10 data + 1 parity- UnRAID) system. Obviously there would be more hardware with the mirrored system, more complexity, and potentially more that could fail. However the flipside is that all 10 drives would have to fail for any data loss. As long as those 10 held different set's of data. Now that sounds like good odds.

 

Given the same or similar pricing I'm just trying to convince myself of the advantage of the 11 drive UnRAID over the 20 drive mirrored system and having trouble seeing it other then simplicity. Any help?

 

Actually, with twice as many drives, your chance of a single drive failing is increased dramaticaly in any given period of time.  Also,

since if any two of the same drive in a mirrored set fails, you lose the data from that pair.  That is, in my opinion, no better than a standard RAID with a parity disk.  You too lose data when two drives fail... not when 10 drives fail.

 

If your data is that important to you, then mirrored, plus parity is what you need.  Even with that, all it takes is one good flood, or fire, or lightning hit, and your mirrored drives are all toast. You need a duplicate data center for your physical server, in a different location, on a different network.  It is a game of odds.  In my house, if I know I have a failed drive, I can shut down my server, order a new disk drive, and a few days later, after swapping out the drive, play files from the network.  In the interim, I can go to the shelf, pick a DVD, and play it.  If I am in the middle of a movie being served from the server, and a disk fails, I'll finish the move, wait till guests go home, and then see what happened.  Odds are high I won't even know the failure occurred till I check my mail and see the alert it sent me.

 

If you are a major department store, or a hospital, or NASA, and you must have access to your records to manage sales, or inventory, or diagnosis, or safety, then mirroring and redundant hot spare systems are in order. Those are not the unRaid product and not its target market. 

 

 

Very well stated, Bill and Joe.  I wish I could write as clearly and concisely.

 

I wish I could write as clearly and concisely.

Your response looks very clear and very concise to me.

Just one question,

 

if i have a fail drive, when i change it, what is the procedure to recover my data ?

Just one question,

 

if i have a fail drive, when i change it, what is the procedure to recover my data ?

 

Per the online documentation:

 

"  1. Stop the array.

  2. Power down the unit.

  3. Replace the failed hard disk with a new one.

  4. Power up the unit.

  5. Start the array.

"

 

Bill

It does not protect you from your own stupidity. If you overwrite a critical file, odds are it is gone. 

 

With this in mind, could the future hold some sort of "shadowcopy" scheme where the UnRAID system could help protect my users from their own stupidity?

 

That and GUI based Rsync could my my like sort of " Utipian like" 

... where the UnRAID system could help protect my users from their own stupidity

 

Wasn't there a song about 'dreaming the impossible dream'?

 

You can't protect them completely, but you can 'manage the damage', by setting up your own versioning.  Determine what data is important enough, and even a rudimentary 'grandfathering' scheme of aged backups can be a very good safety net, and is managed by you and not them.

 

Actually I am aware that as long as two drives don't fail at the same time you're good as gold and that they can be swapped out as they fail. I was more concerned about two drives failing at the same time. I do understand that the odds might be low so as to be a none issue. But prefer to err on the side of more protection. Of course I'm trying to get best protection for the dollar like are many which is why UnRAID is such a good option.

 

The point of looking at the mirrored set-up was that while extra drives IE say 20 (2 sets of 10 drives -mirrored) plus cases, server, all parts seems to compare in price to a 11 drive (10 data + 1 parity- UnRAID) system. Obviously there would be more hardware with the mirrored system, more complexity, and potentially more that could fail. However the flipside is that all 10 drives would have to fail for any data loss. As long as those 10 held different set's of data. Now that sounds like good odds.

 

Given the same or similar pricing I'm just trying to convince myself of the advantage of the 11 drive UnRAID over the 20 drive mirrored system and having trouble seeing it other then simplicity. Any help?

 

 

Hey Jeremymc7,

 

I see many of your points and indeed it is about finding the right solution at a comfortable cost like many have expressed along with your self.  I know many organizations that has to maintain and protect sensitive information spend a good penny on their infrastructure including but not limited to load balancing, data warehousing and server clustering.  For a medium level data protection, RAID 0+1 could provide the redundancies one would look for however when we get far, it's not just about the hard drive but the up time where the switch, cpu, memory, motherboard or power supply failing and for those who have dual hot swap power supply, what if both PSU goes out at the same time.  Is there another standby server w/ mirrored data that can go online.  I even know companies that have mirrored NOC facilities in a different states in case of natural disasters or is that going to far in ones view.  For what we need is simply data protection without the uptime being equal.  It is of course plausible to have two things fail at the same time but remote.  The way I use my unRaid is to use it as my backup from my laptops and desktops data.  These core data files is less than 20GB so pretty easy to have it in a single folder.  I do have folders w/ critical data sync up using small apps like the synctoy from MS.  So my critical files and folders are copied/sync'd to unRAID so if i loose my desktop copy, i have a back on unRAID.  For my laptop, I use a different sync tool that i got off download.com as the SyncToy requires .Net framwork to get installed and didn't want to add that on my laptop.  This method works well as it provides a mirror (folder mirrors) of critical data.

 

I could take it one step further to run another sync on the \\disk1\archive folder to \\disk1\arhcive2 folder if i am in fear of two drives going down at the same time however that would be giving me three mirrors but i'm ok with just having 2 so far.  As for the rest of the remaining TB of data is my movies, mp3 and pictures.  Oh yes, i do sync up my picture folder on this desktop to unRaid so i can view pictures from my xbmc in the living room.

 

If I'm using unRaid as my mapped out folder on this desktop then I would have it sync to another disk to have a second backup.

 

I'm not exactly sure of how much data is critical, maybe mirroring folders could help for you as well?

 

Good Luck,

M

One other factor to keep in mind is unRAID scheme where individual disks are usable outwith the unRAID array. The importance of this cannot be over-emphesised. In a traditional hardware of software RAID array data is extremmely hard or more likely imposible for normal users to recover from a single disk. This means that if the RAID array breaks in some way (and it does happen) you potentially can lose everything.

 

I say this with some confidence as directly prior to moving my data to unRAID i played with an Adaptec 16 ports SATA RAID card using most of the RAID options. When the array was upa nd running it was perfect but if you similate breaking something it was amazing how many times i couldn't recover the array