From the Main page:

"Similar to other RAID systems, unRAID Server permits reconstruction of a single failed hard drive. However in the unlikely event of multiple hard drive failures, data loss would be isloated to only those hard drives which failed. In traditional RAID systems, multiple simultaneous hard drive failure results in complete data loss."

 

Does this mean that UR can recover from only a single failed drive, and that the data from a second failed drive is completely lost?

 

 

From the Main page:

"Similar to other RAID systems, unRAID Server permits reconstruction of a single failed hard drive. However in the unlikely event of multiple hard drive failures, data loss would be isloated to only those hard drives which failed. In traditional RAID systems, multiple simultaneous hard drive failure results in complete data loss."

 

Does this mean that UR can recover from only a single failed drive, and that the data from a second failed drive is completely lost?

If they were to fail AT THE SAME TIME, yes. You are correct, data from both drives is lost.  If one of the two failed drives is the parity drive, then you have only lost one drive's worth of data.

Now, in normal RAID4, or RAID5, loss of two drives are the same time results in loss of ALL the data on ALL of the drives, not just the two failed drives.

 

However, if one drive fails this month and you replace it, and another next month, and you replace it, and another next year, and you replace it, you can keep going without data loss.    The vulnerable window of time is that between a drive's failure and its replacement being in place, and data re-constructed on it.  If it takes you three days to notice a drive has failed, obtain a replacement, and install it, and rebuild the data on it, then if a second drive were to fail in that three day window, data is lost.

 

RAID arrays of any kind are not a replacement for backups, and off-site storage of critical files.  All it would take is one good fire/flood/explosion/direct lightning strike, and you can lose more than a single disk... odds are the entire server might be toast.  (in the case of a flood, soggy toast  )

 

Joe L.

Well, it depends on the type of failure.

 

In simplified terms, drives can fail either marginally or catastrophically. 

 

A marginal failure, means a particular spot on the disk or portion of a file has failed, but the rest of that disk is still readable.

A catastrophic failure is one where the entire disk has failed and is unreadable.

 

In a striped array (i.e. RAID5) lose one disk to either type of failure and the array still functions, using parity.

In a striped array (i.e. RAID5) lose two disks to either type of failure and the array goes offline, and ALL data is unreachable, including data on the failed drives and the non-failed drives.  Potentially, a (very expensive) service can recover some data, but you'll have to ship them ALL the drives (and usually, a complete extra set to send you the data back on).

 

On unRAID, lose one disk to either type of failure and the array still functions, using parity.

On unRAID, lose two disks to either type of failure and the array fails, and the array goes offline.  When this happens there are three important differences from striped RAID:

 

• 
  1) the unaffected disks can be mounted and accessed individually, without data loss.
  

• 2) if either or both of the failed disks' failures were marginal (i.e. only affecting part of the disk or files), the marginally failed drive(s) can usually be mounted and the unaffected portions of the disk can be accessed just fine, and the data copied off to a safe location.
  

• 3) any catastrophically failed disk can be sent to a (less expensive) recovery service and the data on that single disk recovered.  (Note, this does not apply to the parity drive, for which recovery is generally not necessary and not possible, since the data can not be recovered to the same sectors).
  

 

I also disagree that unRAID is not a substitute for backups.  In some cases, it is -- it provides fault tolerance of a type that is acceptable for replicatable data.  I.e., you can re-rip all your media in a catastrophe rather than spending thousands of dollars on a 10 TB backup solution.  Sometimes backups are used to minimize MTTR (downtime) rather than for data security.  In that situation, unRAID is a quite proper substitute for backups.

A couple comments:

 

1.  I am not that familiar with RAID, so can't comment on what it does if a bad sector occurs, but unRAID handles these gracefully if parity protection is in place. After it gets the read error, unRAID will read all the corresponding sectors on the other disks and reconstruct the data it couldn't read.  It will WRITE that data back to the offending disk.  This write will cause a remapping to occur within the drive - basically correcting the error.  All this will happen without user involvement.  A number of these could happen on different disks one after another and no data loss would occur.

 

2.  The word ONLY in the statement below casts dispersions on unRAID and most (all?) RAID systems also:

 

Does this mean that UR can recover from only a single failed drive, and that the data from a second failed drive is completely lost?

 

The truth is drives are pretty darn reliable all by themselves, and the chances of 2 simultaneous failures, without some common cause (like a fire), is EXTREMELY unlikely.  Given that you can never reach perfect reliability, and more parity is not going to protect you from things like fire, flood, or (perhaps most importantly) user error, how much is the extra amount of reliability going to benefit you?  Is it worth $200 and a HUGE performance hit?

 

3.  The nice thing about unRAID is that if two drives were to go south at the same time, or more likely some hardware/user error occurs that messes up a few disks, you can still mount each (working) disk and get your data off of it.  With RAID, if two disks go down ALL the data on ALL the drives is lost.  RAID arrays are also tied to a specific controller, so moving RAID arrays between machines can be very difficult.  The RAID controller a single point of failure, and if yours breaks, you may be looking at trying for find one just like it to mount your array.  unRAID arrays can be moved from an old machine to a new machine with no problem.

 

 

 

After it gets the read error, unRAID will read all the corresponding sectors on the other disks and reconstruct the data it couldn't read.  It will WRITE that data back to the offending disk.  This write will cause a remapping to occur within the drive - basically correcting the error.  All this will happen without user involvement.  A number of these could happen on different disks one after another and no data loss would occur.

 

 

Will this happen during normal use, or only when a parity check is performed? 

 

Ex. - I'm watching a movie from disk 3 and a read error occurs.  Will it auto correct?

Will this happen during normal use, or only when a parity check is performed? 

 

Ex. - I'm watching a movie from disk 3 and a read error occurs.  Will it auto correct?

Yes, the same corrective action will occur.

If the other disks in your array are not currently spinning, you will notice a slight pause in the playing of the movie as they all spin up to supply the data to re-construct the block that could not be read.

 

A full parity check allows these errors to be caught early... I know I'll never read every block of every disk in years of just watching movies.  If you have any size collection, neither will you.

 

Joe L.

Any time unRaid  gets a read error it will do this - during parity check or normal operations. Parity just needs to be valid.

 

The danger in REBUILDING parity is that you are not protected from a disk failure or a sector error.

Thanks for the info, guys! 

 

I've only experienced two 'catastrophic' drive failures in my time, but frequently have seen bad sectors pop up as a disk ages.

 

 

It is probably worth noting that your drive can actually predict that a sector will fail soon and remap it automatically before it actually fails. This functionality works on any drive whether in unRaid or not.

 

A full parity check gives the the drive a chance to examine each and every sector. It is a good preventative maintenance thing to do periodically (e.g. Monthly).

I mentioned this in my own thread, but I thought I would bring it up here too...

 

I do storage support for a living, and we have done all sorts of nasty math-heavy studies on data loss using information collected from tens of thousands of customers running exabytes worth of total storage.  This enables us to develop numbers for array size/reliability/disk-technology/RAID-level tradeoffs, and then make a decision as to the largest arrays we will suggest customers create with a given configuration.

 

The most common cause of data loss with a single-parity array (RAID 3/5) is NOT catastrophically losing two drives; instead, it is what we refer to as a "stripe kill".  In a stripe kill, you lose one drive entirely and during the rebuild, discover that you have some unrecoverable tracks on one of your remaining drives.  For large businesses, which run several dozen arrays of 12+ drives each, this happens distressingly often.  When this happens, you can now no longer trust your filesystems/databases/etc., and it's time to pull out the backup tapes.  (Most enterprise arrays will actually stop the rebuild in its tracks if encounters a single unreadable track on one of the "intact" drives.  The LAST thing most customers want is silent data corruption... if you have a single unreadable track, you are likely to have more, and you don't want to run the risk that a bad track will sneak through error detection.)

 

unRAID does help out with this in that when this does happen, you only lose a single drive's worth of data, instead of losing a dozen terabytes in one fell swoop.

 

You can make this less likely to happen one of three ways:

1) Don't use RAID 3 or 5... just take the perf and capacity hit and mirror or find a product that will give you RAID 6, which delivers double parity.

2) Decrease the array size.  With consumer-level drives, I wouldn't run any more than half a dozen in an array.

3) Run parity checks often.

 

SirWired

Thanks SirWired!  This is really useful information.  I hope that everyone reads and understands your message.

 

Catistrophic drive failure is very noticable**, but individual sector failures are undetectable.  unRAID can easily fix the individual sector errors while parity protection is in place IF but only if you try to access those sectors., It is so automatic you might not even know it happened, except for a slight delay.  But if you have a catastrophic drive failure, and unRAID needs to rebuild that disk, it is expecting that there are NO bad sectors ON ANY OTHER disk.  Even one bad sector will lead to the condition that SirWired called "stripe kill", but for unRAID I would just call "imperfect drive reconstruction".  I don't believe that unRAID would stop reconstructing, but the user would be left wondering what file(s) got corrupted without any way of knowing for sure.

 

(** I say that catastrophic drive failure is easy to notice, but that isn't really true with unRAID.  The nature of these arrays is that certain disks go unaccessed for weeks, months, maybe even years for some users.  Unless unRAID has some reason to go looking, it will likely not even know that a disk has failed.  And even if unRAID knows, YOU may not know because unRAID will simulate a failed disk.  Unless you notice a slowdown or look at the Web GUI you won't know!)

 

This begins to argue more loudly for an option for a second parity disk, not so much to protect against two simultaneous catastrophic drive failures (which is highly unlikely), but as a way to ensure that perfect drive reconstruction will be possible if you have one catastrophic failure.  The problem with the second parity disk is cost and (write) performance.  Perhaps with the 4.3 cache disk feature, write performance is not quite as bad an issue as it once would have been.  But if write speeds to the array went from 45 MB/s without parity to 15 MB/s with 1 parity, I could see performance going to 5 MB/s (or even slower) with a second parity disk.  I'm also not sure if you can implement a RAID-6 like mechanism without striping.  2 parity drives with each other drive maintaining its own integral structure may just not work.

 

So what to do?

 

1 - Bring up the unRAID GUI often (at least once a week) and make sure that you're not seeing any disk failures (red balls).  (Some users have installed monitoring software to send themselves an email if a drive were to fail.  I think this would be a good idea for folks that don't think they can be that diligent.  Perhaps one of our Unix gurus could create some scripts and directions for setting this up.  Hint Hint!)

 

2 - Run full parity checks.  I suggest monthly, but others do it quarterly.  Frequency is up to you, but running them periodically is critical. Realize you will NEVER access all your sectors on all your disks through normal use.  A clean parity check is the ONLY way you will know that you are protected in the event of a SINGLE catastrophic drive failure.  (There has been much discussion of renaming the maligned "Restore" button, but I think that the "Parity Check" button should also be renamed to "Preventative Maintenance Cycle", and the documentation should clearly identify the need to press it periodically.  Even better, would be an easy way to schedule the "Preventative Maintenance Cycle" via the Web GUI to occur on a periodic basis - for example I would like to run it at 1:00 am on the first Thusday of each month, and have the results emailed to me.)

 

3 - Run smartctl on your drives periodically.  Running them before and after each parity check cycle would likely be best.  Monitoring some of the important stats like max temperature, Reallocated_Sector_Ct (should be 0 or a very small number), Current_Pending_Sector (ditto), and the list of recent errors.  Monitoring this may give you an idea that the drive is starting to go south before you learn about it in a less friendly way.

 

With a little care and feeding unRAID provides a tremendous amount of data protection.  But if you just trust the technology and wait until a drive fails, you may be disappointed that you are not able to get the perfect reconstruction you hoped for!  When and if a second parity option becomes available, perhaps care and feeding will be less critical, but until then these are the best practices for keeping your data safe.