Putting together a primary 15TB+ stock unraid system and I'm trying to figure out if I should wait until the beta movement is over. The biggest feature that I want to be up and running is the dual parity drives (P+Q Redundancy) (http://download.lime-technology.com/develop/infusions/aw_todo/task.php?id=23). 

 

Is there any update on the dual parity drives option?

 

Any idea when unraid will come out of it's Beta movement?

 

Should I trust the latest beta release in my situation?

Waiting for 5.x to come out of beta is going to be a while. No published time frame. 5.x has been in beta since Aug 2010.

 

Dual parity, Is not on any published schedule.

 

5.x beta's are somewhat reliable depending on which version and hardware. I think .12-.14 is fairly solid.

4.7 is reliable.

Waiting for 5.x to come out of beta is going to be a while. No published time frame. 5.x has been in beta since Aug 2010.

 

Dual parity, Is not on any published schedule.

 

5.x beta's are somewhat reliable depending on which version and hardware. I think .12-.14 is fairly solid.

4.7 is reliable.

 

Tom's recent post under move news is the closest thing to a road map for a "final" v5 that I have seen.

http://lime-technology.com/forum/index.php?topic=18512.120

Though it doesn't get to a time frame, it seems to imply soon.

 

As noted, I have never seen more than a footnote about dual parity drives, and would like to see it myself.  However, would be happy with a stable v5 that supports all popular hardware and glad to see we are close.

Is there any update on the dual parity drives option?

 

Nope.

 

Any idea when unraid will come out of it's Beta movement?

 

Nope.

 

Should I trust the latest beta release in my situation?

 

Check your hardware is supported carefully if it is then you'll likely be OK. I've found 5.x to be a little bit flakey around nfs (borderline unusable, not just me from other forum reports) and I've also experienced a lot of glitches when configuring through the web interface - settings not being applied correctly, strange things happening as settings are half applied and the web interface and unraid getting in a flap about what's going on as a result etc. However I'm running beta12 due to hardware requirements so they may have been fixed in later betas.

 

You should be ok but do keep in mind it's a beta release and sometimes it shows. I've certainly had more issues with 5 than I did with 4.

 

As to whether running 15TB+ without dual parity is sane?

 

Filesystem            Size  Used Avail Use% Mounted on
shfs                   16T   11T  5.2T  68% /mnt/user

 

I hope so! But in seriousness as unraid doesn't stripe the data if you have more than one drive failure your risk is mitigated by only losing the data on the drive(s) that fail. Not the whole lot. Dual parity would be nice but it's no where near as necessary as a traditional striped setup. Depending on your usage patterns and config you'll also likely find that most of your disks stay spun down the majority of the time. And, as ever, parity protection (in whatever form, be it unraid or striped) is for availability and is not a substitute for backups.

IMHO, dual parity is more of a convenience than a necessity with unRAID.  In a "normal" setup like RAID5, if 2 drives each have a single read error, your whole array is toast.  That's why multiple parity and mirroring started taking off as arrays grew.  But unRAID doesn't work like that.  Even if your array gets blasted with cosmic rays and bits get flipped across all the drives, only the data that's corrupted is affected.  You can still read all of the drives and the files on those drives individually.  If you're hit by a drive-by and 3 of your drives get shot, the rest of the drives can still be read as individual drives and their data is intact.

 

I don't think there's any reason to wait for dual parity.

My 2 cents: as data drive density increases, the loss of a single drive becomes more problematic. I, for one, am very keen on the incorporation of dual parity in UnRAID.

Why wouldn't it be? I have 51TB in my unRAID server. 20 data drives is 20 data drives. You'd still have to lose 2 drives at the same time to lose data, and you'd still only lose 2 drive's worth of data in that extremely rare case.

 

I'd see no problem with 20x 10TB drives and 1 parity, if they existed. Now if you wanted to expand beyond 20 data drives (unRAID can't right now), then I would wait for dual parity.

+1 on the previous statement... If you -have- 20 drives storing your media collection then loss of 2 drives will hurt but you can still run a fair size video rental store with what you have left :-)

 

Basically if you have data that is THIS critical you probably need to think of another solution (remember that unraid only helps you with failing drives, not with all kinds of other stuff that might cause loss of data)..  Probably off-site storage of critical data is a better option (example: I have placed a copy of all my photos and personal data on a few older drives I had lying around and have placed them at another location, if the house catches on fire I will have that critical data saved...

 

On the other side: I would probably activate double parity if it was there, just for the heck of it (y'r a nerd or not ;-)

Speaking frankly, the loss of two drives within a short time frame seems to happen far more often then we would like to consider.  It's of paramount importance to do monthly parity checks so you are aware of any pending sectors that would cause difficulty in a recovery situation.

Great input! thanks guys,

 

I guess the parity checking function would be the biggest reliance.  Does the function do a full read from all disks?  Does it notify you of all errors? Is there a way to automate the parity checking and ship the log files out to an email address through the web interface?

 

2TB of data loss is still substantial. The experts say that the majority of issues are going to be hardware/hard drive related. There are user controlled issues that can be the issue, but they are not as likely (around 25%). Dual parity is insurance for the backup system that would really make that dreaded few days of hard drive replacement a lot less stressful. I wouldn't care about sacrificing the cycles for the overhead if i knew that there was that extra layer if something does happen.

 

What time frame am I looking at to recover from a 2TB drive failure?

 

What time frame is a full parity check take?

 

Any other recommendations for a new 15+ unRaid (besides hardware clearance)?

I guess the parity checking function would be the biggest reliance.  Does the function do a full read from all disks? 

 

Yes, so it gives the system a good work out.

 

Does it notify you of all errors?

 

Yes, and it gives you the block numbers that are at issue - note this does not pin point which drive is at issue, just that at a certain point in the array one (or more) of the drives had an issue.

 

Is there a way to automate the parity checking and ship the log files out to an email address through the web interface?

 

The checking can be set to run periodically, the log file can be viewed or downloaded through the web interface.  There might be a way to get this emailed out but I don't think the standard system does this.

 

2TB of data loss is still substantial. The experts say that the majority of issues are going to be hardware/hard drive related. There are user controlled issues that can be the issue, but they are not as likely (around 25%).

 

The common experience here is that the most common issues are:

 

1. drive problems

2. cabling problems (loose SATA connectors, power connectors and especially power cable splitters)

3. power supply problems

4. crashes due to loss of AC power (for those that don't use a UPS)

 

Dual parity is insurance for the backup system that would really make that dreaded few days of hard drive replacement a lot less stressful. I wouldn't care about sacrificing the cycles for the overhead if i knew that there was that extra layer if something does happen.

 

What time frame am I looking at to recover from a 2TB drive failure?

 

The best you can do is to have a hot spare drive, then you just have to stop the array (but not power it down), reconfigure (just exchange the faulty and spare drives through the web interface) and restart the array.  Once the rebuild is done you can shutdown the system and physically remove the bad drive for further testing and put another hot spare in its place.  The rebuild will take about 8-10 hours (maybe a bit longer than a parity check), but this will be slower if you are using the system at the same time.

 

What time frame is a full parity check take?

 

On a system with 2TB drives something like 8-10 hours.  Mine has 8 data and 1 parity drive and it usually takes about 9 hours.

 

Any other recommendations for a new 15+ unRaid (besides hardware clearance)?

 

Get a well ventilated case with plenty of drive bays and a good single 12V rail power supply (something in the 500-600W range will usually do) because you'll be wanting to add more drives in the future.

 

 

Regards,

 

Stephen

... Depending on your usage patterns and config you'll also likely find that most of your disks stay spun down the majority of the time.

 

This is a major variance that most "experts" do not test for. In enterprise environments hard drives are running 24/7/365 to ensure constant availability and performance. This also promotes multiple drive failures within a small window of time if all the drives were installed at the same time from the same batch.

 

One of the primary advantages of unRAID is it's ability to spin down inactive drives to conserve power and theoretically extend lifespans. Even if all the drives were installed together, and from the same batch, the staggered write / read mechanisms create gaps in usage that over time create a lag factor, helping to reduce simultaneous failures.

 

In a "normal" setup like RAID5, if 2 drives each have a single read error, your whole array is toast.  That's why multiple parity and mirroring started taking off as arrays grew.  But unRAID doesn't work like that.  Even if your array gets blasted with cosmic rays and bits get flipped across all the drives, only the data that's corrupted is affected.  You can still read all of the drives and the files on those drives individually.  If you're hit by a drive-by and 3 of your drives get shot, the rest of the drives can still be read as individual drives and their data is intact.

 

This is the other facet of the equation. In traditional RAID schemes, the loss of too many drives destroys the entire array. In unRAID, it's a much smaller exposure of total data loss.

 

 

 

Is it possible for 2 drives to fail within a few days of each other on unRAID? Of course. If your system was hit by lightning, you can kiss ALL your data goodbye, pending a reasonable backup protocol. P + Q parity (which I admit _I_ really want) gives us one extra drive failure protection, but in practice, if you have 1 drive fail and you don't have another spare laying around, you shut down until you can rebuild. Assuming it is not a commercial server running always available data, following that procedure further reduces the likelihood of data loss.

 

I guess the point is that there are several steps that can be taken to mitigate and reduce the exposure to data loss related to a hard drive failure, regardless of the level of layered parity protection provided by the OS.

IMHO, dual parity is more of a convenience than a necessity with unRAID.  In a "normal" setup like RAID5, if 2 drives each have a single read error, your whole array is toast.  That's why multiple parity and mirroring started taking off as arrays grew.  But unRAID doesn't work like that.  Even if your array gets blasted with cosmic rays and bits get flipped across all the drives, only the data that's corrupted is affected.  You can still read all of the drives and the files on those drives individually.  If you're hit by a drive-by and 3 of your drives get shot, the rest of the drives can still be read as individual drives and their data is intact.

 

I don't think there's any reason to wait for dual parity.

 

That's exactly it: Unraid protects us from data loss by cosmic rays and drive by shootings !  Lets see drobo top that :-)

That's exactly it: Unraid protects us from data loss by cosmic rays and drive by shootings !  Lets see drobo top that :-)

 

This should be the Lime-Tech slogan!

 

unRAID

... protects your data against cosmic rays AND drive by shootings!!!