When was the last time you backed up 12 TERABYTES?

 

My two UnRAID servers have a total of 34TB of data.  It's ALL backed up.  12TB is only 3 4TB disks -- perhaps $450 if you watch the sales.

 

 

unRAID is not a backup.

 

... a point a lot of folks don't seem to understand.

 

 

There is no piece of software that will save you from a natural disaster. I lived through 2 of them in my lifetime.

 

...  The only thing that saved my critical data was an offsite mirror.

 

... clearly you were well backed up.    Nothing like a natural disaster to really bring home how important that is.    You seem to have recovered nicely from the hurricane ... but I shudder to think how much different it might have been without your backups.

 

 

Would I pay $2300 to backup my data.  Yes. However, I would also be wise about what really needs to be backed up.

It would not cost me $2300.

 

I haven't come close tp spending $2300 ... and that's for 34TB of backups.    But even if it did cost that ... if the data's important enough to keep on a fault-tolerant server, then I suspect it's important enough to have a backup.  If you consider dual-fault-tolerance a good enough "substitute" for a backup (an opinion that would likely change the first time you were hit with a real disaster) ... then just use RAID-6.

 

I like the idea of dual parity, but what about a hot spare? (Sorry if that is already done I'm new) The SAN's (Nimble, HP Lefthand) we use at work have that and that seems to help when failures happen. Also I don't know if MTBF numbers are captures by smart and they could be used to suggest replacing a drive before it dies.

 

 

unRAID is not a backup.

 

... a point a lot of folks don't seem to understand.

 

 

I'm kinda sick of hearing this ... it misses the point.  It is an apriorism that ignores real world practicalities.

 

Why do you use unRAID rather than a bunch of spindles and regular backups? 

 

For me, it is because I know drives fail ("fault") and I don't want to wast the time dealing with that fault, I want to keep on running (fault tolerance).  Those faults happen with enough likelihood that I expect the frequency and the "cost" of dealing with it (restore from backup or lose data) to justify the "cost" (in resources and performance) of using unRAID.

 

I do have backups, but even those there is a limit as to what you will "pay" versus likelihood of what you are protecting against.  Do you want a single backup onsite?  That's riskier (but cheaper) than an offsite backup... what about simultaneous loss of your data, and loss of the backup?  Better have 2 offsite backups.  There is a possibility loss of all three, so better have a 3rd backup... etc.  And by the way, better have real-time backup so you don't lose even a minute's worth of work.  You reach the point of diminishing returns.  No matter WHAT your backup strategy, I can come up with an event that would cause you data loss.  So you adopt a backup strategy taking into account both the likelihood of a event, with the costs of protecting against that event.  EVERYONE does this whether they acknowledge it or not.

 

You also have to consider what is on your unRAID box to protect.  I use one server primarily as a convenience storage --- copies of things that are actually archived off-line so I can access it quickly without digging through off-line storage.  So the need to back it up is very small and thus only backed up infrequently (and some not at all).  A small portion of what is on that box is new incoming data, and the need to back that up is higher --- so that gets backed up (single backup, onsite, daily). 

 

With larger drives, the liklelihood of a 2-disk failure grows.  For the same reason I have single parity (fault tolerance) I want double parity.... not because I don't have a backup, but because I want fault tolerance so I don't need to restore from a backup because that is a costly (time, resources, not $$$) process for me.

 

But beyond that, if your fault tolerance passes your own personal threshold of "protection versus costs" of a backup, then fault tolerance (i.e. RAID) is "good enough" for you as a backup.  This is like having 2 cars -- one you have full coverage (including collision and comprehensive) and the other with liability coverage only.  For one of 2 cars, I'm willing to have the much lower premium versus what I judge the risk of it being needed is.  I don't have flood insurance, since I have judge the likelihood of a flood to be low enough to not justify it, although at another house, I do have flood insurance because I *do* consider the likelihood of a flood there to be higher.

 

All my data is not treated equally.  I have a small amount of stuff that is triple backed up to 3 different off-site locations.  I have a lot that is single backup onsite.  I have some that is not backed up at all... not because I don't want to back it up, but because the "cost" versus "benefit" is not worth it.... like a car that you don't carry collision coverage on.

 

If unRAID has double parity, that gives me 2 things -- 1) the things I chose not to back up now have an order of magnitude more protection against loss and 2) some things I am backing up now, I won't back up in the future because double parity is "good enough" for that data. 

 

But even with double (or triple) parity, there will still be some stuff triple-backed up to 2 or 3 off-site locations.... just like with no parity at all, there is still some stuff I do not back up at all and never will.

 

unRAID is not a backup.

 

... a point a lot of folks don't seem to understand.

 

 

I'm kinda sick of hearing this ... it misses the point.  It is an apriorism that ignores real world practicalities.

 

Why do you use unRAID rather than a bunch of spindles and regular backups?

 

You are absolutely right. You use it to backup or store and serve your media or backup files from another host.

The key to this statement is, if it's important and it only exists on the unRAID server, then unRAID is not a backup.

 

However if you have more then one copy, locally, USB drive or whatever, then unRAID is or can be your backup.

For me it was the backup as all the original media was destroyed.

 

When I had multiple workstations, the unRAID server was my backup to them.

Files that were important were also backed up to another location.

 

However if the unRAID copy is your only copy, then unRAID is not a backup.

It always needs to be qualified in terms of importance and amount of copies.

I suppose I should qualify it more if I express it.

 

That being said, I would be happy with a hot spare or dual parity if I had large arrays.

 

There were many opponents to the hot spare idea. Arguments were, many failures usually occur due to an intermittent problem with hardware, re movable trays, wiring.  I agree yet I'm still a proponent.

It was already available in the md driver and could probably be implemented easier then dual parity.

 

On the other side of that, the time invested in hot sparing could probably used for dual parity

Arrays are getting pretty large, so it's a cost effective use of time to go dual parity.

My concern has always been speed. I don't have to use it now.

My architecture these days is multiple smaller arrays on multiple smaller machines with larger spindles.

 

I can accept either that will provide a better fault tolerance experience.

The insurance analogy is reasonably accurate -- you should insure what you can't afford to lose; just as you should backup what you can't afford (or don't want to) lose.

 

... some things don't need to be insured; just as some data doesn't need to be backed up.    And clearly even if you DO want to backup your data, you have to ask just "how many 9's" you need, as the cost of each additional 9 is exponentially higher than the previous one ... i.e. to be 99% protected is much cheaper than to be 99.9%, vs 99.99%, vs 99.999%, etc.   

 

My point is simply that if you ONLY have your data in a single system -- no matter how fault tolerant it is (single parity, dual parity, triple parity, etc.) it is NOT backed up.    It may, in fact, be reliable enough that you don't think you need to back it up -- but for most folks family pictures, financial data, etc. are NOT in that category.    ... but there have been numerous instances where folks have posted in a panic on this board because they've lost exactly that category of data.

 

It's certainly true that even if you have multiple backups at multiple sites you can come up with a scenario where there will still be data loss ... but those tend to be well beyond the number of 9's that most of us are willing to pay for (and many of those scenarios are doomsday events that will make data preservation the last of your worries !!).

 

I also agree Dual (or more) parity should be the absolute No.1 priority, PERIOD.

 

I have considerably more than 100TB of data, increasing at 4-5TB/month, to back it up takes nearly 2 months running flat out over gigabit.  It's just not practical.

 

Your system is stressed the most when it is rebuilding a failed drive, the chance of second (and third) failure during this process is frighteningly high.

 

My critical stuff (which is only a few GB’s) is stored on multiple RAID boxes in 2 physical locations.

 

My point is simply that if you ONLY have your data in a single system -- no matter how fault tolerant it is (single parity, dual parity, triple parity, etc.) it is NOT backed up.

 

Some fault tolelerance arangements exceed the protection of some backups.... depends on how you score them.  But regardless of whether it is a "backup" or "fault tolerance" nothing is 100% perfect.  In each case it is "protected" to whatever number of 9's you have... single parity has more 9's than no parity, and double parity has more 9's than single parity.

Most "high availability" systems require at least 5 9's of reliability (i.e. 99.999%) ... which equates to something like 6 minutes/year of downtime.  Amazon's S3 "durable storage" class claims 11 9's ... i.e. 99.999999999% reliable.  Clearly this is achieved with multiple duplicate diversely located data centers that all have very high fault tolerance.

 

But I don't think ANY individuals are going to spend what it takes for this level of reliability 

 

 

Yes, and the incremental cost of adding 9's in the example of adding dual parity to a single parity RAID, is actually very inexpensive compared to other methods of adding 9's do an existing single-parity RAID.

Dual parity is definitely #1 on my list of new features.  I have 6 drives in my system.  It used to be the case that I needed 4 drives for storage (plus one cache and one parity), but now that hard drives are getting so large I would easily be able to dedicate 2 of them to parity: 2 parity, 3 data, 1 cache is plenty of space now that 4TB drives are commonplace.

 

Hard drives are currently growing more rapidly than the data that I'm accumulating, and ensuring that the array can survive 2 concurrent failures is very important to me.

++1 on dual parity. As some drives get older, the chances of a failure are increased. If we are able to replace the drives as they get older we decrease this likelihood but with a dual parity we can be 'encouraged' to keep utilizing the older drives until a failure occurs. As others have pointed out, the probability of a similar aged drive failing during a disk rebuild is higher and a dual parity would greatly increase the confidence that data is safe.

 

Dual parity is definitely #1 on my list of new features.  I have 6 drives in my system.  It used to be the case that I needed 4 drives for storage (plus one cache and one parity), but now that hard drives are getting so large I would easily be able to dedicate 2 of them to parity: 2 parity, 3 data, 1 cache is plenty of space now that 4TB drives are commonplace.

 

Hard drives are currently growing more rapidly than the data that I'm accumulating, and ensuring that the array can survive 2 concurrent failures is very important to me.