Hey all unRAIDers, I'm super excited to finally get my rig up and running. I've been crawling the wiki and forums all day today, tons of useful stuff here.

 

Anyway, my question is, WHY is it necessary to "clear" a new hard drive? I have seen the excellent preclear script and perused that thread. I understand that it's GOOD PRACTICE to try to read and write every bit of the disk, but if unRAID just checks for a special "code" to let it know that the disk has been cleared, why can't I just write that code to my disk without actually clearing it?

 

I know that there is risk involved in this, but what if I'm willing to take that risk? It seems to me that what clearing does is to spend 10-15 hours clearing a 2TB disk, so that we know each and every bit is good (or have reallocated the bad bits). What if I value my time more than the integrity of every bit? Especially with a factory fresh drive, I just don't see the point.

 

Am I just being naïve here? Can someone please enlighten me?

 

Thanks! K

Until you assign a parity drive and initially calculate parity it is not necessary to take any special steps to maintain that parity to protect your data. 

 

After you establish parity, any drive you add to the array will be cleared by unRAID while the array is kept offline unless it is already precleared.  The time off-line will depend on the size of the disks but it will easily be 8 hours or more with a 2TB drive.

(The entire array will be un-usable during this time)

 

As far as trusting the brand new disk...  bad idea.  I'm constantly reading of disks that are bad when first used with the pre-clear script.      If not weeded out initially, there is too high a chance of a disk problem and it would not just affect a specific disk but the ability to recover from a failure on any disk.

 

The need to write zeros is because unRAID works with even parity.  Let's pretend all your disks were only 1 bit.  Let's pretend you had 19 existing disks plus parity. 

 

The values on the disks are:

parity = 1

disk1  = 0

disk2  = 0

disk3  = 1

disk4  = 0

disk5  = 0

disk6  = 0

disk7  = 1

disk8  = 1

disk9  = 0

disk10 = 0

disk11 = 1

disk12 = 0

disk13 = 0

disk14 = 0

disk15 = 0

disk16 = 1

disk17 = 0

disk18 = 0

disk19 = 0

 

There are an even number of bits set to a "1" (even parity)

To add a new disk  we must ensure its "bit" is a zero to maintain even parity.  This is to permit continues parity protection for all your other data.    Normally unRAID will write zeros to every byte in a new disk to ensure continued parity protection.  (Adding a bit with a zero is guaranteed to not result in an odd number of bits set to "1")

 

A disk with a pre-cleared signature is trusted to have all bits zero, so the lengthy clearing process is skipped.  A disk without a pre-cleared signature will have all its bits set to zero to allow continued parity protection of all your other disks (and your precious data.)

 

Forcing a pre-cleared signature would potentially risk the continued parity protection of your other disks if it was not actually cleared.    It is completely your call.  You do not have to clear/pre-test your drives, but if you value your data, it is a good idea.

 

When initially getting your array up and running you can just not assign a parity drive.  No clearing will be needed so you can play with the software.  You'll not have parity protection but it will let you get started.

 

Joe  L.

 

As Joe L has already pointed out most things I will just add some of my comments on clearing/pre-clearing

 

Anyway, my question is, WHY is it necessary to "clear" a new hard drive?

It is necessary ONLY when a parity drive is assigned for the array.  If you add a disk to the array and no parity drive is assigned there will be no clearing process done by unRAID.  Pre-clear is a slightly different beast but does essentially the same thing.

 

I understand that it's GOOD PRACTICE to try to read and write every bit of the disk, but if unRAID just checks for a special "code" to let it know that the disk has been cleared, why can't I just write that code to my disk without actually clearing it?

You could write just that part of the code, but if you then assigned that drive to the array and every bit was not 0 you would lose parity protection if a drive where to fail.  If you did a parity check after adding the new drive it would come back with errors if it found a single 1 on the disk you just added.  You are playing with fire by not having a drive cleared, either via unRAID itself or Joe L's pre-clear script.

 

I know that there is risk involved in this, but what if I'm willing to take that risk?

As I said about, you are playing with fire and you could be putting your data at risk.  You would be gaining 8ish hours of use for a drive if you let unRAID clear it and 25ish hours of use for a drive if you did a full clear via pre-clear.

 

What if I value my time more than the integrity of every bit?

Then your a little crazy .  In the case of an unRAID server which is meant to protect your data, it is stupid to value time of data integrity.

 

Especially with a factory fresh drive, I just don't see the point.

DO NOT trust "factory fresh" drives.  I have had 3 fail on me within a month of going into use.  I pre-clear all my drives now before they will ever touch my array.  I typically do 3 passes on a drive before trusting it.

 

Am I just being naïve here? Can someone please enlighten me?

You are being a little naïve, but live and learn.  If you attempt this and then one of your drives goes down you will lose all data on the drive and more than likely NOT be able to recover it (at least fully).

 

The whole point of unRAID (and any RAID for that matter) is to give us a line of defense against hard drive failures.  It's not not a matter of if a hard drive will fail it is a matter of when.

I want to add my comments as well.

 

Generally, hard drives failures fall into three types (there are more, but for this topic I am limiting them to 3)

 

1. Drive is DOA.

 

2. The drive fails after a short time (seconds, minutes or hours)

 

3. The drive fails after many long hours of faithful service.

 

In the first case, you data is safe, nothing can be written to or read from the drive. The third example is what we are all using unRAID for, to recover from a drive failure. T

 

he middle example is what Joe L's wonderful script is trying to protect us from. Working with a "soon to fail drive" can cost the user (you) many hours, and much frustration. If the drive is wonky, it can make your server wonky. This script can protect you from that. By forcing the drive to run through its paces you are forcing the drive to either prove itself, or fail SOONER rather than later. That information can then be used to contact your supplier for a timely replacement, while the drive is still under warranty.

 

I think many of us believe that it is better to spend a few hours up front, using the preclear script rather than spending it on the backend dealing with weird unexplained crashes, or trying to recover data from a drive that lasts just long enough to make exchange difficult, or impossible.

 

It is your call, but the script has already found two bad drives for me, out of 8.

 

I, for one, am willing to spend the time upfront, and have a smoother experience while using my server.

 

Bruce

I think there is a little confusion because preclear accomplishes two very different things ...

 

1 - It performs a pretty significant burn-in test of the drive

2 - It leaves the drive in the precleared state for easy adding to an unRAID array.

 

#1 is useful for every new disk - whether destined for your unRAID server or not.  I use it for disks that go in my personal workstation.  The fact that it comes out precleared is irrelevant, it just does a good job of beating up the disk to weed out drives subject to early failure.

 

#2 is useful only for disks that you plan to add to an already parity protected array.  Not for parity disks, and not for disks you plan to use to replace other disks (i.e., failed disks or upsize smaller disks)- only for disks that will be added as new data disks and increase the number of drives in your array.

 

So when should you use it?  Every new disk you buy.