I've used the latest pre-clear script several times with good success. I've been swapping my 1TB drives for 2TB drives and then slapping the 1TB drives back into the system on the add-on card. I have a question though. After I do a 21 hour pre-clear and it passes just fine, then I add the drive into the system why does unRAID do another clear that takes not 21 hours, but quite a long time?

 

 

Which version of unRAID and what options are giving to preclear?

Which version of unRAID and what options are giving to preclear?

 

I have version 4.7 and I just use default pre-clear. Example:

preclear_disk.sh /dev/sdb

 

I have vanilla WD Black drives. I just added a replaced drive back into the system and it is doing a "clear" again. This drive was pre-cleared when first used, been running in the array for a few weeks and it has to clear again? I'm not against good efforts in keeping things safe, but if I'm %100 sure the drive is good couldn't there be a bypass option or something?

 

If it was precleared but then formatted after the fact, then unRAID will clear it on its own to ensure the drive is prepared properly.  There is no workaround for this except to preclear it again or let unRAID do its thing.  It's not a matter of you trusting the drive, it's a matter of unRAID reinitializing the disk.  Don't confuse "preclearing" with testing a drive's reliability.  While preclearing over several cycles can help indicate if the drive is failing, it is in fact a method to clear the drive outside the array before adding it.

If a drive is pre-cleared and then directly added to the array it should not be cleared again. This is one of the key benefits of pre-clear. It allows adding a drive with little array downtime. I have never seen a drive cleared again after I pre-cleared it and then directly added it to the array. Which version of pre-clear are you running?

 

And, you should executing: preclear_disk.sh -A /dev/sdb To ensure 4K-alignment.

From the looks of it he just added a 1TB drive so it should be a regular preclear.

 

Did you preclear the 1TB drive you just put in or did you just pull out the 1TB and put in the 2TB and follow up with putting the 1TB back in the machine?

Interesting...

 

The first time I put a brand new WD20EARS and setup unRaid 4.7 for the first time, I didn't even have to preclear or even clear the drive.  I just formatted the drive and it worked.  Total time from disk insert to server start in less than 5min.  Is that supposed to happen?  I didn't even know about preclears until after reading this board.  I went back and precleared the drive just as a burn-in test.  But the fact remains that I never had to clear the drive at all.  Just a quick format in unRaid and it worked.

If you had no parity drive installed then that is normal.  Likewise if you installed the drive as a cache drive then that is normal.  If you did have a parity drive installed and weren't installing the new drive as a cache drive, then that is not normal.

This drive was pre-cleared when first used, been running in the array for a few weeks and it has to clear again?

 

Every drive that is added to expand an existing array has to be cleared to preserve parity once it is added. A drive holding all 0's can be added without affecting the parity so the drive is first 0'd (cleared) and then added to the array. The preclear script makes the drive clear and ready to add without having to let unRAID do the clearing itself. In this sentence you are indicating that you are adding a used 1T drive to expand the array. The disk will have to be cleared.

 

Peter

Interesting...

 

The first time I put a brand new WD20EARS and setup unRaid 4.7 for the first time, I didn't even have to preclear or even clear the drive.  I just formatted the drive and it worked.  Total time from disk insert to server start in less than 5min.  Is that supposed to happen?  I didn't even know about preclears until after reading this board.  I went back and precleared the drive just as a burn-in test.  But the fact remains that I never had to clear the drive at all.  Just a quick format in unRaid and it worked.

 

Ya, in a new array unRAID just builds the parity based on the existing bits stored on the disks. Any "leftover" existing 1 bits are accounted for when the parity drive is written. This is why you had  that 7 or 8 hours of parity drive writing when you first started the array.

 

Peter

 

If you had no parity drive installed then that is normal.  Likewise if you installed the drive as a cache drive then that is normal.  If you did have a parity drive installed and weren't installing the new drive as a cache drive, then that is not normal.

 

So is it recommend to make sure you start with adding a Parity Drive first before proceeding to add any data disks (non cache), bring up the array, stop the array then add the data disks which would clear each one (time consuming). OR if you are starting off fresh (completely new build, MB, Drives, etc...) and lets say you ran preclear_disk on all your new drives as a personal preference to make sure they are good to then (first) add them in as data disks, then bring up the array then stop the array and ONLY then put the parity drive in (being last) in order to skip the unRaid clear process of each data disk?

Interesting...

 

The first time I put a brand new WD20EARS and setup unRaid 4.7 for the first time, I didn't even have to preclear or even clear the drive.  I just formatted the drive and it worked.  Total time from disk insert to server start in less than 5min.  Is that supposed to happen?  I didn't even know about preclears until after reading this board.  I went back and precleared the drive just as a burn-in test.  But the fact remains that I never had to clear the drive at all.  Just a quick format in unRaid and it worked.

 

Ya, in a new array unRAID just builds the parity based on the existing bits stored on the disks. Any "leftover" existing 1 bits are accounted for when the parity drive is written. This is why you had  that 7 or 8 hours of parity drive writing when you first started the array.

 

Peter

 

 

I am confused, "yelnatsch517" never stated he had 7-8 hours of parity writing when he first started his array? He stated "But the fact remains that I never had to clear the drive at all.  Just a quick format in unRaid and it worked."

 

Interesting...

 

The first time I put a brand new WD20EARS and setup unRaid 4.7 for the first time, I didn't even have to preclear or even clear the drive.  I just formatted the drive and it worked.  Total time from disk insert to server start in less than 5min.  Is that supposed to happen?  I didn't even know about preclears until after reading this board.  I went back and precleared the drive just as a burn-in test.  But the fact remains that I never had to clear the drive at all.  Just a quick format in unRaid and it worked.

 

Ya, in a new array unRAID just builds the parity based on the existing bits stored on the disks. Any "leftover" existing 1 bits are accounted for when the parity drive is written. This is why you had  that 7 or 8 hours of parity drive writing when you first started the array.

 

Peter

 

 

I am confused, "yelnatsch517" never stated he had 7-8 hours of parity writing when he first started his array? He stated "But the fact remains that I never had to clear the drive at all.  Just a quick format in unRaid and it worked."

 

 

Raj had it correct.  I started it without a parity drive which completely makes sense since the sole purpose of preclear is to preserve the integrity of the parity drive and if I didn't have one, then it wouldn't be required.

 

Sorry about the confusion.

If you had no parity drive installed then that is normal.  Likewise if you installed the drive as a cache drive then that is normal.  If you did have a parity drive installed and weren't installing the new drive as a cache drive, then that is not normal.

 

So is it recommend to make sure you start with adding a Parity Drive first before proceeding to add any data disks (non cache), bring up the array, stop the array then add the data disks which would clear each one (time consuming). OR if you are starting off fresh (completely new build, MB, Drives, etc...) and lets say you ran preclear_disk on all your new drives as a personal preference to make sure they are good to then (first) add them in as data disks, then bring up the array then stop the array and ONLY then put the parity drive in (being last) in order to skip the unRaid clear process of each data disk?

 

My recommendation is to preclear all new disks no matter what with at least 2 passes.  Not only will this test out the new disks, but your new server hardware as well.  It will also allow you to skip the lengthy unRAID clearing process (though to be fair in this scenario preclear will take much longer than unRAID's default clearing cycle, and you will have to wait longer before you have access to your server.  The 'minimal downtime' benefit of preclear only comes into play when adding a drive to an existing array).

 

The order in which you install the parity drive won't matter.  It will still take 8+ hours to calculate parity.  It doesn't matter if the drives are precleared or not.  The array is usable during a parity sync, but you might see slow performance.

 

That actually gives me an idea...in this scenario of a completely new server with all precleared disks, is it possible to tell unRAID that every drive, including the parity drive, contains all zeros and therefore parity is 'instantly' valid (after all, an infinite number of zeros will always return even parity, right)?  Is this a bad idea?  Still, it seems like if you have already put several days into preclearing a bunch of new drives, then it would be nice to be able to skip that final 8+ hours of parity sync....

Geeze, on a new array you have to start the array and allow the parity to buid. So, the preclear does not matter one bit to array setup speed. In this case, the preclear is just used to burn-in the disks and look for disks that could prematurely fail. A new array will always be usable immediately even though parity is being built.

 

On an existing array you have to run the preclear to add a disk so unRAID does not go off-line and clear it. If you do not add a properly pre-cleared disk then unRAID will go off-line and clear the disk.

 

 

That actually gives me an idea...in this scenario of a completely new server with all precleared disks, is it possible to tell unRAID that every drive, including the parity drive, contains all zeros and therefore parity is 'instantly' valid (after all, an infinite number of zeros will always return even parity, right)?  Is this a bad idea?  Still, it seems like if you have already put several days into preclearing a bunch of new drives, then it would be nice to be able to skip that final 8+ hours of parity sync....

 

I would think you could use the "trust my array" procedure and it might work OK, just stop the parity check after you hit the start button. You would have to test it by getting the array started and then doing another parity check to see if there are any errors.

 

Peter

 

Geeze, on a new array you have to start the array and allow the parity to buid. So, the preclear does not matter one bit to array setup speed. In this case, the preclear is just used to burn-in the disks and look for disks that could prematurely fail. A new array will always be usable immediately even though parity is being built.

 

On an existing array you have to run the preclear to add a disk so unRAID does not go off-line and clear it. If you do not add a properly pre-cleared disk then unRAID will go off-line and clear the disk.

 

 

That actually gives me an idea...in this scenario of a completely new server with all precleared disks, is it possible to tell unRAID that every drive, including the parity drive, contains all zeros and therefore parity is 'instantly' valid (after all, an infinite number of zeros will always return even parity, right)?  Is this a bad idea?  Still, it seems like if you have already put several days into preclearing a bunch of new drives, then it would be nice to be able to skip that final 8+ hours of parity sync....

 

I would think you could use the "trust my array" procedure and it might work OK, just stop the parity check after you hit the start button. You would have to test it by getting the array started and then doing another parity check to see if there are any errors.

 

Peter

 

 

Nice idea, Peter!  I could see doing this followed by a parity check, and if it returns 0 sync errors being good to go.  The normal array build process requires a build followed by a check to know you are protected.

 

But there is still a lingering feeling that the strenous parity build has value in proving that your system as a whole is working well, and that parity is able to be reliably maintained.  The exact same process is used to do a drive rebuild, so in a way you are proving a drive rebuild would work on your system.

 

 

Given that I normally preclear 6 drives simultaneously in different slots and on different controllers on all my builds, I'm not sure if a parity build is really that much more strenuous.  If you are building parity across more than 6 drives then I could see that as a valid argument.  I know they are fundamentally different operations (preclear being intra-disk, and parity build being inter-disk), but they seem pretty comparable to me in terms of testing new hardware.  Obviously spending the extra 8 hours to build parity the traditional way is worth it if it actually does something to test the hardware that simultaneous preclears and then a subsequent parity check does not.  I'm open to discuss the point, I haven't made up my mind either way.  Or maybe I just need to test it out a bit...I'll at least out Peter's idea as a point of curiosity.

 

In all honestly, shaving 8 hours off my 2 week burn in process for client builds is pretty negligible in the end.  Still, those power bills do add up eventually...

Given that I normally preclear 6 drives simultaneously in different slots and on different controllers on all my builds, I'm not sure if a parity build is really that much more strenuous.  If you are building parity across more than 6 drives then I could see that as a valid argument.  I know they are fundamentally different operations (preclear being intra-disk, and parity build being inter-disk), but they seem pretty comparable to me in terms of testing new hardware.  Obviously spending the extra 8 hours to build parity the traditional way is worth it if it actually does something to test the hardware that simultaneous preclears and then a subsequent parity check does not.  I'm open to discuss the point, I haven't made up my mind either way.  Or maybe I just need to test it out a bit...I'll at least out Peter's idea as a point of curiosity.

 

In all honestly, shaving 8 hours off my 2 week burn in process for client builds is pretty negligible in the end.  Still, those power bills do add up eventually...

 

You guys building servers are the ones that would really benefit the most.  I would not suggest the "trust my parity" as the first thing a newbie would do.  I also don't think it makes sense for Tom to spend time on implementing special functionality to detect the "all disks are precleared" condition and skipping the build altogether.  This is just too infrequent a situation, and there are benefits of the parity build for new users on new hardware.  The parity build mimics the rebuild - and even if all you are doing is practicing the rebuild, it is valuable.

 

But for professionals like you guys, that know the motherboards are compaibile, and you are preclearing on the same motherboards, I would think skipping the parity build and shaving 8 hours off the build process would be a viable option. But if you get in a mode that you are preclearing on a dedicated preclear machine, I think the parity build should stay.

Given that I normally preclear 6 drives simultaneously in different slots and on different controllers on all my builds, I'm not sure if a parity build is really that much more strenuous.  If you are building parity across more than 6 drives then I could see that as a valid argument.  I know they are fundamentally different operations (preclear being intra-disk, and parity build being inter-disk), but they seem pretty comparable to me in terms of testing new hardware.  Obviously spending the extra 8 hours to build parity the traditional way is worth it if it actually does something to test the hardware that simultaneous preclears and then a subsequent parity check does not.  I'm open to discuss the point, I haven't made up my mind either way.  Or maybe I just need to test it out a bit...I'll at least out Peter's idea as a point of curiosity.

 

In all honestly, shaving 8 hours off my 2 week burn in process for client builds is pretty negligible in the end.  Still, those power bills do add up eventually...

 

You guys building servers are the ones that would really benefit the most.  I would not suggest the "trust my parity" as the first thing a newbie would do.  I also don't think it makes sense for Tom to spend time on implementing special functionality to detect the "all disks are precleared" condition and skipping the build altogether.  This is just too infrequent a situation, and there are benefits of the parity build for new users on new hardware.  The parity build mimics the rebuild - and even if all you are doing is practicing the rebuild, it is valuable.

 

But for professionals like you guys, that know the motherboards are compaibile, and you are preclearing on the same motherboards, I would think skipping the parity build and shaving 8 hours off the build process would be a viable option. But if you get in a mode that you are preclearing on a dedicated preclear machine, I think the parity build should stay.

I agree.  I would never want to wait until a true re-construction is needed to learn of an issue that prevents a reconstruction. (a reconstruction being 99% identical to a parity sync with the only difference being the disk being written to.