Preclear plugin


Recommended Posts

It should also be noted that even with Preclear disk not properly loading, it is still leaving behind traces of its failed install which may cause issues for other plugins.  EG: other plugins & dynamix itself utilize the default dynamix CSS, but preclear disk for some reason overrides that style sheet instead of utilizing its own style sheet which has undesired effects on the displays under certain circumstances.  (This would also happen if the plugin installed properly)

 

Since the plugin doesn't even install correctly at this point in time, I would recommend everyone who has updated to 6.4.1 to delete preclear.disk.plg from /config/plugins on the flash drive to eliminate the possibility of any stray code entering the system.

Edited by pluginCop
Link to comment
On 2/4/2018 at 5:26 AM, loz678 said:

is there an alternative to the pre-clear plugin?

Yes, there is.  It is Joe L.'s preclear script which you can find here:

 

       https://lime-technology.com/forums/topic/2732-preclear_disksh-a-new-utility-to-burn-in-and-pre-clear-disks-for-quick-add/

 

 

 I don't believe that it is being actively supported at this time.  However, Joe L. did issue a patch which allows it to work on recent releases.  You can find that here.

 

        https://lime-technology.com/forums/topic/12391-re-preclear_disksh-a-new-utility-to-burn-in-and-pre-clear-disks-for-quick-add/?page=53#comment-460592

 

and a second one that is required for (atleast ) 6.4.1 and later unRAID versions:

 

       https://lime-technology.com/forums/topic/12391-re-preclear_disksh-a-new-utility-to-burn-in-and-pre-clear-disks-for-quick-add/?page=54&tab=comments#comment-632319

 

You can use the new Terminal window (use the right Mouse button in the Terminal window) in version  6.4.1 to apply the mod.  I would suggest that you copy and paste that sed command.  (sed is a very powerful UNIX editing tool and it does not give much feed back so be careful!   Footnote:  sed was written back in the day when Teletype machines were still being used as terminals so the less the output the quicker (and quieter) things were.)   

 

 

Using it is a bit of a pain but read (and print) the first post, you will have the basics.  For a more complete set of instructions, you can go to this Wiki:

 

      https://wiki.unraid.net/Configuration_Tutorial#Preclear_Hard_Drives

 

I would suggest that you print it out also.  Having both of these in hand will make it relatively simple to use the script.  

 

Edit 1:   Modified Link to Configuration Tutorial to point to Preclear Section.

 

Edit 2:  Added link to substitute strings-BDS for strings  which is not in release 6.4.1.

 

Edit 3:  Fixed link to Preclear WIKI

Edited by Frank1940
Modified Link to Configuration Tutorial to point to Preclear Section.
  • Upvote 1
Link to comment
31 minutes ago, wgstarks said:

I did a little testing with Joe L's modded script. It still works with 6.4. You won't get any sort of progress in the unraid webUI of course, only in the screen session. If you have UD installed you will see the file system change to "precleared" once the test is complete.

 

Good to know that it still works with version 6.4.  That saves me the bother of having to run a test using a 3TB drive. 

Link to comment

Thanks Frank1940 - i need to run preclears and this saved me a lot of digging to re-enable joes script.   Fingers crossed that this plugin will live again in the near future!

 

I feel the following is too important to not post here as well:

So Disk2 which has been in use for a few years now was kicked offline in the array... but the disk has no negative smart report items.

 

I moved the Disk2 to another slot on the backplane, set the array to use NONE in disk2, used MC to move the emulated Disk2 data to disk3 and disk4, then began a preclear on Disk2 (which was no longer physically assigned to the array) 

 

Running a preclear on this disk has given me the confidence that the physical port of my backplane has faulted and not the physical disk.

 

To the 6.4.1 naysayeers... Preclear is a critical component of Unraid.  It is great to have as a plugin, but will use screen and terminal if the Joe L script is all we are allotted. 

 

I personally do not feel comfortable just tossing a bare drive into the array without a few cycles of passes... and have had a goodly portion of drives fail a preclear before ever touching my array.

 

In addition, any data drive that is taken OUT of commission has a preclear run on it to destroy data prior to recycling or sale. --- so long as it is physically able to run. 

Edited by landS
Link to comment
On 2/4/2018 at 7:15 AM, Frank1940 said:

Using it is a bit of a pain but read (and print) the first post, you will have the basics.  For a more complete set of instructions, you can go to this Wiki:

 

      http://lime-technology.com/wiki/Configuration_Tutorial#Preclear_Hard_Drives

 

I would suggest that you print it out also.  Having both of these in hand will make it relatively simple to use the script.  

 

 

Funnily enough, that link recommends using the preclear plugin on unRAID v6 systems...

Link to comment
On 2/4/2018 at 8:15 AM, Frank1940 said:

Yes, there is.  It is Joe L.'s preclear script which you can find here:

 

       https://lime-technology.com/forums/topic/2732-preclear_disksh-a-new-utility-to-burn-in-and-pre-clear-disks-for-quick-add/

 

 I don't believe that it is being actively supported at this time.  However, Joe L. did issue a patch which allows it to work on recent releases.  You can find that here.

 

On 2/4/2018 at 5:31 AM, dlandon said:

It's not really necessary.

 

On 2/4/2018 at 5:30 AM, bonienl said:

Many users live happily without the preclear plugin...

 

 

The plugin isn't supported, the script isn't supported and many users live happily without the preclear plugin?  Can someone explain and recommend what the process should be to a new or relatively new unRAID user like myself?   Is the consensus of developers and or recommendation from @limetech not to run a test on new drives before adding them to the array?    Or is the consensus to hold off using the preclear plugin and script until they are updated and ready for use?   

 

Sorry, but I'm a little confused as there are numerous forum posts, wiki instructions, videos, etc. recommending and indicating how important it is to use the preclear plugin and or running a script before adding a disk to the array.  If those recommendations and or processes have changed, shouldn't someone update the documentation, videos, etc. to indicate what the new recommended process is?   Isn't the purpose of plugins to make certain tasks easier and more foolproof for the end user?  Keep in mind that some users might not be as comfortable navigating and or running command line type scripts.

 

Please don't take my comments / questions the wrong way, I appreciate everything that the community, developers and @limetech do... I'm just looking for some clarity and direction in regards to what the new process should be.

Link to comment

@luisv, the preclear function is strictly optional.  It is not required for your server to run properly.  If the drive requires clearing, the unRAID OS WILL clear the disk before it is put into use.

 

Now a quick bit of history.  Several generations of unRAID version ago, the array was not mounted while the preclear the operation was being performed.  This probably was not to long a period when drive capacities were small but as drive captivity increased so did the time when the array was offline until the preclear was finished.  It was proposed and a shell script was written to do this prior to adding the drive to array which reduced the time from a many hours to couple of minutes.  

 

Now for the tricky part.  Disk drives have a failure curve that follows the 'bathtub failure' mode (Google for details)  and it was decided that doing a bit of testing would probably catch many cases of infant morality on a drive before it was placed in service.  (The first couple of computers that I purchased were burned in for 72 hours before the vendor would ship it to prevent returns for this class of failures.  I don't think that many vendors are doing this today.)  Thus, the preclear script evolved...

 

It is really your choice.  Many of us old timers would really like some assurance that the drive is not going to fail a couple of days after it goes into service.  (Just read the forums and the number of questions from users when a drive does have problems and some of the situations which folks have gotten themselves into when they panic a bit!)  How many defective (or soon to be defective) drives are caught by running a few preclear cycles.  I would say it is probably less than 1%.  BUT, If you should get one of the one-percent, would you prefer to deal with the issues before or after you put it in an array?  Your choice. 

Edited by Frank1940
  • Like 1
  • Upvote 1
Link to comment
6 minutes ago, luisv said:

 

 

 

 

The plugin isn't supported, the script isn't supported and many users live happily without the preclear plugin?  Can someone explain and recommend what the process should be to a new or relatively new unRAID user like myself?   Is the consensus of developers and or recommendation from @limetech not to run a test on new drives before adding them to the array?    Or is the consensus to hold off using the preclear plugin and script until they are updated and ready for use?   

 

Sorry, but I'm a little confused as there are numerous forum posts, wiki instructions, videos, etc. recommending and indicating how important it is to use the preclear plugin and or running a script before adding a disk to the array.  If those recommendations and or processes have changed, shouldn't someone update the documentation, videos, etc. to indicate what the new recommended process is?   Isn't the purpose of plugins to make certain tasks easier and more foolproof for the end user?  Keep in mind that some users might not be as comfortable navigating and or running command line type scripts.

 

Please don't take my comments / questions the wrong way, I appreciate everything that the community, developers and @limetech do... I'm just looking for some clarity and direction in regards to what the new process should be.

Opinions on this issue a split.

 

One opinion is that preclears are the best way to test a drive by putting severe read/write stresses on it and see if it survives. The drawback is that the 3 cycles (the usual recommended minimum) of preclears will take a long time. IIRC the last 8TB drive I precleared took almost 2 weeks for 3 cycles. Can’t remember for sure, but more than a week anyway. This would mean that an existing array with a failed drive could be waiting for a replacement and at risk of data lose for an extended period while the new drive was being precleared if you don’t keep a spare drive on standby (hot spare).

 

The other opinion is that preclears aren’t needed since unRAID will now clear new drives as part of the process of adding them to the array. UnRAID supports SMART and an extended SMART test will reveal any defects with a new drive. This process is much faster than preclearing (still probably measured in days though for large drives) and can all be done while the array is online just like preclearing.

 

If you search the forum you’ll find a few discussions of preclear vs smart. I’m not qualified to offer any opinion on that. In the end it’s your decision which is best suited for your needs. In my case if I were building a new machine and stocking it with new drives I’d probably go ahead and preclear them since there wouldn’t be a rush to get the array online. This would be especially true if I was shucking the drives from external enclosures since many of them still don’t support SMART and once shucked you won’t be able to RMA them. If I had an existing array with a failed drive I’d probably skip the preclear to save time in this unprotected condition and just trust SMART.

  • Like 1
Link to comment
23 minutes ago, johnnie.black said:

The script is supported, just needs to be patched, though there are multiple other disk tools to burning disks before use for users that want to do that, badbocks for example which is also included with unRAID.

 

I'm certainly not a seasoned unRAID user, but this is the first I'm hearing about badblocks, so that's part of the frustration and confusion.  If there are multiple ways to perform this, what are they?

Link to comment
9 minutes ago, Frank1940 said:

@luisv, the preclear function is strictly optional.  It is not required for your server to run properly.  If the drive requires clearing, the unRAID OS WILL clear the disk before it is put into use.

 

Now a quick bit of history.  Several generations of unRAID version ago, the array was not mounted while the preclear the operation was being performed.  This probably was not to long a period when drive capacities were small but as drive captivity increased so did the time when the array was offline until the preclear was finished.  It was proposed and a shell script was written to do this prior to adding the drive to array which reduced the time from a many hours to couple of minutes.  

 

Now for the tricky part.  Disk drives have a failure curve that follows the 'bathtub failure' mode (Google for details)  and it was decided that doing a bit of testing would probably catch many cases of infant morality on a drive before it was placed in service.  (The first couple of computers that I purchased were burned in for 72 hours before the vendor would ship it to prevent returns for this class of failures.  I don't think that many vendors are doing this today.)  Thus, the preclear script evolved...

 

It is really your choice.  Many of us old timers would really like some assurance that the drive is not going to fail a couple of days after it goes into service.  (Just read the forums and the number of questions from users when a drive does have problems and some of the situations which folks have gotten themselves into when they panic a bit!)  How many defective (or soon to be defective) drives are caught by running a few preclear cycles.  I would say it is probably less than 1%.  BUT, If you should get one of the one-percent, would you prefer to deal with the issues before or after you put it in an array?  Your choice. 

 

 

Thanks for the history and details, much appreciated.  I understand it's not needed for unRAID to function, but I for one would like to at least have some sort of warm and fuzzy feeling that a drive was tested and therefore shouldn't fail shortly after placed into operation.  I know it might not be 100% perfect, and placebo or not, it provides a warm and  fuzzy feeling. 

Link to comment
7 minutes ago, luisv said:

If there are multiple ways to perform this, what are they?

There multiple ways and utilities for testing disks including preclear, but all are optional, clearing a disk is only mandatory when it's added as a new disk to a parity protected array, and unRAID clears the disk in the background since v6.2, it you want to burn in the disk you can use preclear, badblocks, the disks own testing tools like WD lifeguard, Seatools, etc.

  • Like 1
Link to comment
15 minutes ago, wgstarks said:

Opinions on this issue a split.

 

One opinion is that preclears are the best way to test a drive by putting severe read/write stresses on it and see if it survives. The drawback is that the 3 cycles (the usual recommended minimum) of preclears will take a long time. IIRC the last 8TB drive I precleared took almost 2 weeks for 3 cycles. Can’t remember for sure, but more than a week anyway. This would mean that an existing array with a failed drive could be waiting for a replacement and at risk of data lose for an extended period while the new drive was being precleared if you don’t keep a spare drive on standby (hot spare).

 

The other opinion is that preclears aren’t needed since unRAID will now clear new drives as part of the process of adding them to the array. UnRAID supports SMART and an extended SMART test will reveal any defects with a new drive. This process is much faster than preclearing (still probably measured in days though for large drives) and can all be done while the array is online just like preclearing.

 

If you search the forum you’ll find a few discussions of preclear vs smart. I’m not qualified to offer any opinion on that. In the end it’s your decision which is best suited for your needs. In my case if I were building a new machine and stocking it with new drives I’d probably go ahead and preclear them since there wouldn’t be a rush to get the array online. This would be especially true if I was shucking the drives from external enclosures since many of them still don’t support SMART and once shucked you won’t be able to RMA them. If I had an existing array with a failed drive I’d probably skip the preclear to save time in this unprotected condition and just trust SMART.

 

Totally understandable and agreed, I'm not qualified to accurately articulate the differences nor say which is better, a preclear and smart, but  shouldn't there be some guidelines and or recommendations?  Just as it was highly recommended to run the preclear script and or plugin, shouldn't there be replacement as a direction for new users that wish to perform some sort of test?

Link to comment
27 minutes ago, wgstarks said:

One opinion is that preclears are the best way to test a drive by putting severe read/write stresses on it and see if it survives. The drawback is that the 3 cycles (the usual recommended minimum) of preclears will take a long time. IIRC the last 8TB drive I precleared took almost 2 weeks for 3 cycles. Can’t remember for sure, but more than a week anyway.

 

For another opinion I'd never consider running more than one preclear cycle or waiting more than than the 45 hours it takes for a single preclear cycle to complete on an 8 TB drive on my system.

 

In my view the perceived benefits of multiple preclear cycles outweigh the cost but I agree that everyone has to decide for themselves the degree to which they want to pursue this sort of piece of mind.

Link to comment
7 minutes ago, johnnie.black said:

There multiple ways and utilities for testing disks including preclear, but all are optional, clearing a disk is only mandatory when it's added as a new disk to a parity protected array, and unRAID clears the disk in the background since v6.2, it you want to burn in the disk you can use preclear, badblocks, the disks own testing tools like WD lifeguard, Seatools, etc.

 

2 minutes ago, bonienl said:

 

And because it is optional, you will see two camps. There is no right or wrong here, you decide what works best for you.

 

 

No sure and I understand optional, no right nor wrong.   Some folks might not have another system to attach multiple drives to perform tests like WD Lifeguard, etc, so within the system that will or is running unRAID what are those options.  Since preclear, plugin or script, was the optional standard and recommendation, whats the new one?

 

Link to comment

My mode of operation is to use the standard clear function of unRAID, which writes all zeroes to the disk, then bring the disk online (format) and run a non-correcting parity check, which does a complete read of the new disk. If at this point I don't receive any SMART notifications, I consider the disk good to go.

 

Link to comment
17 minutes ago, luisv said:

 

Totally understandable and agreed, I'm not qualified to accurately articulate the differences nor say which is better, a preclear and smart, but  shouldn't there be some guidelines and or recommendations?  Just as it was highly recommended to run the preclear script and or plugin, shouldn't there be replacement as a direction for new users that wish to perform some sort of test?

The preclear script and plugin are both user provided and maintained. They are not provided by limetech and to my knowledge have never been recommended by limetech. Limetech does offer the ability to clear the drives, which is all that is required. The rest is user preference. If you prefer to do preclears then the script still works although the creator hasn’t been offering any support recently.

Link to comment
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.