Preclear plugin


3042 posts in this topic Last Reply

Recommended Posts

22 minutes ago, Sean M. said:

 

Not 100% sure the disk is okay, part of the reason for the pre-clear.  It came from an old desktop that had been decommissioned for a bit, last I remember it was working fine though.

 

Full copy of diagnostics and pre-clear logs attached. 

 

The drive in question is: ST33000651AS_9XK0AKZH

Preclear.disk-20180429-0756.zip

Diagnostics-20180429-0756.zip

 

SMART is disabled on that disk. Enable it.

 

smartctl -s /dev/sdX

 

If you haven't rebooted since that diagnostics then sdX = sdb

Link to post
  • Replies 3k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Hi guys, this is a simple plugin that allows users to clear their disks before add them to the array.   The main characteristics of this plugin are: Modularity: can be used standalon

I think Thanks should be directed at all those members that have been dealing with the mess the last few months. @dlandon and @Squid have been working behind the scenes to try and clean up, and o

Dontlookoverthere over on the unRAID subreddit figured out how to get the plugin running again with a simple edit to the plg:   https://www.reddit.com/r/unRAID/comments/7wjpus/preclear_worki

Posted Images

Just now, Frank1940 said:

 

A disk that was working fine in an other computer may have surface defects (in an unused area of that disk) that would render it unusable for unRAID!  

 

For some reason, the SMART attributes were not reported for this disk.  It appears that their reporting was turned off for some reason. Perhaps, someone else might have some inslight into what your next course of action might be.

 

Ok guys, there's an error in the 0.9.5 version of the script causing this anomaly. It should copy zeroes to the entire disk, but dd is skipping the last 10 (2097152 bytes sized) blocks of the disk, leaving it in an inconsistent state. I'm trying to fix it right now, but every change need 24 hours to be tested on my test disk, so it will take some time to debug it.

Link to post
On 3/4/2018 at 9:06 AM, dlandon said:

Unfortunately this is probably a short term fix at best for several reasons:

  • It is not a necessary to clear disks with preclear.  unRAID handles clearing a disk without taking the array offline as in the past.
  • This plugin has been tagged as not compatible with 6.4 and later versions of unRAID.
  • There are style sheets in the plugin that replace some of the built in unRAID style sheets and break other things.
  • The 6.5 series of releases has warnings from php 7.2 turned off for the moment.  Those of us actively supporting plugins are fixing these warnings in our plugins now.  Eventually as the next versions of php are released, the warnings will be errors and preclear may fail unless the plugin code is actively supported.  Changes may have to be made to address the php errors.  Right now the warnings are an annoyance if turned on and appear in the webgui, later they will be failures.
  • Some changes need to be made to address issues that some users have experienced and there is no one actively supporting the plugin to make these changes.
  • I doubt any other plugin author will adopt this plugin because it is not necessary for unRAID operation.  Those wishing to pound on a disk just to make yourself feel good can find other tools to do the same thing.  It is my understanding that the preclear script is still working.
  • It has been more trouble that it is worth.
  • As the supporter of UD, I can no longer offer any assistance or support for the plugin and don't appreciate the kludge offered here because I will end up with support requests I don't have the time or patience to deal with.
  • UD will eventually be incorporated into the unRAID core and any preclear hooks in UD will probably be removed.

I know those of you that carry the preclear banner will not like what I have said here, but preclear is on it's way out.  Find another way!

 

Not to derail the current conversation about the latest Preclear plugin working for v6.5, but I want to comment that there is some conflicting advice on whether to preclear or not. It seems that some senior members of the community such as dlandon and Squid feel that using this Preclear plugin may not be necessary and sometimes harmful. But then you have official Unraid documentation in the wiki that encourages users to preclear (https://lime-technology.com/wiki/Replacing_a_Data_Drive).

  • Quote

    If you have purchased a replacement drive, we always recommend Preclearing the replacement drive first, to make sure it's a good drive that won't fail for a few years at least. The Preclearing is not strictly necessary, as replacement drives don't have to be cleared, they are going to be completely overwritten. But Preclearing new drives one to three times provides a thorough test of the drive, eliminates 'infant mortality' failures.

     

Then there are those that say avoid the plugin and just use the script, but even that may require patches.

 

There has obviously been a lot of hard work put in by these developers like gfjardim, and I'm very appreciative of their work. I've precleared all my drives before putting them in my array. But as a user that doesn't visit the Unraid forums often, it's hard to keep up with what the latest advice and caution is. It would be nice if Unraid updated their documentation on the wiki or made it very clear what their stance was on preclearing.

Link to post
3 minutes ago, limefrog said:

It would be nice if Unraid updated their documentation on the wiki or made it very clear what their stance was on preclearing.

 

While I do not speak for LimeTech, I think one may safely infer that they are neutral on the subject.  They do not feel that testing of new drives is necessary.  Otherwise, they would be obligated to include a testing module as a part of the basic unRAID system.  And 99%+ of the time, that is correct.  Pretesting of hard drives are not required if they don't fail in the near term.  (All Hard drives will eventually fail and it is only a matter of when...)

 

However, what is your comfort level if you happen to get one of those outliers that fail in less then a week after you put the drive into your system?  If you find this unacceptable, then you should be preclearing.  If you are comfortable dealing with the problem, you scan skip oit.  

Link to post
11 minutes ago, Frank1940 said:

While I do not speak for LimeTech, I think one may safely infer that they are neutral on the subject.

 

And much of the wiki is maintained by the user community. Not sure preclear is ever mentioned in the "official wiki". I don't recall anyone from limetech ever commenting on preclear.

 

Probably there was some reluctance to even give out the "magic cookie" information that unRAID uses to recognize a cleared disk, since letting others use that functionality could result in code out of limetech's control potentially letting disks into the array that wouldn't be consistent with parity and so could cause data loss.

 

More likely limetech never gave out that information and it was reverse-engineered by Joe L. I wasn't around then.

Link to post
2 minutes ago, Squid said:

I've never said its harmful.

 

My apologies on the poor english. Should have just written that you and dlandon have suggested it may not be necessary and that dlandon suggested it may be more harmful than good.

Link to post
6 minutes ago, trurl said:

More likely limetech never gave out that information and it was reverse-engineered by Joe L. I wasn't around then.

 

I recall reading  that @Joe L. was in contact with Tom at the time when he was writing the preclear script and Tom gave @Joe L. the 'magic cookie' on a confidential basis.  (Tom has been known in the past to provide people with proprietary information and trust they will not reveal it to the general public.)   I think that Tom (and LimeTech) feels that the majority of unRAID users are trustworthy and will bend over backwards to provide them with any information and service necessary to further the usability and functionality of unRAID.  He does not seem to have a 'Not invented here' type of mentality toward innovation.  

Link to post
 
And much of the wiki is maintained by the user community. Not sure preclear is ever mentioned in the "official wiki". I don't recall anyone from limetech ever commenting on preclear.
 
Probably there was some reluctance to even give out the "magic cookie" information that unRAID uses to recognize a cleared disk, since letting others use that functionality could result in code out of limetech's control potentially letting disks into the array that wouldn't be consistent with parity and so could cause data loss.
 
More likely limetech never gave out that information and it was reverse-engineered by Joe L. I wasn't around then.
It was never reverse engineered, the md driver was always open source.

Guys , let's be straight: adding new functionality is always an opportunity to inadvertently introduce some bugs into the code. Some fixes I made (e.g. fail dd in case of read errors) led to an odd behavior. I'd try to circumvent this, but it takes some time.

Enviado de meu SM-G955F usando Tapatalk

Link to post
12 minutes ago, limefrog said:

 

My apologies on the poor english. Should have just written that you and dlandon have suggested it may not be necessary and that dlandon suggested it may be more harmful than good.

Actually, in retrospect I did say that.  Rather than relying on the built-in css from dynamix, the preclear plugin redefines some css which does have an adverse effect on certain aspects of CA (ie: CA et al looks slightly different depending upon if Preclear is installed or not.  And a uninstall is not enough to fix those problems - a reboot is necessary)

 

I'm not a big believer of:

  • Redefining stock elements of unRaid that can have an adverse effect on unRaid and/or other plugins
  • Injecting code into every single page of unRaid's UI including plugins.
Edited by Squid
Link to post
6 hours ago, gfjardim said:

 

Ok guys, there's an error in the 0.9.5 version of the script causing this anomaly. It should copy zeroes to the entire disk, but dd is skipping the last 10 (2097152 bytes sized) blocks of the disk, leaving it in an inconsistent state. I'm trying to fix it right now, but every change need 24 hours to be tested on my test disk, so it will take some time to debug it.

 

Just been trying to preclear a drive using 0.9.5 version of the script and 2018.04.24 version of the plugin. I have been getting back an error: 

 

--> FAIL: Post-Read verification failed. Your drive is not zeroed.

 

Is that in relation to the issue you have mentioned there? I was about to run a memtest to see if I had any other issues at play.

 

Thanks

Link to post

I just started to use the sript, but when i start a preclear, i get the following error message:

 

./preclear_disk.sh: line 1307: strings: command not found
cat: write error: Broken pipe
sfdisk: invalid option -- 'R'
Try 'sfdisk --help' for more information.
Sorry: Device /dev/sdg is busy.: 1

 

Any idea how I can fix this?

Link to post
 
Just been trying to preclear a drive using 0.9.5 version of the script and 2018.04.24 version of the plugin. I have been getting back an error: 
 
--> FAIL: Post-Read verification failed. Your drive is not zeroed.
 
Is that in relation to the issue you have mentioned there? I was about to run a memtest to see if I had any other issues at play.
 
Thanks
Yep, that's the one.

Enviado de meu SM-G955F usando Tapatalk

Link to post
26 minutes ago, gfjardim said:

Yep, that's the one.

Enviado de meu SM-G955F usando Tapatalk
 

 

Lovely stuff, glad that I am not going to have to run through all potential hardware issues! 

 

Is that issue fixed in the latest version of the plugin - 2018.04.27

 

Thanks

Link to post
 
Lovely stuff, glad that I am not going to have to run through all potential hardware issues! 
 
Is that issue fixed in the latest version of the plugin - 2018.04.27
 
Thanks
Nope, still testing it and will release it later this week.

Enviado de meu SM-G955F usando Tapatalk

Link to post
9 hours ago, trurl said:

 

SMART is disabled on that disk. Enable it.

 

smartctl -s /dev/sdX

 

If you haven't rebooted since that diagnostics then sdX = sdb

 

Enabled smart, ran a short test, downloaded (attached).

 

I saw gfjardim mentioned there was an issue with the plugin so I'll probably just wait and re-try once that's updated as I'm in no rush to move this drive over the array.

Smart-20180429-1747.zip

Link to post

I attempted running the two most recent ver of the plugin on two new drives and both failed step 5.  Swapped cables with working drives and still failed the same.  I attempted just to add one of the drives to the array due to the above noted issues w/ the plugin for my ver of unraid 6.4.x. I get a "unmountable no file system" error for the disk I had attempted preclear on.  So I removed the drive from the array and put it into unassigned disks.   Should I wait it out for the plugin update or is there a better course of action?  The array says it is missing that disk now but I never wrote to it or ran a parity check with it so it seems like it would be fine to leave as is.

 

Thanks

 

Justin

Link to post
1 minute ago, johnnie.black said:

Thanks for your prompt response, it works.

It is difficult to follow all this stuff if you get out of touch for a few months. Does the same patch work on the bjp666 script? i.e. could i try 

sed -i -e "s/print \$9 /print \$8 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_bjp.sh

instead of 

sed -i -e "s/print \$9 /print \$8 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

?

Link to post
On 4/30/2018 at 4:18 PM, johnnie.black said:

 

Edit2 - Decided not to muddy the water any more, if anybody wants to try the older bjp999 faster preclear script patched for unraid v6.5.1 (not tested at present) then please go here, any responses please keep them in that thread not this one.

 

script in this post is now fixed, so please use this one.

 

Edited by binhex
Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.