Jump to content
gfjardim

Preclear plugin

2439 posts in this topic Last Reply

Recommended Posts

So I attempted to preclear 24 HGST 4TB SAS drives at once.  One thing I noticed during the zero'ing phase, the web gui hangs.  I thought unraid crashed and rebooted. I started zero'ing the drives after the reboot. I noticed after about 8 drives, the web gui starts to get slower and slower till it's unresponsive.  I decided to do 12 drives at once instead of 24. 

 

System Specs:
i7-5820k

64gb ram

Areca 1882ix-16

SuperMicro CSE-846 used as HDD enclosure. 

 

I wanted to run a few cycles through these drives, because these are datacenter pulls and I want to be sure these drives are good before putting data on them. 

 

Also, is there any way to just opt in or out when sending usage reports instead of it constantly popping up a dialog asking to send the usage reports? When pre-clearing 24 drives the usage report pop up gets quite annoying. 

Share this post


Link to post

I've been having issues trying to preclear multiple drives at once. Couple nights ago, I managed to successfully preclear 12 drives with 1 cycle. 

Once that finished, I started  another 11 drives. During the Post read phase, one drive timed out and the raid controller failed it. PreClear also failed the rest of the drives due to the one drive timing out.  I'm still not sure if preclear caused it or the raid controller.  But the drive appears fine.

 

So in order to complete the cycle on all the drives that stopped/failed. I used the verify option thinking, it was going to do a pre-read and post read. But after the pre read phase, it started zero'ing the drives again, which is not what I wanted. 

 

This plugin is not stable enough for me to clear a bunch of drives at once.  

 

An enhancement request I have, is to be able to select multiple drives at once to pre-clear.  It takes me close to 10mins to click through 24 drives to select the options and start the pre-clear. Not to mention the massive gui slowdown/stall the more drives you pre-clear at once.

 

 

Share this post


Link to post
On 12/16/2017 at 11:05 AM, lordbob75 said:

Good morning,

 

I've been having some weird slowdowns on my GUI and after a bunch of testing and rebuilding my flash drive, it appears that it's this plugin (preclear) causing this issue.

 

Specifically, the main issue is that when I have the plugin installed, the installation of apps hangs with no updating until the installation is complete.  Chrome reports the tab as unresponsive and nothing besides the banner and top menu load.

If I remove the plugin, it works normally with no issue.

 

Not sure why this is happening but I would be happy to help fix this.

@gfjardim can you provide any assistance with this?

Share this post


Link to post
12 hours ago, Trike said:

I've been having issues trying to preclear multiple drives at once. Couple nights ago, I managed to successfully preclear 12 drives with 1 cycle. 

Once that finished, I started  another 11 drives. During the Post read phase, one drive timed out and the raid controller failed it. PreClear also failed the rest of the drives due to the one drive timing out.  I'm still not sure if preclear caused it or the raid controller.  But the drive appears fine.

 

So in order to complete the cycle on all the drives that stopped/failed. I used the verify option thinking, it was going to do a pre-read and post read. But after the pre read phase, it started zero'ing the drives again, which is not what I wanted. 

 

This plugin is not stable enough for me to clear a bunch of drives at once.  

 

An enhancement request I have, is to be able to select multiple drives at once to pre-clear.  It takes me close to 10mins to click through 24 drives to select the options and start the pre-clear. Not to mention the massive gui slowdown/stall the more drives you pre-clear at once.

 

 

 

Seeing that people have problems clearing 2-4 drives caused by too little ram, I'm not surprised you get problems clearing 11 at once. Try to clear just a couple drives at once and see if that works.

Share this post


Link to post
 
Seeing that people have problems clearing 2-4 drives caused by too little ram, I'm not surprised you get problems clearing 11 at once. Try to clear just a couple drives at once and see if that works.


Just for curiosity: why would writing zeroes to multiple disks at once consume lots of RAM? Except for a progress counter per disk I don‘t see anything that would need to be stored. Do you have an idea?

I did however also unsuccessfully try to clear three 12TB drives at the same time and it is true that unRAID locked up due to lack of memory although I have 32 GB and there was nothing else running, no dockers, no VMs. Alas I did not have a lot of time for analysis and the diagnostics script was unable to collect its data to due lack of memory. So I did a reboot which obviously wiped out all traces of what had filled memory.

Share this post


Link to post
2 hours ago, tstor said:

 


Just for curiosity: why would writing zeroes to multiple disks at once consume lots of RAM? Except for a progress counter per disk I don‘t see anything that would need to be stored. Do you have an idea?

I did however also unsuccessfully try to clear three 12TB drives at the same time and it is true that unRAID locked up due to lack of memory although I have 32 GB and there was nothing else running, no dockers, no VMs. Alas I did not have a lot of time for analysis and the diagnostics script was unable to collect its data to due lack of memory. So I did a reboot which obviously wiped out all traces of what had filled memory.

 

 

I do not remember why it uses so much ram, but I'm sure someone else will chime in when they see it. If not, there is info about it in this thread or others about preclearing.

Edited by saarg

Share this post


Link to post

@gfjardim I made changes to UD to handle mounting encrypted disks.  rc.diskinfo needs to be updated to handle the parameters UD needs to properly deal with the encrypted disks.  You can review the code at the UD github, but to summarize UD needs:

$disk['luks']	= $disk['device'];
if ($disk['fstype'] == "crypto_LUKS") {
	$disk['device'] = "/dev/mapper/".basename($disk['mountpoint']);
}
$disk['target'] = $disk[device];

the disk mounted check on $disk['device'] should work properly to detect the mounted status.

Let me know if you need any assistance.

Share this post


Link to post

Is there anyone know why the script is unavailable on the forum? I try to get it since yesterday but all the links to any preclear script is "File not found".

Edited by empireqc

Share this post


Link to post
4 minutes ago, empireqc said:

Is there anyone know why the script is unavailable on the forum? I try to get it since yesterday but all the links to any preclear script is "File not found".

Is this what you are looking for:

        https://lime-technology.com/forums/topic/2732-preclear_disksh-a-new-utility-to-burn-in-and-pre-clear-disks-for-quick-add/

 

 

If you are planning to use it, see this post:

       https://lime-technology.com/forums/topic/12391-re-preclear_disksh-a-new-utility-to-burn-in-and-pre-clear-disks-for-quick-add/?page=53#comment-460592

 

Share this post


Link to post

 

Just of curiosity.. How could SMART report this disk as PASSED??? It failed the post-read verification and the disk does not look healthy at all.... 

 

Also the google forms failed to upload this report.. I let it sit for 5 minutes before i reloaded the webpage to get it to respond and then clicked dismiss. The next report that were ok were possible to upload fine in 2 secs. 

 

image.png

Edited by Meph

Share this post


Link to post
1 hour ago, Meph said:

Just of curiosity.. How could SMART report this disk as PASSED??? It failed the post-read verification and the disk does not look healthy at all....

I agree that it does not look healthy!    SMART always seems to be conservative about failing a disk, so I always assume that if a disk is OK according to SMART it may be OK in real life.    If SMART reports it as failing I treat that as meaning urgent action is required.

Share this post


Link to post

There are often a number of SMART attributes that have threshold 0, which means the attribute will never be able to go below the threshold value and so will never be in a failing state. So these attributes will only be treated as indicators for an intelligent reader to ponder.

 

Having a current pending sector is a warning - not a failure - for a disk.

On one hand, it can be a physically damaged sector - which doesn't mean the disk will fail but that the specific sectors needs to be remapped with spare sectors.

Or it can be a physically damaged head, meaning that the drive is permanently bad at reading and/or writing data.

But it can also be a power issue etc that has nothing to do with the disk itself.

 

Another thing is that the drive doesn't keep track of write times for the individual sectors. You know that you have recently written to every sector, which means that you know that 2560 offline uncorrectable is a extremely bad figure, because it means the recent fails did fail because head or surface is bad.

 

The above is a reason why many SMART programs informs about changes - expecting the owner to react if some of these fields ticks up and to try to make an intelligent decision of what to do next.

Share this post


Link to post

Ok, this is the infamous Seagate 3TB disk.. I bought 4 of them and now 3 of them has failed... :) But they are old now.... 1 of the disks still tests fault free... But i will keep an eye on it.. Changed from QNAP to unRAID also because it offers more security in case QNAP hardware fail or the RAID5... :)

Share this post


Link to post

@Meph, you have found the one of the disks that the preclear script and its cousin ,  the Preclear plugin, were developed to find and identify.  Without it, you probably would have installed this disk back into your array and it would have been installed.  As has been pointed out, the SMART system has declared that this disk is fine.  And, it probably is for 90% of the applications where it might be used.  But not for unRAID (and virtually any other RAID setup)!  Remember the SMART system was developed by the HD manufacturers and is designed to discourage the return of disks using it as a basis for a warranty  claim!  

 

Both Preclear processes present the user with the information necessary to make a decision as whether to use the disk or not.  The real philosophical  question is whether the "SMART over-all health assessment test"  results should be included in their reports.    I have never seen a case where it indicated that a disk has failed where a prudent person looking at the other attributes would have assumed that the disk was good.  Perhaps, if someone has encountered one, I would be the first to say leave it in the report as I believe to err on the side of caution in this type of situation.  

 

PS--- If this were a disk of mine, I would be running to the trash can to pitch it in!  I would not even use it for secondary storage...

Edited by Frank1940

Share this post


Link to post

@Frank1940 Yes, I agree.. I would never use this disk with those many errors.. It's in the pile marked as "Defective" and is going to the recycling center next time im going there.. 

 

One thing is strange though. The report for this disc tried to upload as usual to the developer of the plugin, but upload failed. I tried like 3-4 times.. All other reports (like 10-15 reports) have reported just fine. This is the only report where the preclear plugin did not run to the end. 

Share this post


Link to post

I found an odd issue, while preclearing a 8tb red via usb, it will pause/stop after 27 hours. After stopping the preclear and starting/resuming the preclear, it will start moving again. 

Share this post


Link to post

Any idea why the upgrade instructions for unraid 6.4 say to remove the preclear plugin?  I've been searching for known issues and haven't been able to find any.

Share this post


Link to post
25 minutes ago, Excessus said:

Any idea why the upgrade instructions for unraid 6.4 say to remove the preclear plugin?  I've been searching for known issues and haven't been able to find any.

Yes. There are many issue with the plugin. Most have been reported in the U D support thread. Quite a few people have had boot failures in 6.4 and solved the problem by deleting the plugin.

 

YMMV. I haven’t had any issues that I know of.

Edited by wgstarks

Share this post


Link to post
1 hour ago, wgstarks said:

Yes. There are many issue with the plugin. Most have been reported in the U D support thread. Quite a few people have had boot failures in 6.4 and solved the problem by deleting the plugin.

 

YMMV. I haven’t had any issues that I know of.

 

I figured that it was something along those lines.  I was just wondering about the specific issue.

 

In any case, I have faith that the author will figure it out in time; it's too useful a plugin to be killed by a vague incompatibility issue.

Share this post


Link to post
20 hours ago, Excessus said:

 

I figured that it was something along those lines.  I was just wondering about the specific issue.

 

In any case, I have faith that the author will figure it out in time; it's too useful a plugin to be killed by a vague incompatibility issue.

 

Here is one specific issue mentioned

...

If you want to keep the preclear plugin installed, go to the Settings>Unassigned Devices and set the "Enable Preclear rc.diskinfo daemon?" to "No" so it is not used to update the UD status.  The rc.diskinfo has not been updated to support the UD mounting of encrypted disks.  @gfjardim will need to update the preclear plugin.

...

 

Share this post


Link to post

I was told that 6.4.0 has issues and I see here what's going on..   But I removed the preclear plug in and I'm getting this in my logs, over and over again.. 

 

HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "localhost", referrer: "http://localhost/Main"
Jan 20 14:07:21 WHS nginx: 2018/01/20 14:07:21 [error] 10083#10083: *1239641 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 127.0.0.1, server: , request: "POST /plugins/preclear.disk/Preclear.php 

 

Any Idea what it may be about? It's filling my logs and giving me errors that my log is full..  Thanks in advance...

Share this post


Link to post
12 minutes ago, Melocco said:

I was told that 6.4.0 has issues and I see here what's going on..   But I removed the preclear plug in and I'm getting this in my logs, over and over again.. 

 

HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "localhost", referrer: "http://localhost/Main"
Jan 20 14:07:21 WHS nginx: 2018/01/20 14:07:21 [error] 10083#10083: *1239641 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 127.0.0.1, server: , request: "POST /plugins/preclear.disk/Preclear.php 

 

Any Idea what it may be about? It's filling my logs and giving me errors that my log is full..  Thanks in advance...

Reboot your server.

Share this post


Link to post

I did. This is still happening. I rebooted it right after I removed Preclear..  I will try rebooting again though..   Thanks for the advice.

Share this post


Link to post

I'm currently running the stable release 6.4.0 and I have only one problem with the Preclear plugin.

 

After I preclear the drive in the unassigned devices section, It says that the preclear completed successfully, but I have the option to cancel currently running preclear as can be seen in the attached screenshot.

 

5a66429b18ec5_ScreenShot2018-01-22at12_35_04PM.thumb.png.b10251b2d966a454bb23c2e6e694aff0.png

 

I cannot get that message to disappear unless I stop the unRAID server and reboot the server, which then gives me this attached screenshot.

 

5a66429c063f9_ScreenShot2018-01-22at12_39_18PM.thumb.png.a47d4197a27cd055183ef67112a2b509.png

 

Is this normal behaviour with this plugin?

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.