Jump to content
gfjardim

Preclear plugin

2305 posts in this topic Last Reply

Recommended Posts

New Topic

I'm preclearing an 8tb drive

Its zeroing at between 10 and 12  mb/s is that normal? 

Share this post


Link to post
25 minutes ago, jonF said:

New Topic

I'm preclearing an 8tb drive

Its zeroing at between 10 and 12  mb/s is that normal? 

Where are you seeing the speed you are reporting? (Also, there is a difference between MB/s and Mb/s, and no such thing as mb/s)

 

You would have to post a diagnostic for us to have any idea what the problem might be.

Share this post


Link to post
9 minutes ago, jonF said:

New Topic

I'm preclearing an 8tb drive

Its zeroing at between 10 and 12  mb/s is that normal? 

If I'm right with my rough calculations, that means it would take about 9 days or so to preclear the drive. Probably shouldn't take that long. Try restarting the system after cancelling the preclear and see what the speed is at that point.

Maybe set the array to not auto-start before rebooting, but that shouldn't have anything to do with the preclear. I've tried preclearing drives with and without the array running, and haven't seen any difference in speed of preclear.

Even though I see the screen messages from my LSI SAS9201 controller BIOS, I have the BIOS set to disabled, since it's not using any of the BIOS functions and just running JBOD in IT mode firmware; not sure if that makes any difference either.

Share this post


Link to post

 

4 minutes ago, jonF said:

diagnostics attached 

Your plugins are way out-of-date, including the preclear plugin.

Share this post


Link to post

Thanks! the update did it . 189 MB/s 

Thank you!

Best

Jon

Edited by jonF

Share this post


Link to post

New Topic

I've seen recommendations somewhere else for doing 3 preclear cycles. It seemed like 3 passes was almost a standard. 

I've been doing only one. Any thoughts on that topic?
Thanks

Jon

Share this post


Link to post
1 hour ago, jonF said:

New Topic

I've seen recommendations somewhere else for doing 3 preclear cycles. It seemed like 3 passes was almost a standard. 

I've been doing only one. Any thoughts on that topic?
Thanks

Jon

I've personally never done more than one. I think the rationale for more is to try to get the disk past infant mortality. Like most electronics, if it doesn't fail early, it may last until obsolete.

 

But if the rest of your array is healthy and you are diligent you should be able to survive a single disk with infant mortality. Do you have Notifications setup to alert you by email or other agent as soon as Unraid detects a problem? Do you have backups of anything important and irreplaceable?

 

The original purpose of preclear was to allow you to clear (zero) a disk so it can be added to a new data slot without invalidating parity. If you try to add an unclear disk to a new data slot, Unraid will clear it so parity is maintained.

 

Older versions of Unraid took the array offline while it cleared a disk, so preclear was invented to clear the disk before adding it to a new data slot. Later version of Unraid clear the disk without taking the array offline, so that original purpose isn't really necessary anymore.

 

And, note that except for adding a disk to a new data slot, Unraid doesn't need a clear disk for any other scenario. Such as replacing a disk, since the disk contents would be completely overwritten by the rebuild anyway.

 

Additional functionality of preclear includes some things that also test the disk. but it is possible to test a disk using other tools, such as badblocks or one of the utilities provided as free downloads by the disk manufacturer.

 

 

  • Upvote 2

Share this post


Link to post

Seeing these errors in the syslog.  Anything I need be concerned about?

 

Dec 21 18:36:11 MEDIASERVER2 root: error: /plugins/preclear.disk/Preclear.php: wrong csrf_token
Dec 21 18:36:13 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:15 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:17 MEDIASERVER2 root: error: /plugins/preclear.disk/Preclear.php: wrong csrf_token
Dec 21 18:36:17 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:18 MEDIASERVER2 root: error: /plugins/preclear.disk/Preclear.php: wrong csrf_token
Dec 21 18:36:19 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:21 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:23 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:23 MEDIASERVER2 root: error: /plugins/preclear.disk/Preclear.php: wrong csrf_token

Share this post


Link to post
Quote

There are 3 different errors that you may see logged in your syslog:

Hint: yours isn't the first one.

Share this post


Link to post

Beautiful script! Thanks so much for this. I remember trying unraid once many years ago when reiserfs was the primary option. Running pre-clears was CLI only and no where near as elegant as you've made it.

Preclear.png

Share this post


Link to post

hi !

First : thank you for this plugin

 

second: Could you add the name of the system in the email subject status report ?

i was pre-clearing 2 drives in 2 differents system at the same time and couldn't know which system it was in the email.

 

thx.

Share this post


Link to post
4 hours ago, pgnyc said:

Could you add the name of the system in the email subject status report ?

i was pre-clearing 2 drives in 2 differents system at the same time and couldn't know which system it was in the email.

Perhaps name either systems email addresses differently in senders so you'll know who it's coming from, for example:

unraid-location@domain.tld

unraid-secondlocation@domain.tld

Share this post


Link to post
20 minutes ago, Trunkton said:

Perhaps name either systems email addresses differently in senders so you'll know who it's coming from, for example:

unraid-location@domain.tld

unraid-secondlocation@domain.tld

This is what I do.

Share this post


Link to post
Posted (edited)

Preclear seems to be working but I'm getting some icon spam on the webUI. image.thumb.png.32549ca297ffbfc595f8527d16926527.png
Hovering over the icon gives no details. If i go to the preclear page it shows it's working fine. Any reason why this is happening?

Edited by slimshizn

Share this post


Link to post

Hi all!

Running unRAID v6.6.6 and Preclear Disks plugin v2018.10.07

 

Have two sets of questions:

 

1) I ran preclear on 3 of my drives, with two 3TB drives completing successfully and one 4TB drive failing the post-read. I also got a notification that the raw value for the current pending sector count on the 4TB drive was now 2 (fairly certain it was 0 before preclear). 

 

Some cursory googling brought up the possibility of a memory issue. However, since the 2x 3TBs were precleared successfully, am I right in thinking memory probably isn't the issue?

 

I do have two more 4TB drives that I'm also planning to preclear. EDIT: I subsequently ran preclear on the remaining 2x 4TB drives. Both completed successfully.

 

Should I try preclearing the failed drive again or should I just plan on swapping it?

 

Here is the failed drive's preclear results:

2117953882_PreclearFailed-4TBDrive.PNG.d82dd24ed053d997efc14dff9fd02c9f.PNG

Here are the current SMART data from the same drive:

1930286131_PreclearFailed-SMARTdata.thumb.PNG.a66c0a713385edae17b10e06816701fc.PNG

 

2) Also, I noticed that even though the preclear completed successfully on the 2x 3TB drives, the "Stop Preclear" button is still there.

Do I still need to click that red 'x' even after a successful preclear? Or can I just start assigning it to the array?

And why is the failed drive denoted as "precleared" even though it failed and the two precleared drives aren't denoted as such?

Disks.thumb.PNG.53760bb5417111277bd7ecd2e7a496ee.PNG

 

Thanks for any and all help!

 

Edited by rragu
update on subsequent preclear results

Share this post


Link to post
On 1/11/2019 at 1:46 AM, rragu said:

2) Also, I noticed that even though the preclear completed successfully on the 2x 3TB drives, the "Stop Preclear" button is still there.

Do I still need to click that red 'x' even after a successful preclear? Or can I just start assigning it to the array? 

I noticed this as well. It showed precleared successful, but I had to X to close it. Unfortunately, it seems like the final step of tagging the drive as precleared is not occurring either, so Unraid is now spending 12 hours clearing it...

 

Share this post


Link to post
1 hour ago, DoeBoye said:

I noticed this as well. It showed precleared successful, but I had to X to close it. Unfortunately, it seems like the final step of tagging the drive as precleared is not occurring either, so Unraid is now spending 12 hours clearing it...

 

 

I asked the same question on the unRAID subreddit and got the following explanation:

Quote

I think the preclear jobs actually run in tmux sessions, I imagine the red X just ends that tmux session. No harm in exiting them with the red X.

 

So, I waited until the preclear plugin notified me that all the preclearing was completed for my second set of drives. Then I restarted the array. The "Stop Preclear" button was gone and the relevant drives were properly denoted as precleared.

Share this post


Link to post

This plugin makes me want to go back to RAID 5.  Who knew it would take 3 days to add a drive and it might not even work, or the disk might fail a day after this plugin said it was OK.  This feels like taking vitamin tablets and 'knowing' it's making a difference, when in reality there's no proof and it's all in my mind.

 

Sorry to sound so negative (not really my style), but why people would want to do multiple read passes etc on modern disks that are designed to handle these kinds of errors as a near daily occurrence and quite normally correcting them in hardware without issue is beyond me.  If you want true confidence of your data, get a self-repairing file system like ZFS coupled with ECC RAM.  That peace of mind will need expensive hardware, but ZFS is stunning in that regard.

 

However, in my opinion none of this multi-pass pre-clearing stuff gives you any guarantee over your data.  You might pick up a failed drive, but you also might not.  If I was this worried about my data, I'd go the ZFS route.  For now I just want my drive to be part of the array and not have to wait a year while it figures out which side of the lottery it's going to fall on - correct or not.

 

All said, I'm very new to unRAID, so hey, perhaps I've gotten something wrong (and I'm sure someone is about to tell me)! :D

 

Again sorry if this sounds overly negative.

Edited by Marshalleq
Correction to include multi-pass

Share this post


Link to post

Hmm... Preclear is just the really cautious making sure the drives are not part of the infant mortality batch.

 

Have you ever replaced a disk in a raid 5 array and watch the rebuild scream with errors as the destination disk fails almost immediately? Not a pretty sight, specially when the stakeholders bearing down over your shoulder.

 

Even our vendor decided to give our expensive fc raid 5 array multiple complete write overwrite shakedowns over a week before they declared it ready for usage.

 

Unraid is an excellent balance between price and function versus ZFS which is also excellent, but it's better for people with a lot more money. Imagine that I can't add a single disk to my ZFS pool to resize it... Most raid 5 will let you and so will unraid.

Share this post


Link to post

The use of the Preclear plugin is optional.

 

The standard way for Unraid is to clear (write zeroes) to a newly added disk. This allows the disk to be added to the array afterwards without invalidating the parity disk. Depending on your disk the clearing process may take from a couple of hours to a day to finish (e.g. I have 10TB disks which take about 16 hours).

 

After clearing is complete, the new disk is added to the array and the user is asked to format it (write an empty file system). Formatting is done in a couple of minutes, and now the disk is ready to be used.

 

Some people like to have more testing done before putting the disk in service. This is the main feature of the preclear plugin nowadays, but this is not mandatory and really depends on the perception of the user to test disk reliability on beforehand. Also note that disk reliability can be tested using different tools, e.g. test tools from the disk manufacturer itself.

 

 

Share this post


Link to post
7 hours ago, Marshalleq said:

All said, I'm very new to unRAID, so hey, perhaps I've gotten something wrong (and I'm sure someone is about to tell me)!

Not exactly wrong, but incomplete and not really up-to-date on how preclear fits into Unraid these days.

 

7 hours ago, bonienl said:

The standard way for Unraid is to clear (write zeroes) to a newly added disk. This allows the disk to be added to the array afterwards without invalidating the parity disk.

Also note that the only time Unraid requires a clear disk is when adding it to a new data slot in an array that already has valid parity, so parity will remain valid. A replacement disk, for example, doesn't need to be clear since it will be completely overwritten by the rebuild anyway.

 

And, Unraid will clear a disk in that one scenario where it requires a clear disk. The historical reason for preclear is because Unraid would take the array offline when it cleared a disk, so preclear was invented to clear the disk before adding it so Unraid wouldn't need to take the array offline. Additional testing functionality was builtin to preclear to help weed out the "infant mortality".

 

Recent versions of Unraid do not take the array offline when it clears a disk, so the only function preclear has these days is testing.

 

Share this post


Link to post
On 1/11/2019 at 1:46 AM, rragu said:

Here are the current SMART data from the same drive:

1930286131_PreclearFailed-SMARTdata.thumb.PNG.a66c0a713385edae17b10e06816701fc.PNG

 

Since no one has taken the time to reply about the state of this drive, let me give you my thoughts.

 

I would never add a drive to a array which had any Attributes numbered 187, 188, 197 or  198 that were not zero.  I would not add a drive to an array whose  Attribute number 5 was not zero UNLESS that drive had been through THREE  preclear cycles with increasing!  

 

Attribute number 199 is the one  Unraid 'flagged' attribute which is usually not related to a potential drive failure.  It is usually a cabling or (rarely) controller issue. 

 

In your case, another complete preclear cycle should clear up the two #197 errors but that should also result in a corresponding increase in the Reallocated Sector Count (#5).  (I, personally, would then run another two preclear cycles to make sure that everything was now staple!) 

 

The reason for my caution is that replacement of a failed array drive always has the potential to cause data loss.  Rebuilding a failed drive requires (in a single parity setup) that every sector on every other drive in the array must be able to read perfectly.  If only one sector on any drive cannot be read during the rebuild then the rebuild will fail.  For this reason, I never want to deliberately put any drive into any array that I  have the least question about its initial reliability! I would RMA this drive if possible.  (My past experience is that HD manufacturers  don't argue about warranty claims.  They have always shipped the replacement to me on the day they receive the old one! After all, ripping out a drive out of most computers for warranty replacement is really big hassle and every few people do it on a whim!) 

Edited by Frank1940
  • Upvote 3

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now