Preclear plugin


Recommended Posts

9 minutes ago, jonF said:

New Topic

I'm preclearing an 8tb drive

Its zeroing at between 10 and 12  mb/s is that normal? 

If I'm right with my rough calculations, that means it would take about 9 days or so to preclear the drive. Probably shouldn't take that long. Try restarting the system after cancelling the preclear and see what the speed is at that point.

Maybe set the array to not auto-start before rebooting, but that shouldn't have anything to do with the preclear. I've tried preclearing drives with and without the array running, and haven't seen any difference in speed of preclear.

Even though I see the screen messages from my LSI SAS9201 controller BIOS, I have the BIOS set to disabled, since it's not using any of the BIOS functions and just running JBOD in IT mode firmware; not sure if that makes any difference either.

Link to comment
1 hour ago, jonF said:

New Topic

I've seen recommendations somewhere else for doing 3 preclear cycles. It seemed like 3 passes was almost a standard. 

I've been doing only one. Any thoughts on that topic?
Thanks

Jon

I've personally never done more than one. I think the rationale for more is to try to get the disk past infant mortality. Like most electronics, if it doesn't fail early, it may last until obsolete.

 

But if the rest of your array is healthy and you are diligent you should be able to survive a single disk with infant mortality. Do you have Notifications setup to alert you by email or other agent as soon as Unraid detects a problem? Do you have backups of anything important and irreplaceable?

 

The original purpose of preclear was to allow you to clear (zero) a disk so it can be added to a new data slot without invalidating parity. If you try to add an unclear disk to a new data slot, Unraid will clear it so parity is maintained.

 

Older versions of Unraid took the array offline while it cleared a disk, so preclear was invented to clear the disk before adding it to a new data slot. Later version of Unraid clear the disk without taking the array offline, so that original purpose isn't really necessary anymore.

 

And, note that except for adding a disk to a new data slot, Unraid doesn't need a clear disk for any other scenario. Such as replacing a disk, since the disk contents would be completely overwritten by the rebuild anyway.

 

Additional functionality of preclear includes some things that also test the disk. but it is possible to test a disk using other tools, such as badblocks or one of the utilities provided as free downloads by the disk manufacturer.

 

 

  • Upvote 2
Link to comment
  • 4 weeks later...

Seeing these errors in the syslog.  Anything I need be concerned about?

 

Dec 21 18:36:11 MEDIASERVER2 root: error: /plugins/preclear.disk/Preclear.php: wrong csrf_token
Dec 21 18:36:13 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:15 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:17 MEDIASERVER2 root: error: /plugins/preclear.disk/Preclear.php: wrong csrf_token
Dec 21 18:36:17 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:18 MEDIASERVER2 root: error: /plugins/preclear.disk/Preclear.php: wrong csrf_token
Dec 21 18:36:19 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:21 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:23 MEDIASERVER2 root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token
Dec 21 18:36:23 MEDIASERVER2 root: error: /plugins/preclear.disk/Preclear.php: wrong csrf_token

Link to comment
  • 2 weeks later...

Hi all!

Running unRAID v6.6.6 and Preclear Disks plugin v2018.10.07

 

Have two sets of questions:

 

1) I ran preclear on 3 of my drives, with two 3TB drives completing successfully and one 4TB drive failing the post-read. I also got a notification that the raw value for the current pending sector count on the 4TB drive was now 2 (fairly certain it was 0 before preclear). 

 

Some cursory googling brought up the possibility of a memory issue. However, since the 2x 3TBs were precleared successfully, am I right in thinking memory probably isn't the issue?

 

I do have two more 4TB drives that I'm also planning to preclear. EDIT: I subsequently ran preclear on the remaining 2x 4TB drives. Both completed successfully.

 

Should I try preclearing the failed drive again or should I just plan on swapping it?

 

Here is the failed drive's preclear results:

2117953882_PreclearFailed-4TBDrive.PNG.d82dd24ed053d997efc14dff9fd02c9f.PNG

Here are the current SMART data from the same drive:

1930286131_PreclearFailed-SMARTdata.thumb.PNG.a66c0a713385edae17b10e06816701fc.PNG

 

2) Also, I noticed that even though the preclear completed successfully on the 2x 3TB drives, the "Stop Preclear" button is still there.

Do I still need to click that red 'x' even after a successful preclear? Or can I just start assigning it to the array?

And why is the failed drive denoted as "precleared" even though it failed and the two precleared drives aren't denoted as such?

Disks.thumb.PNG.53760bb5417111277bd7ecd2e7a496ee.PNG

 

Thanks for any and all help!

 

Edited by rragu
update on subsequent preclear results
Link to comment
On 1/11/2019 at 1:46 AM, rragu said:

2) Also, I noticed that even though the preclear completed successfully on the 2x 3TB drives, the "Stop Preclear" button is still there.

Do I still need to click that red 'x' even after a successful preclear? Or can I just start assigning it to the array? 

I noticed this as well. It showed precleared successful, but I had to X to close it. Unfortunately, it seems like the final step of tagging the drive as precleared is not occurring either, so Unraid is now spending 12 hours clearing it...

 

Link to comment
1 hour ago, DoeBoye said:

I noticed this as well. It showed precleared successful, but I had to X to close it. Unfortunately, it seems like the final step of tagging the drive as precleared is not occurring either, so Unraid is now spending 12 hours clearing it...

 

 

I asked the same question on the unRAID subreddit and got the following explanation:

Quote

I think the preclear jobs actually run in tmux sessions, I imagine the red X just ends that tmux session. No harm in exiting them with the red X.

 

So, I waited until the preclear plugin notified me that all the preclearing was completed for my second set of drives. Then I restarted the array. The "Stop Preclear" button was gone and the relevant drives were properly denoted as precleared.

Link to comment

This plugin makes me want to go back to RAID 5.  Who knew it would take 3 days to add a drive and it might not even work, or the disk might fail a day after this plugin said it was OK.  This feels like taking vitamin tablets and 'knowing' it's making a difference, when in reality there's no proof and it's all in my mind.

 

Sorry to sound so negative (not really my style), but why people would want to do multiple read passes etc on modern disks that are designed to handle these kinds of errors as a near daily occurrence and quite normally correcting them in hardware without issue is beyond me.  If you want true confidence of your data, get a self-repairing file system like ZFS coupled with ECC RAM.  That peace of mind will need expensive hardware, but ZFS is stunning in that regard.

 

However, in my opinion none of this multi-pass pre-clearing stuff gives you any guarantee over your data.  You might pick up a failed drive, but you also might not.  If I was this worried about my data, I'd go the ZFS route.  For now I just want my drive to be part of the array and not have to wait a year while it figures out which side of the lottery it's going to fall on - correct or not.

 

All said, I'm very new to unRAID, so hey, perhaps I've gotten something wrong (and I'm sure someone is about to tell me)! :D

 

Again sorry if this sounds overly negative.

Edited by Marshalleq
Correction to include multi-pass
Link to comment

Hmm... Preclear is just the really cautious making sure the drives are not part of the infant mortality batch.

 

Have you ever replaced a disk in a raid 5 array and watch the rebuild scream with errors as the destination disk fails almost immediately? Not a pretty sight, specially when the stakeholders bearing down over your shoulder.

 

Even our vendor decided to give our expensive fc raid 5 array multiple complete write overwrite shakedowns over a week before they declared it ready for usage.

 

Unraid is an excellent balance between price and function versus ZFS which is also excellent, but it's better for people with a lot more money. Imagine that I can't add a single disk to my ZFS pool to resize it... Most raid 5 will let you and so will unraid.

Link to comment

The use of the Preclear plugin is optional.

 

The standard way for Unraid is to clear (write zeroes) to a newly added disk. This allows the disk to be added to the array afterwards without invalidating the parity disk. Depending on your disk the clearing process may take from a couple of hours to a day to finish (e.g. I have 10TB disks which take about 16 hours).

 

After clearing is complete, the new disk is added to the array and the user is asked to format it (write an empty file system). Formatting is done in a couple of minutes, and now the disk is ready to be used.

 

Some people like to have more testing done before putting the disk in service. This is the main feature of the preclear plugin nowadays, but this is not mandatory and really depends on the perception of the user to test disk reliability on beforehand. Also note that disk reliability can be tested using different tools, e.g. test tools from the disk manufacturer itself.

 

 

Link to comment
7 hours ago, Marshalleq said:

All said, I'm very new to unRAID, so hey, perhaps I've gotten something wrong (and I'm sure someone is about to tell me)!

Not exactly wrong, but incomplete and not really up-to-date on how preclear fits into Unraid these days.

 

7 hours ago, bonienl said:

The standard way for Unraid is to clear (write zeroes) to a newly added disk. This allows the disk to be added to the array afterwards without invalidating the parity disk.

Also note that the only time Unraid requires a clear disk is when adding it to a new data slot in an array that already has valid parity, so parity will remain valid. A replacement disk, for example, doesn't need to be clear since it will be completely overwritten by the rebuild anyway.

 

And, Unraid will clear a disk in that one scenario where it requires a clear disk. The historical reason for preclear is because Unraid would take the array offline when it cleared a disk, so preclear was invented to clear the disk before adding it so Unraid wouldn't need to take the array offline. Additional testing functionality was builtin to preclear to help weed out the "infant mortality".

 

Recent versions of Unraid do not take the array offline when it clears a disk, so the only function preclear has these days is testing.

 

Link to comment
On 1/11/2019 at 1:46 AM, rragu said:

Here are the current SMART data from the same drive:

1930286131_PreclearFailed-SMARTdata.thumb.PNG.a66c0a713385edae17b10e06816701fc.PNG

 

Since no one has taken the time to reply about the state of this drive, let me give you my thoughts.

 

I would never add a drive to a array which had any Attributes numbered 187, 188, 197 or  198 that were not zero.  I would not add a drive to an array whose  Attribute number 5 was not zero UNLESS that drive had been through THREE  preclear cycles with increasing!  

 

Attribute number 199 is the one  Unraid 'flagged' attribute which is usually not related to a potential drive failure.  It is usually a cabling or (rarely) controller issue. 

 

In your case, another complete preclear cycle should clear up the two #197 errors but that should also result in a corresponding increase in the Reallocated Sector Count (#5).  (I, personally, would then run another two preclear cycles to make sure that everything was now staple!) 

 

The reason for my caution is that replacement of a failed array drive always has the potential to cause data loss.  Rebuilding a failed drive requires (in a single parity setup) that every sector on every other drive in the array must be able to read perfectly.  If only one sector on any drive cannot be read during the rebuild then the rebuild will fail.  For this reason, I never want to deliberately put any drive into any array that I  have the least question about its initial reliability! I would RMA this drive if possible.  (My past experience is that HD manufacturers  don't argue about warranty claims.  They have always shipped the replacement to me on the day they receive the old one! After all, ripping out a drive out of most computers for warranty replacement is really big hassle and every few people do it on a whim!) 

Edited by Frank1940
  • Upvote 3
Link to comment

@trurl @ken-ji @bonienl Thanks for your replies - all your points are good ones.  This was a very very long thread and I read probably too much of it.  What jumped out at me were some poor people panicking and either negatively misinterpreting the results or over expecting the plugin like it's a miracle cure.  

 

Sometimes I think we forget that not everyone (possibly even most on here) do not have IT as a first job (like at least two of us seem to) and can take thinks a little too literally or not have the background to interpret the results.  e.g. 'I read that I must preclear three times for every disk, it's not working and what do I do?  Is my raid safe? Must I return my drive?! Panic!' someone said.  That poor soul is more likely to lose data by not having enough Unraid space available than by worrying about pre-clear status.

 

So my point is there seems to be an unnecessary bias or unbalanced viewpoint provided (at least through the parts of this thread that I read).  There should be some kind of disclaimer that balances things out for some of these people - might help prevent a few panicked comments on a very long thread too!

 

Point of note - when I did run the preclear plugin I got a small error, which doesn't actually matter.  When I added it to the array, I got that same error during its (much faster) preclear anyway.  And in both cases it happened within first 10 seconds.  Probably the case for most people - if they see anything, otherwise the vendor would include it in the product right? :D And pre-clear seems quite buggy.

Link to comment
50 minutes ago, Marshalleq said:

do not have IT as a first job

These days IT is a collections of specialties that often have trouble communicating with each other. I'm sure many IT professionals know little about many of the things that are important when dealing with Unraid. I get the impression that most people do not know what "format" actually does, for example.

Link to comment
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.