Unofficial Faster Preclear


Recommended Posts

Ah, OK.

You can compare your drive with this one:

 

========================================================================1.15b
== invoked as: ./preclear_disk15b.sh -A -f /dev/sdi
== ST4000DX000-1CL160   Z1Z05P4J
== Disk /dev/sdi has been successfully precleared
== with a starting sector of 1 
== Ran 1 cycle
==
== Using :Read block size = 8388608 Bytes
== Last Cycle's Pre Read Time  : 9:10:07 (121 MB/s)
== Last Cycle's Zeroing time   : 8:30:30 (130 MB/s)
== Last Cycle's Post Read Time : 9:59:49 (111 MB/s)
== Last Cycle's Total Time     : 27:41:29
==
== Total Elapsed Time 27:41:29
==
== Disk Start Temperature: 27C
==
== Current Disk Temperature: 33C, 
==
============================================================================

Link to comment

That seems reasonable -- it fits with Brian's comments on his faster method, which is focused on speeding up the post-read.  The first two phases are consistent with what Joe's script takes; but the post-read is notably faster.  In fact, based on our results, Joe's script is actually faster for both the pre-read and the zeroing, but takes 8 hours longer for the post-read.

 

It IS a nice improvement ... but as I noted before, when you're waiting, you're waiting.    Doesn't seem to make a lot of difference whether you're waiting 28 hours or 35 hours.    Nevertheless, I'll use Brian's script the next time I do a pre-clear  :)

 

Link to comment

Doesn't seem to make a lot of difference whether you're waiting 28 hours or 35 hours.    Nevertheless, I'll use Brian's script the next time I do a pre-clear  :)

 

It makes a days difference when you're using something sane like 3 pass minimum on new drives,  84 hours versus 108 hours. You rounded down on the slow method but rounded up on the fast method., its 28 versus 36. That is substantial and those aren't even the largest drives available.

 

Link to comment

I agree it's a reasonable difference ... as I noted I'll use Brian's script for my next pre-clear.

 

But the reality is it really doesn't matter.  For a 3-pass pre-clear it does indeed make a day's difference.  But as I said several times, when you're waiting, you're waiting.  And when that wait is on the order of days, I really don't much care whether it's 3 1/2 or 4 1/2 days.

 

Link to comment

The extra day matters when you are in an emergency situation and have to preclear a drive for a rebuild.

 

In my case, I usually do a 5-7 pass badblocks test before even doing a preclear.

Then I stage the drive in the emergency repair zone.

That can take almost a week.

 

I'm sure many people will appreciate saving a day!

Thanks bjp999!

Link to comment

 

The extra day matters when you are in an emergency situation and have to preclear a drive for a rebuild.

 

Very true but I don't think Gary knows what an emergency situation is... I hear he has a backup server on the moon! You never know when the earth will blow up after all

And I bet it's in a Lian Li Q25 :)

Link to comment

The extra day matters when you are in an emergency situation and have to preclear a drive for a rebuild.

 

In my case, I usually do a 5-7 pass badblocks test before even doing a preclear.

Then I stage the drive in the emergency repair zone.

That can take almost a week.

 

I'm sure many people will appreciate saving a day!

Thanks bjp999!

Would you care to share how you do this step by step for a new user ?

 

Explain this also if you will.

"Then I stage the drive in the emergency repair zone"

Link to comment

 

The extra day matters when you are in an emergency situation and have to preclear a drive for a rebuild.

 

Very true but I don't think Gary knows what an emergency situation is... I hear he has a backup server on the moon! You never know when the earth will blow up after all

 

Actually my ultimate backup server is on Mars  :)

 

I'm well aware of what emergency situations are => but the way to prepare for them is to not WAIT until one hits to start the process of preparing a new drive.    I ALWAYS have at least one pre-tested spare drive available (currently I have 2), so when I DO need one, I have it NOW ... not 2 days later.

 

I definitely appreciate - and will use - Brian's faster script ... I agree with him that Joe's pre-clear script takes too long for the post-read -- and in discussions with Joe it wasn't clear why that's the case.    My point was simply that while it's nice to have the increased speed, it's still a long process -- "when you're waiting, you're waiting."    Pre-clear is NOT something you should have to do in an emergency ... if it is, then YOU don't understand what an emergency situation is, and consequently haven't prepared for one.

 

 

 

Link to comment

Pre-clear is NOT something you should have to do in an emergency ... if it is, then YOU don't understand what an emergency situation is, and consequently haven't prepared for one.

 

Not everyone has the budget to have a drive as large as the parity drive laying around pre-cleared waiting to replace a failed drive.  I remember someone saying, 'my spare drive is at newegg.com' I call out for it when I need it. heh.

 

We get the fact of what people should do to be prepared, but still having hardware laying around is not in everyone's budget.

 

 

That being said, SMART monitoring can aid in dealing with a situation beforehand. Although this is not 100% (which is understood).

A faster preclear will aid in getting that drive prepared !

Link to comment

You should always insure what you can't afford to lose -- e.g. most of us have insurance on our houses and cars, but not on our bicycles [although I have friends with carbon-fiber bikes that cost as much as some cars  :)].

 

If you can't afford the downtime from a drive failure, it's simply "insurance" to keep a spare drive on the shelf.  If a drive failure isn't a big enough deal that you need to do that, then I'd argue that it's also not an "emergency."    I'd also argue that if you can afford to buy a drive when you need it; you can afford to buy it in advance as insurance against unwanted system downtime.

 

Also, remember that a pre-clear is NOT a mandatory thing -- if you prefer to buy drives "just in time," and consider these "emergency" purchases; then simply buy quality drives and USE them when you get them.    84 hours is certainly better than 108 hours of delay between getting the drive and using it;  but ZERO hours is even better !!  :)

 

Link to comment

I ALWAYS have at least one pre-tested spare drive available (currently I have 2), so when I DO need one, I have it NOW ... not 2 days later.

 

Well, as I already experienced the hard way, a drive laying around (especially WD's) can also fail when you need it!

After all, this fact was the last bit that I needed to finally build my unRAID server.

 

Conclusion: A spare drive should also be subject to regular checks!

(I'm sure Gary has a regime to do so.)

 

Link to comment

Conclusion: A spare drive should also be subject to regular checks!

(I'm sure Gary has a regime to do so.)

 

Yes, I do indeed ... I run a Data Lifeguard extended test on my spares once every 6 months if I haven't used the drive.    I also run annual tests on all my stored backup drives to ensure they're still okay.

 

Link to comment

Thanks for posting bjp999.

 

The script I downloaded a couple days ago has a preread_skip_pct parameter that is hard coded to 47.  This results in almost half of the disk being skipped in the preread stage.  I'm guessing that was used for testing and maybe the posted version should be revised to avoid any skipping.

Link to comment

Thanks for posting bjp999.

 

The script I downloaded a couple days ago has a preread_skip_pct parameter that is hard coded to 47.  This results in almost half of the disk being skipped in the preread stage.  I'm guessing that was used for testing and maybe the posted version should be revised to avoid any skipping.

 

Sorry about that. The skip_pct was something that I was working on to resume a preclear that was partially started. I accidentally posted the version I was testing with.

 

EVERYONE, please download the version below. The original post has also been updated to download this corrected version.

 

Here is the link to the download file Fast Preclear

Link to comment

 

Just precleared a Corsair x100

 

Preclear Successful

... Total time 1:35:54

... Pre-Read time 0:19:07 (223 MB/s)

... Zeroing time 0:27:31 (155 MB/s)

... Post-Read time 0:47:53 (89 MB/s)

 

Its giving me the Hpa warning on unmenu though

 

FYI, You should not need to preclear an SSD unless you are putting it into the array as a data drive.

If it's for a cache/apps drive, it should not be precleared.

Link to comment

You only need to preclear it if it's an array drive.

As a cache/apps drive, no need to preclear.

While recent drives handle writes better, once a cell is written, it now has to be erased before being re-written.

 

This may have the side effect of not reaching the speeds that you expect.

I doubt it will be an issue.

You can read up on trim here.

 

http://en.wikipedia.org/wiki/Trim_(computing)

 

Frankly, I've had a 256GB SSD in my daily use internet laptop for a couple years. I beat the heck out of it and I've never noticed any slow down. 

 

Many drives also have garbage collection in the firmware or software, so by leaving the drive plugged in and idle it may help.

Link to comment

... Many drives also have garbage collection in the firmware or software, so by leaving the drive plugged in and idle it may help.

 

Virtually all modern SSDs do this -- so it's not nearly the issue is used to be whether or not your OS supports TRIM, as long as the drive will be "idle" enough for its firmware garbage collection to do the trick.    I've put SSDs in quite a few older systems running XP for folks ... and they run superbly.

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.