Jump to content
Biggy2872

Re: preclear_disk.sh - a new utility to burn-in and pre-clear disks for quick add

1381 posts in this topic Last Reply

Recommended Posts

Now the strange problem: 

 

My initial or 1st Drive seemed to exhibit a similar problem so I was not particularly worried, albeit frustrated at the loss of time but...  So I decided I would just re-boot (when the drive fails preclear it ceases to be seen) and low and behold the Windows machine would not boot off any USB stick regardless of location and/or Boot Selection attempt???  Very strange behavior, I finally re-attached my Windows HD and booted successfully, then totally cleared the 2nd 5TB Drive thinking that something was amiss.  Still no change despite 3 different valid USB bootable sticks.

 

Finally this evening, I went into the Bios again and "Restored Default Settings" and low & behold UnRaid now Boots and I've re-started the PreClear Plug-In.

 

So the "burning" question is How Did PreClear Failure somehow write "code" to the BIOS!  Is there anything that this will do to a system in the future?

 

First, no software (including Preclear) writes to the BIOS.

 

This is actually a common problem with many motherboards.  Whenever you change the installed drives list for the system, the BIOS may decide to "help" you, and reorder the boot order so that the most likely hard drive will be booted, which is usually NOT the USB drive you had configured!  You did the right thing by going into the BIOS and correcting the boot order, making sure the right drive is booted, not what the BIOS *thinks* is the right drive.

 

Thanks Rob - I agree in a sense, but I actually selected a "seen" USB Bootable Hard Drive and it/they still failed.  Maybe the Bios still changed it to the Cleared (not PreCleared) hard drive as it showed "no Bootable disc found".

 

Still an interesting and "freaky" thing to witness.  It worked fine until the PreClear "failed" then would not boot until it was reset.

 

Dave

Even though the pre-clear had failed (detected it had not filled the disk as expected), it could have written what looks to the BIOS as a valid master-boot-record to the hard-disk being cleared. 

 

In other words, as RobJ said, your bios was trying to "help" you by choosing one of your hard-disks to boot from that it thought had a valid master-boot-record, and since none contain actual code to boot from, nothing would boot until you set the bios back to boot from the correct usb-flash-drive.

 

 

Share this post


Link to post

Now the strange problem: 

 

My initial or 1st Drive seemed to exhibit a similar problem so I was not particularly worried, albeit frustrated at the loss of time but...  So I decided I would just re-boot (when the drive fails preclear it ceases to be seen) and low and behold the Windows machine would not boot off any USB stick regardless of location and/or Boot Selection attempt???  Very strange behavior, I finally re-attached my Windows HD and booted successfully, then totally cleared the 2nd 5TB Drive thinking that something was amiss.  Still no change despite 3 different valid USB bootable sticks.

 

Finally this evening, I went into the Bios again and "Restored Default Settings" and low & behold UnRaid now Boots and I've re-started the PreClear Plug-In.

 

So the "burning" question is How Did PreClear Failure somehow write "code" to the BIOS!  Is there anything that this will do to a system in the future?

 

First, no software (including Preclear) writes to the BIOS.

 

This is actually a common problem with many motherboards.  Whenever you change the installed drives list for the system, the BIOS may decide to "help" you, and reorder the boot order so that the most likely hard drive will be booted, which is usually NOT the USB drive you had configured!  You did the right thing by going into the BIOS and correcting the boot order, making sure the right drive is booted, not what the BIOS *thinks* is the right drive.

 

Thanks Rob - I agree in a sense, but I actually selected a "seen" USB Bootable Hard Drive and it/they still failed.  Maybe the Bios still changed it to the Cleared (not PreCleared) hard drive as it showed "no Bootable disc found".

 

Still an interesting and "freaky" thing to witness.  It worked fine until the PreClear "failed" then would not boot until it was reset.

 

Dave

Even though the pre-clear had failed (detected it had not filled the disk as expected), it could have written what looks to the BIOS as a valid master-boot-record to the hard-disk being cleared. 

 

In other words, as RobJ said, your bios was trying to "help" you by choosing one of your hard-disks to boot from that it thought had a valid master-boot-record, and since none contain actual code to boot from, nothing would boot until you set the bios back to boot from the correct usb-flash-drive.

 

Thanks Joe/Rob,

 

The disk has now failed to PreClear again, so its probably at best suspect and should be returned if possible (???).  It failed originally inside of my active UnRaid box and now 2x in a separate box (aka the Windows Machine w/o Windows HD) using a "spare" key.

 

I'm a bit frustrated that when it fails (didn't see it this time), I don't get a "log" that I could post for analysis.  When if fails, it just ceases to be seen by the computer and/or the PreClear "plug-in" (disappears) leaving only the flash drive and the Sata (Generic) Controller listed.

 

Pending any thoughts or recommendations, I will let it "run" before Powering Down in case there is a "hidden" file/log that might shed some light on the failure.

 

If you have time, please let me know.

 

Dave

Share this post


Link to post

I had a case of disappearing drives whilst preclearing whilst connected to a Marvell controller. Switch to Intel and all well. Did 4 cycles just to be sure.

No issues since, done 4 parity checks since then without problem.

Share this post


Link to post

I'm a bit frustrated that when it fails (didn't see it this time), I don't get a "log" that I could post for analysis.  When if fails, it just ceases to be seen by the computer and/or the PreClear "plug-in" (disappears) leaving only the flash drive and the Sata (Generic) Controller listed.

 

I recently experienced a similar situation preclearing a 5T Seagate that was pulled from an external enclosure. The drive precleared perfectly until the very end, at which time preclear was trying to prepare the boot sector and install the preclear signature. The drive become unresponsive and was kicked by the controller. Took a reboot for drive to again be recognized. No smart issues or signs of any problem from the disk's perspective. With my inner knowledge, I was able to rerun the final steps of preclear and it happened again. So this was not a fluke, it was repeatable.

 

I am not sure if this was a bad disk, or if Seagate was implementing some protection in the firmware to protect the kind of update the preclear script was doing.

 

I returned the drive.

Share this post


Link to post

I had a case of disappearing drives whilst preclearing whilst connected to a Marvell controller. Switch to Intel and all well. Did 4 cycles just to be sure.

No issues since, done 4 parity checks since then without problem.

Me to.  I have a SASLP-MV8 which uses a Marvel controller in one server.  I tried to preclear a couple of 5TB Toshibas and I lost all drives connected to controller - including some array drives - until I rebooted the server.  Switched to MB ports on a different box and those drives precleared 3 cycles just fine.  I am planing to swap controllers to a DELL H310 this weekend.  It is in my N54L with 2 array drives connected and 4 lines snaked outside the case to do preclears - or so I thought.  Think I'll be OK once I switch controllers but it is a pain to have to remove MB to change the controller on these so may not get it done this weekend.

Share this post


Link to post

I had a case of disappearing drives whilst preclearing whilst connected to a Marvell controller. Switch to Intel and all well. Did 4 cycles just to be sure.

No issues since, done 4 parity checks since then without problem.

Me to.  I have a SASLP-MV8 which uses a Marvel controller in one server.  I tried to preclear a couple of 5TB Toshibas and I lost all drives connected to controller - including some array drives - until I rebooted the server.  Switched to MB ports on a different box and those drives precleared 3 cycles just fine.  I am planing to swap controllers to a DELL H310 this weekend.  It is in my N54L with 2 array drives connected and 4 lines snaked outside the case to do preclears - or so I thought.  Think I'll be OK once I switch controllers but it is a pain to have to remove MB to change the controller on these so may not get it done this weekend.

 

Might want to consider a LSI sas card that ha a mini-sas connector and one of the HighPoint External Mini-SAS to eSATA Cable.

http://www.bhphotovideo.com/c/product/981507-REG/highpoint_ext_ms_1mes_external_mini_sas_to_esata.html

 

i.e. LSI SAS 9212-4i4e

 

There's also this.

 

CoolGear Internal to External eSATA 4 Port PCI Bracket

http://www.amazon.com/gp/product/B004G9V4NI?psc=1&redirect=true&ref_=oh_aui_detailpage_o00_s00

Share this post


Link to post

I had a case of disappearing drives whilst preclearing whilst connected to a Marvell controller. Switch to Intel and all well. Did 4 cycles just to be sure.

No issues since, done 4 parity checks since then without problem.

Me to.  I have a SASLP-MV8 which uses a Marvel controller in one server.  I tried to preclear a couple of 5TB Toshibas and I lost all drives connected to controller - including some array drives - until I rebooted the server.  Switched to MB ports on a different box and those drives precleared 3 cycles just fine.  I am planing to swap controllers to a DELL H310 this weekend.  It is in my N54L with 2 array drives connected and 4 lines snaked outside the case to do preclears - or so I thought.  Think I'll be OK once I switch controllers but it is a pain to have to remove MB to change the controller on these so may not get it done this weekend.

 

Might want to consider a LSI sas card that ha a mini-sas connector and one of the HighPoint External Mini-SAS to eSATA Cable.

http://www.bhphotovideo.com/c/product/981507-REG/highpoint_ext_ms_1mes_external_mini_sas_to_esata.html

 

i.e. LSI SAS 9212-4i4e

 

There's also this.

 

CoolGear Internal to External eSATA 4 Port PCI Bracket

http://www.amazon.com/gp/product/B004G9V4NI?psc=1&redirect=true&ref_=oh_aui_detailpage_o00_s00

Looks cool and I might switch to the first option if I need to use the H310 elsewhere but I have two or 3 available currently.  Can't use the second option on the N54L because I'm using the IPMI card in the second slot.  But that would work well for some Windows VMs on my servers in the basement.  Thanks.

Share this post


Link to post

Me to.  I have a SASLP-MV8 which uses a Marvel controller in one server.  I tried to preclear a couple of 5TB Toshibas and I lost all drives connected to controller - including some array drives - until I rebooted the server.  Switched to MB ports on a different box and those drives precleared 3 cycles just fine.  I am planing to swap controllers to a DELL H310 this weekend.  It is in my N54L with 2 array drives connected and 4 lines snaked outside the case to do preclears - or so I thought.  Think I'll be OK once I switch controllers but it is a pain to have to remove MB to change the controller on these so may not get it done this weekend.

The SASLP-MV8 controller is widely used on unRAID without issues.  I have several of them and they work fine.  I have found cases where a bad drive takes down the whole controller but that does not seem to be specific to the SASLP-MV8.    The other time I have had problems was when I had not seated the SASLP-MV8 perfectly into the PCIe slot (because the backplate was slightly bent).    I found that this was very hard to diagnose as the everything seemed to work fine until one put the system under heavy load, but since corrected there have been no issues with that controller.

Share this post


Link to post
The other time I have had problems was when I had not seated the SASLP-MV8 perfectly into the PCIe slot (because the backplate was slightly bent).    I found that this was very hard to diagnose as the everything seemed to work fine until one put the system under heavy load, but since corrected there have been no issues with that controller.

That could be the case here since it is very hard to make sure the x4 is seated properly and slide the MB back into the case on the N54L - For me at least.  I've had better luck with an H310 in my N40L because it is x8 PCIe and stays seated better.

Share this post


Link to post

I had my problem with the drive on a motherboard port. No other drives were affected.

Share this post


Link to post

Sorry if this has been answered - I couldn't find anything with search.

 

Is it possible to run pre_clear.sh using a non-unRAID machine (e.g. a Ubuntu box)? My unRAID box is out of SATA ports and I have a failing disk I need to replace.

Share this post


Link to post

Sorry if this has been answered - I couldn't find anything with search.

 

Is it possible to run pre_clear.sh using a non-unRAID machine (e.g. a Ubuntu box)? My unRAID box is out of SATA ports and I have a failing disk I need to replace.

My first guess is no..  It does some checks to see if the disk is in the array.  That might break..  But that's just a guess...

Someone else may have a better, more accurate, answer. 

I've done a spare usb drive with unraid eval before on another machine.. 

 

Boot to usb.

Run unraid.

Go back to normal os. 

 

Might be a little more complicated now that you have to get a temp key from limetech.

 

 

Jim

Share this post


Link to post

Sorry if this has been answered - I couldn't find anything with search.

 

Is it possible to run pre_clear.sh using a non-unRAID machine (e.g. a Ubuntu box)? My unRAID box is out of SATA ports and I have a failing disk I need to replace.

See search tips in my sig.

 

Post from the author, in this thread.

 

Be careful to select the correct disk since it won't be able to save you from yourself if you run it on another system.

Share this post


Link to post

Sorry if this has been answered - I couldn't find anything with search.

 

Is it possible to run pre_clear.sh using a non-unRAID machine (e.g. a Ubuntu box)? My unRAID box is out of SATA ports and I have a failing disk I need to replace.

 

Yes & No.  You can't run it on a Ubuntu box; but as long as you boot UnRAID you can run it on any machine you want.    So just create a USB flash disk with the free version of UnRAID (v5), then boot it on any machine you want to use pre-clear on and pre-clear away.  No need to even have any disks assigned to the array.

 

However -- since you're not adding a disk to the array, but simply replacing one, you don't really need to run pre-clear at all.    You can use the manufacturer's diagnostics to test your new disk before doing the replacement ... and can run those on any supported system.    For example, I test all of my new disks on a Windows box using WD's Data Lifeguard for Windows (this works fine for any disk- not just WD's).

 

Share this post


Link to post

BTW, if you decide to boot an UnRAID flash drive on a different system to do the pre-clears, be CERTAIN you're pre-clearing the disk(s) you want -- NOT one of the actual disks from that system !!

 

The safest way is to simply disconnect all of the other disks and only have the disk(s) you need to pre-clear attached to the SATA ports on that system.    Then there's ZERO chance of making a mistake  :)

Share this post


Link to post

Garycase, Joe L, bjp99 or Trurl,

 

I just got another 5TB drive to help with my "problem" - (see Help Please??? :-X).  Somewhere I remember seeing one of you talking about removing Hard Drives from External Cases for use in UnRaid and having to "clear" the "hidden" partition that Seagate/WD may use.  I can't find the command or link using search, but believe it allowed us to look for that partition and then delete it for Pre-Clear.  I'm having the same problem with this disk as my others, it loses connection with the Sata controller/UnRaid Pre-Clear script at about the time it gets through the initial round (of 3 @~20 hours) and leaves no trace or record on the Flash Drive to analyze/share.

 

This is very frustrating >:( and is probably d/t some mfg "protection" to prevent us purchasing less expensive larger hard drives via the External USB route vs "raw" desktop drives.  I'm referring to 5TB drives for ~ $120 or less vs $150-$250 or more for the same/similar drive.

 

Dave

Share this post


Link to post

Garycase, Joe L, bjp99 or Trurl,

 

I just got another 5TB drive to help with my "problem" - (see Help Please??? :-X).  Somewhere I remember seeing one of you talking about removing Hard Drives from External Cases for use in UnRaid and having to "clear" the "hidden" partition that Seagate/WD may use.  I can't find the command or link using search, but believe it allowed us to look for that partition and then delete it for Pre-Clear.  I'm having the same problem with this disk as my others, it loses connection with the Sata controller/UnRaid Pre-Clear script at about the time it gets through the initial round (of 3 @~20 hours) and leaves no trace or record on the Flash Drive to analyze/share.

 

This is very frustrating >:( and is probably d/t some mfg "protection" to prevent us purchasing less expensive larger hard drives via the External USB route vs "raw" desktop drives.  I'm referring to 5TB drives for ~ $120 or less vs $150-$250 or more for the same/similar drive.

 

Dave

 

I had the same experience.

Hard drive probably has a small hidden partition as the first partition for recovery or a drive larger than 2tb may have a protected EFI partition

Mount the drive on a Windows machine. My computer right click >Manage>>disk storage >Disk management. You should be able to see the hidden partion then follow the instructions in the link.

http://www.howtogeek.com/215349/how-to-remove-an-efi-system-partition-or-gpt-protective-partition-from-a-drive-in-windows

 

Share this post


Link to post

Trying to setup a "burn box" with a trial copy of unRAID 6.1.3. Set it up and tried to run preclear_disk.sh -l, but I get an error saying it cannot find disk.cfg on line 202.

 

Tried simply running preclear_disk.sh /dev/sdb and got the same error about disk.cfg.

 

Thoughts?

Share this post


Link to post

Trying to setup a "burn box" with a trial copy of unRAID 6.1.3. Set it up and tried to run preclear_disk.sh -l, but I get an error saying it cannot find disk.cfg on line 202.

 

Tried simply running preclear_disk.sh /dev/sdb and got the same error about disk.cfg.

 

Thoughts?

 

Disk.cfg needs to exist, you can create a one disk array on the test server and then do a new config so the disk becomes available to preclear, or copy the disk.cfg from you other server.

Share this post


Link to post

Disk.cfg needs to exist, you can create a one disk array on the test server and then do a new config so the disk becomes available to preclear, or copy the disk.cfg from you other server.

 

So start the array with the disk to be precleared so it builds "disk.cfg", stop the array, remove the drive from array, build "new config", then preclear_disk.sh should work?

Share this post


Link to post

Disk.cfg needs to exist, you can create a one disk array on the test server and then do a new config so the disk becomes available to preclear, or copy the disk.cfg from you other server.

 

So start the array with the disk to be precleared so it builds "disk.cfg", stop the array, remove the drive from array, build "new config", then preclear_disk.sh should work?

 

I just tried it to make sure and in fact you only need to do a new config, it creates the disk.cfg

Share this post


Link to post

I had the same experience.

Hard drive probably has a small hidden partition as the first partition for recovery or a drive larger than 2tb may have a protected EFI partition

Mount the drive on a Windows machine. My computer right click >Manage>>disk storage >Disk management. You should be able to see the hidden partion then follow the instructions in the link.

http://www.howtogeek.com/215349/how-to-remove-an-efi-system-partition-or-gpt-protective-partition-from-a-drive-in-windows

 

Hi Thestewman ;), Just a quick note of thanks, I've done a lot of Windows Disk Management, but not the Diskpart>Select Disk>Clear routine.  I tried to download Gary's "hdat2" but working via a Mac through Parallels didn't quite work.  Interestingly, I had to use the Mac Disk Utility to replace what Pre-Clear had attempted to write in order for Windows to actually see the disk so I could "Clear" it via the Command Line.  If one tries to Pre-Clear and it fails for the reasons noted then it may take a bit of a process to be able to Clear a hidden and perhaps invisible partition even to Windows Disk Management.  Of note, I had to go through the Clear twice to obtain a confirmation.

 

Disk is currently being Pre-Cleared, so we'll see if this has resolved my (perhaps other's) problem.

 

Dave

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.