Re: preclear_disk.sh - a new utility to burn-in and pre-clear disks for quick add

[dynamis_dk]

Does the preclear_disk.sh create a log file by default?? or is it possible to push output to a file??

 

The script output is spanned across two screens so the top line is just the '== disk /dev/sdb has been successfully precleared' and i'd like to see the whole output. Afraid i ran the preclear script from the server itself not a putty session so i can't just scroll up

 

Afraid i'm a Windows person so i would normally do this like 'preclear_disk.bat >> preclear_results.log'

 

Is there something similar??

 

[Joe L.]

Does the preclear_disk.sh create a log file by default?? or is it possible to push output to a file??

 

The script output is spanned across two screens so the top line is just the '== disk /dev/sdb has been successfully precleared' and i'd like to see the whole output. Afraid i ran the preclear script from the server itself not a putty session so i can't just scroll up

 

Afraid i'm a Windows person so i would normally do this like 'preclear_disk.bat >> preclear_results.log'

 

Is there something similar??

 

It is in your system log.  Individual SMART reports are in /tmp

and in the "next" version (which I'm working on) the final report will also be saved to your flash drive.

 

Joe L.

[umax]

Hm, one of my brand new WD20EARS refuse to preclear, it stops at 3% of pre-read. I have tried on mb sata ports, saslp-mv8 and even second unraid server, same thing all the time. It never spits out any error and I can terminate it with ctrl+C. Any other test I could try if the drive is bad?

[Joe L.]

Afraid i ran the preclear script from the server itself not a putty session so i can't just scroll up

Actually, you can.

 

If you hold the Shift key down, PageUp will scroll up through recent output to the

virtual terminal.

 

Joe L.

[Joe L.]

@Joe, looks like an excellent script - great work!!

 

I'm slightly worried that in the future I might at one time forget to use the "-A" switch for AF drives. Would it be possible for you to read out the official unRAID "Default partition format" setting and use that as the default for your script? Reading out that setting would also be helpful once unRAID adds support for GPT partitions.

This feature is now in place in the new 1.3 version of the preclear script. 

You can override the "Default Partition Format" setting with either the "-A" (for sector 64) or "-a" (for sector 63) start positions for the partition.

If you give neither the "-a" or "-A" option, the setting from "Default Partition Format" is used.

 

Joe L.

[kizer]

Joe L. how about putting pre_clear in unMENU so we can click, click update?

[Mntneer]

Can I run preclear on a new drive, while not taking the array off-line?  Or is it best to take the array off-line and let preclear run?

[BRiT]

Can I run preclear on a new drive, while not taking the array off-line?  Or is it best to take the array off-line and let preclear run?

 

It has no effect on the array, so feel free to keep it online.

[Joe L.]

Can I run preclear on a new drive, while not taking the array off-line?  Or is it best to take the array off-line and let preclear run?

That was the whole idea of me originally creating the preclear script.  If I let unRAID clear the disk it was offline for a few hours.. and this was with the large 250Gig drives I had back then.

 

By all means, keep the array online.  Stop it only to assign the newly cleared drive.

 

Joe L.

[Mntneer]

Thanks guys 

[tkenn1s]

How long will it "normally" take to pre-clear a 2TB WD EARS drive? It's been running for just over 23 hours now and is about 80% through Step 10.

[Joe L.]

How long will it "normally" take to pre-clear a 2TB WD EARS drive? It's been running for just over 23 hours now and is about 80% through Step 10.

For most people, it takes between 33 and 35 hours to clear a 2TB drive.  It sounds like you will be in that range.

 

Generally reading and writing averages somewhere around 80MB/s.

 

Joe L.

[Benni-chan]

don't know, if this has been fixed in 1.3 (currently running 1.2):

it's just a cosmetical error. there is no step 9, but two step 10...

= Disk Pre-Clear-Read completed                                 DONE

= Step 1 of 10 - Copying zeros to first 2048k bytes             DONE

= Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE

= Step 3 of 10 - Disk is now cleared from MBR onward.           DONE

= Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4       DONE

= Step 5 of 10 - Clearing MBR code area                         DONE

= Step 6 of 10 - Setting MBR signature bytes                    DONE

= Step 7 of 10 - Setting partition 1 to precleared state        DONE

= Step 8 of 10 - Notifying kernel we changed the partitioning   DONE

= Step 10 of 10 - Verifying if the MBR is cleared.     DONE

= Step 10 of 10 - Verifying the clear has been successful.

 

 

Edit:

I'm currently running the 5.0-beta3 (yes, I know), and after the post read process there were some errors:

 

./preclear_disk.sh: line 1694: heDisk: command not found

./preclear_disk.sh: line 1726: syntax error near unexpected token `then'

./preclear_disk.sh: line 1726: `          then'

 

[Joe L.]

don't know, if this has been fixed in 1.3 (currently running 1.2):

it's just a cosmetical error. there is no step 9, but two step 10.

I think I've fixed that... but I'll take a look.  The second line should have been rendered on line higher.

 

The "syntax" error is more interesting.  It looks like a error where "$theDisk" was seen as "heDisk"

I do not see it in the 1.3 version.  And I don't see it in the 1.2 version either.

Perhaps you can type the following command post the output.

sed -n "1690,1733p" preclear_disk.sh

 

[Benni-chan]

here is the output:

    dd if=$theDisk count=1 | od -x
    if [ $use_mail -ge 1 ]
    then
        if [ "$verify_only" = "y" ]
        then
            echo -e "Preclear Verify Disk $theDisk FAILED!!!!.  " | mail -s "Preclear: FAIL! Verify Disk $disk_basename Failed!!! Cycle $cc of $cycle_count" $mail_rcpt
        else
            echo -e "Preclear Disk $theDisk FAILED!!!!.  " | mail -s "Preclear: FAIL! Preclearing Disk $disk_basename Failed!!! Cycle $cc of $cycle_count" $mail_rcpt
        fi
    fi

    exit 1
  else
    if [ "$post_read_flag" = "y" ]
    then
      posttmr=$(timer) #get postread start time
      read_entire_disk $theDisk postread display_progress
    fi
  fi
  dcyctmr=$(timer $cyctmr) #delta cycle time
  cyctmr=$(timer)  #reset cycle timer
  if [ $use_mail -ge 2 ]
      then
      if [ $cc -lt $cycle_count ]
          then
          report_out="========================================================================$ver\n"
          report_out+="==\n"
          report_out+="== Disk $theDisk has successfully finished a preclear cycle\n"
          report_out+="==\n"
          report_out+="== Finished Cycle $cc of $cycle_count cycles\n"
          report_out+="==\n"
          if [ "$pre_read_flag" = "y" ]  # Don't report Pre read time if there was no Pre/Post read.
          then
              report_out+="== Using read block size = `format_number ${units}` Bytes\n"
              report_out+="== Last Cycle's Pre Read Time  : $dpretmr ($cal_pre_read_speed MB/s)\n"
          fi
          if [ "$verify_only" != "y" ] # Don't report zero time if there was no zeroing.
          then
              report_out+="== Last Cycle's Zeroing time   : $dzerotmr ($cal_zero_write_speed MB/s)\n"
          fi
          if [ "$pre_read_flag" = "y" ] # Don't report Post read time if there was no Pre/Post read.
          then
              report_out+="== Last Cycle's Post Read Time : $dposttmr ($cal_post_read_speed MB/s)\n"
          fi

[Carpet3]

Having a problem with a new 10EARS hdd.

 

I've run the preclear twice now and each time it fails at stage 10 and says the drive cant be completed.

 

I've run DLGDIAG afterwards and on a quick scan it says the drive is fine but on a long scan it says errors were found but fixed.

 

I've just run it again on a long scan and it says there were no errors detected with the drive. It's now doing a third preclear so i'll see what happens.

 

Is there anything else I can do or it the drive on the way out?

 

 

Edit: added pic

[Carpet3]

Would clearing the mbr help?

 

Weird thing is this is very consistent

[Joe L.]

Would clearing the mbr help?

 

Weird thing is this is very consistent

That last line, "0 bytes copied" indicates the drive has stopped responding at all.  Right now it cannot be read, and probably not written either.  Right now, if you type

fdisk -l /dev/sdb

it will probably not return any output.

 

In every case in the past, at least with EARS drives, it will take a power cycle to get that drive to respond to anything.

If you power cycle (power down the server, then restore power), and then clear the mbr, and then try again, you might be successful.

 

It has been the experience of other unRAID users that clearing the MBR will let the drive complete a preclear cycle.

 

Joe L.

[Carpet3]

My apologies, upon searching this thread for the command to clear the mbr it appears i'm not the first with this sort of problem.

 

Can I just confirm with you that to clear the mbr you need to use the -n switch and then just stop the process once the clearing of the rest of the drive starts?

 

I assume once that's done I can just restart the server and do a normal preclear?

[Joe L.]

My apologies, upon searching this thread for the command to clear the mbr it appears i'm not the first with this sort of problem.

True

Can I just confirm with you that to clear the mbr you need to use the -n switch and then just stop the process once the clearing of the rest of the drive starts?

Unfortunately, the blocks in the MBR are not zeroed until step5.  After the bulk of the drive has been cleared.  You cannot abort it in the first few seconds and have the MBR cleared.

  step5="Step 5 of 10 - Clearing MBR code area"

So, the -n option is not what you need.

I assume once that's done I can just restart the server and do a normal preclear?

After the MBR is zeroed, you can do a normal preclear.

 

I'm doing a test on a version of preclear_disk.sh that has a new "-z" option that will let you zero the MBR of a drive. (and only zero the MBR)  It runs in a few seconds after you confirm you want to zero the MBR and should make it much easier for users like yourself.

 

I should be able to post it in about 20 or 30 minutes. 

Otherwise, if you don't want to wait to get the newest version of the preclear script, you will need to use the following BEFORE running the preclear script.

dd if=/dev/zero count=1 of=/dev/sdX

 

Joe L.

[Joe L.]

I'm doing a test on a version of preclear_disk.sh that has a new "-z" option that will let you zero the MBR of a drive. (and only zero the MBR)  It runs in a few seconds after you confirm you want to zero the MBR and should make it much easier for users like yourself.

 

I should be able to post it in about 20 or 30 minutes.  

Otherwise, if you don't want to wait to get the newest version of the preclear script, you will need to use the following BEFORE running the preclear script.

dd if=/dev/zero count=1 of=/dev/sdX

 

Joe L.

The new version 1.4 of the preclear_disk.sh script is now attached to the first post in this thread.

It includes a new "-z" option that will allow you to quickly and easily zero out the MBR of a disk.  It does NOT clear the entire disk, but might help those with EARS drive that need to have their MBR zeroed when they have had their jumper setting changed and the disk locks up when being cleared. (actually, regardless why they locked up it may help.  Zeroing the MBR seems to help and allow a subsequent clear to complete.)

 

To use it, type

preclear_disk.sh -z /dev/sdX

It will only be possible to perform this command on a disk not assigned to your array.   It may not be combined with the other options to preclear_disk.  If the disk has stopped responding, it will probably be necessary to power cycle it before it will respond to any command.

 

Joe L.

[Carpet3]

Thanks very much for this. I've just cleared the mbr on the drive using the -z option and have now started a new -A preclear. See what happens

 

You truly are a huge asset to this forum

[Carpet3]

The cycle has just finished. It went through all the steps ok (monitored it and watched it as it went through)

 

Each step completed but I get the same error at the end.

 

The drive wont respond at the moment but a powercycle will get it back up and running again

 

I've no idea what to do

[Joe L.]

The cycle has just finished. It went through all the steps ok (monitored it and watched it as it went through)

 

Each step completed but I get the same error at the end.

 

The drive wont respond at the moment but a powercycle will get it back up and running again

 

I've no idea what to do

The abbreviation RMA comes to mind.

 

Did you power-cycle both before AND after zeroing the MBR.  (Remember, before you power cycled, the drive would have been unwritable)

 

[Carpet3]

I power cycled before wiping the MBR but not after.

 

I'll try that now. This is so frustrating!