Re: preclear_disk.sh - a new utility to burn-in and pre-clear disks for quick add

[actionfreak]

Sorry... thought this was the other thread

[spinbot]

My computer crashed (not my UnRaid Server) while I was pre-clearing a hard drive.

 

I lost my telnet window, on my main computer, that I was watching its progress on.

 

Is the pre-clear still running on the server?  If so, how do I get the progress of it on my telnet screen again or how do I get to the results, assuming its completed or near completion now?

 

Thanks

[jbuszkie]

Were you running it using the "screen" program?  If yes..  The just reconnect to it..  But it sounds like you were jsut telneted into it... in which

case...  you have to start over...

 

Screen will run in the background and you can connect to the session or disconnect as many times as you want and the session is

still there...

 

I have had this happen before so I always run using screen!

 

Jim

[spinbot]

Crap (it was on the last part of the preclear and about halfway through it)

 

I see the "Screen" package available in UnMenu's Package Manager.

 

I tried to download it, however once I click on the download, I just get this on the blank screen:

e

</BODY></HTML>

0

 

I found instructions on how to manually install it, which appears to have worked when I type "screen" at my telnet prompt.

 

For reference, this post gives some decent direction on how to use it:

http://lime-technology.com/forum/index.php?topic=2817.msg24827#msg24827

crash.txt

[Joe L.]

I tried to download it, however once I click on the download, I just get this on the blank screen:

e

</BODY></HTML>

0

That usually indicates you do not have a nameserver defined in your settings. 

 

What do you get when you type:

net lookup google.com

ping -c 5 google.com

 

[spinbot]

This is the result:

root@Tower:~# net lookup google.com

root@Tower:~# ping -c 5 google.com

ping: unknown host google.com

 

I did get the package installed manually, so I just logged in to the server with Putty.   I've run "screen" and I am going to re-start the pre-clear.

 

I am then going to try:

Control-A d

 

Then log out of Putty.

 

The log back in and run:

screen -r

 

And see if my process is still running  ( simulate my computer crashing as it likes to do it often lately, but that another story )  

 

 

EDIT:  Looks to be doing exactly what I need it to.    Only question remaining is if I do not press Control-A d and my computer crashes while I am looking at the pre-clear process screen, can I still log back into telnet and run the screen -r command ?

 

[papnikol]

EDIT:  Looks to be doing exactly what I need it to.    Only question remaining is if I do not press Control-A d and my computer crashes while I am looking at the pre-clear process screen, can I still log back into telnet and run the screen -r command ?

 

Yes, you can

[Joe L.]

This is the result:

root@Tower:~# net lookup google.com

root@Tower:~# ping -c 5 google.com

ping: unknown host google.com

Based on that output:

 

You do not have both a DNS Nameserver and a Gateway defined in your unRAID "Settings" screen before you can use the unMENU package manager to download packages.   For most people, these should both be set to the IP address of your router.

Once you set them the package manager will work for downloads.

 

It should have looked like this:

root@Tower:/boot# net lookup google.com

74.125.229.17

root@Tower:/boot# ping -c 5 google.com

PING google.com (74.125.229.17) 56(84) bytes of data.

64 bytes from google.com (74.125.229.17): icmp_seq=1 ttl=55 time=19.5 ms

64 bytes from google.com (74.125.229.17): icmp_seq=2 ttl=55 time=19.1 ms

64 bytes from google.com (74.125.229.17): icmp_seq=3 ttl=55 time=22.3 ms

64 bytes from google.com (74.125.229.17): icmp_seq=4 ttl=55 time=18.9 ms

64 bytes from google.com (74.125.229.17): icmp_seq=5 ttl=55 time=18.9 ms

 

--- google.com ping statistics ---

5 packets transmitted, 5 received, 0% packet loss, time 4035ms

rtt min/avg/max/mdev = 18.924/19.784/22.381/1.319 ms

 

 

Joe L.

[spinbot]

I've changed routers a month ago and it has a different IP address, so that could be the issue.

 

I switched to Linksys Router that I installed Tomato/MLPPP on as I am running MultiLink with 2 DSL lines.  It ensures my phone line carrier doesn't throttle my connection, plus I don't have the greatest lines, so doubling my DSL speeds is handy.

 

In the UnRaid setup screen, I am set to "yes" for "Obtain DNS Server Address Automatically"

 

I'm guessing that isn't working properly, so I should assign DNS servers instead.

 

[papnikol]

Hi Joe

I was trying to preclear an EARS 2T and i got the same error i have had with the same type of disk some time ago: it stoped at a point during preread and when i try again to precleal i get the message:

/boot/preclear_disk.sh: line 550: 1+( 2556951756)%() : syntax error: operand expected (error token is ") ")

 

Would you happen to have an update as to why this error occurs? is it preclear-related or would i have had the same kind of problem if i had put it straight to the array?

 

PS:I have two other 2Tb EARS HDDs that were precleared with no problems...

[seanchronnery]

I'm trying to preclear 3 wd20ears disks but so far am not having much luck following the instructions.

This is where I'm at so far;

 

I have put 'preclear_disk.sh' in the root of my flash drive through windows.  I put my flash drive back in my mobo and boot up my unRAID box.  After entering my username and password, I 'cd /boot'.  From there, I can bring up the help by typing 'preclear_disk.sh -?'.  I do not understand what all the '-parameters' really mean.  These disks are new so I would like to preclear and then they will become my array (first build).  Would someone be able to help me with the preclear script?  Where do I find the name of my disk I wish cleared? I know it's obvious to most of you guys but I'm really new.

 

To invoke this script you simply list the name of a disk you wish cleared as an argument to the command, as in this example:

cd /boot

preclear_disk.sh /dev/hdk

 

 

thanks in advance

sean

[Joe L.]

I'm trying to preclear 3 wd20ears disks but so far am not having much luck following the instructions.

This is where I'm at so far;

 

I have put 'preclear_disk.sh' in the root of my flash drive through windows.  I put my flash drive back in my mobo and boot up my unRAID box.  After entering my username and password, I 'cd /boot'.  From there, I can bring up the help by typing 'preclear_disk.sh -?'.  I do not understand what all the '-parameters' really mean.  These disks are new so I would like to preclear and then they will become my array (first build).  Would someone be able to help me with the preclear script?  Where do I find the name of my disk I wish cleared? I know it's obvious to most of you guys but I'm really new.

 

To invoke this script you simply list the name of a disk you wish cleared as an argument to the command, as in this example:

cd /boot

preclear_disk.sh /dev/hdk

 

 

thanks in advance

sean

If you have unRAID loaded and running to where you can access its web-interface you can see the three character devices names in parentases on the drop-down list on the "devices" page.  You'll not be able to see the drop-down list unless the array is stopped by pressing "Stop" on the management console.

 

The other way to see the list of drives and the three character "device" names is to type:

ls -l /dev/disk/by-id

 

It will look a lot like this (with different model/serial numbers for your disks, and different device names at the ends of the lines)

root@Tower:/boot/# ls -l /dev/disk/by-id

total 0

lrwxrwxrwx 1 root root  9 Dec 18 23:53 ata-HDS725050KLAT80_KRVA03ZAG3V5LD -> ../../hdb

lrwxrwxrwx 1 root root 10 Dec 18 23:53 ata-HDS725050KLAT80_KRVA03ZAG3V5LD-part1 -> ../../hdb1

lrwxrwxrwx 1 root root  9 Dec 18 23:53 ata-HDS725050KLAT80_KRVA03ZAG4V99D -> ../../hdd

lrwxrwxrwx 1 root root 10 Dec 18 23:53 ata-HDS725050KLAT80_KRVA03ZAG4V99D-part1 -> ../../hdd1

lrwxrwxrwx 1 root root  9 Dec 18 23:53 ata-MAXTOR_STM3500630A_5QG00FTK -> ../../hdl

lrwxrwxrwx 1 root root 10 Dec 18 23:53 ata-MAXTOR_STM3500630A_5QG00FTK-part1 -> ../../hdl1

lrwxrwxrwx 1 root root  9 Dec 18 23:53 ata-Maxtor_6Y250P0_Y63KH45E -> ../../sdc

lrwxrwxrwx 1 root root 10 Dec 18 23:53 ata-Maxtor_6Y250P0_Y63KH45E-part1 -> ../../sdc1

lrwxrwxrwx 1 root root  9 Dec 18 23:53 ata-QUANTUM_FIREBALLlct15_08_611020017228 -> ../../hdj

lrwxrwxrwx 1 root root 10 Dec 18 23:53 ata-QUANTUM_FIREBALLlct15_08_611020017228-part1 -> ../../hdj1

lrwxrwxrwx 1 root root  9 Dec 18 23:53 ata-ST31000340AS_9QJ0JPJS -> ../../sde

lrwxrwxrwx 1 root root 10 Dec 18 23:53 ata-ST31000340AS_9QJ0JPJS-part1 -> ../../sde1

 

 

You are not interested in the 4 character names ending with a number.  Those are the names of individual partitions on disks.  You want the three character device names starting with either an "h" or "s" followed by the letter "d" and then another letter from "a" through "z".  For SATA drives they'll be sdX, for IDE drives, they'll be hdX.  In the example above, I have a mix of IDE drives and SATA drives.

 

You'll need to identify the specific disk to be cleared by its model and serial number.  If the device name is "sde" you would invoke preclear_disk.sh as

preclear_disk.sh /dev/sde

 

Most of the other options shown when you invoked preclear_disk.sh with -? are not used for normal usage.  For now, you don't have to concern yourself about them.

 

Joe L.

 

[seanchronnery]

thank you so much, Joe L.

[Houseplant]

Hello everyone. I recently finished the hardware side of my first unRAID build and I ran into a major problem during the preclearing of my first two disks. I have tried running remotely through Telnet/Screen and also through the console on the server itself, but randomly during the first 30min of preclearing, my screen will suddenly become very garbled. It's worth noting that this also effectively kills all processes on the server and it requires a hard re-boot. Here is a link to that screen capture: http://img227.imageshack.us/i/photowco.jpg/

 

The two drives in question are a Samsung Spinpoint F4 (with latest firmware patch) and a WD 2TB EARS drive. They are running on an Asus M4A78LT-M motherboard with an AMD Sempron 140 processor (gogo budget builds!). I am giving it one more go while I get some sleep and I will post any syslogs I can find in the morning, but for now I am just wondering if anyone else has experienced this.

 

Thank you for your help!

[papnikol]

Hello everyone. I recently finished the hardware side of my first unRAID build and I ran into a major problem during the preclearing of my first two disks. I have tried running remotely through Telnet/Screen and also through the console on the server itself, but randomly during the first 30min of preclearing, my screen will suddenly become very garbled. It's worth noting that this also effectively kills all processes on the server and it requires a hard re-boot. Here is a link to that screen capture: http://img227.imageshack.us/i/photowco.jpg/

 

The two drives in question are a Samsung Spinpoint F4 (with latest firmware patch) and a WD 2TB EARS drive. They are running on an Asus M4A78LT-M motherboard with an AMD Sempron 140 processor (gogo budget builds!). I am giving it one more go while I get some sleep and I will post any syslogs I can find in the morning, but for now I am just wondering if anyone else has experienced this.

 

Thank you for your help!

 

this seems hardware related. Have you tried a memtest?

[Joe L.]

I have tried running remotely through Telnet/Screen and also through the console on the server itself, but randomly during the first 30min of preclearing, my screen will suddenly become very garbled. It's worth noting that this also effectively kills all processes on the server and it requires a hard re-boot. Here is a link to that screen capture: http://img227.imageshack.us/i/photowco.jpg/

It appears as if you have some memory problems.  You should probably run a memory test (overnight is best) to verify.

That is the most likely cause of the screen garble you are seeing.

 

The BIOS might be set wrong for your specific memory strips.  The clock speed, timing and voltage must be set for your specific memory strips.  Most BIOS set it correctly, some do not, especially if you installed premium RAM that has special needs.

 

Joe L.

[Houseplant]

Thank you both for the replies. I went ahead and swapped my 2x2GB sticks for a single 4GB stick. I figured I'd run a preclear for kicks to see if it would get past the points I was at before and so far it has already been running for twice as long as before so swapping the memory has certainly helped so far

[PCRx]

I Telneted in and started to preclear a new drive. After two hours the system that had the console session open locked up and had to be rebooted. It looks like the preclear on the unraid drive also stopped at that time. I expected it to take over 2 more hours.

 

Did the preclear script exit gracefully when it lost the session connection to the machine that locked up?

Can I just start the process over again or will there be "ghosts" from the previous attempt that need to be addressed?

 

Thanks.

 

[spinbot]

PCRx,

 

Go back one page in this forum and look at my posting.  I had an issue with my main PC Blue Screening on me, so I would have to re-start the pre-clear when that happened.    One of the guys pointed out something called "screen" which keeps your pre-clear session running, even if the machine you started it on is shut down.

 

It works well ( as I tested it out one minute into the process, just to make sure it did what it was suppose to ) 

[PCRx]

Thanks Spinbot. My main concern was there might be something still going on with the server since it didn't exit properly. I'm trying your Screen suggestion and just started preclearing again. It looks to be working fine.

[EMKO]

my preclear of a 2tb ears drive stops at 65% on pre read, i ran it 2 times and both times is stops at 65%.

on the drive i jumper 7 8

 

syslog i see this

 

Dec 27 07:44:44 Tower kernel: hdc: ide_dma_sff_timer_expiry: DMA status (0x21)

Dec 27 07:44:54 Tower kernel: hdc: DMA timeout error

Dec 27 07:44:54 Tower kernel: hdc: dma timeout error: status=0x58 { DriveReady SeekComplete DataRequest }

Dec 27 07:44:54 Tower kernel: hdc: possibly failed opcode: 0x25

Dec 27 07:44:54 Tower kernel: hdc: DMA disabled

Dec 27 07:44:54 Tower kernel: ide1: reset: success

 

 

 

in smart_start i have

 

smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build)

Copyright © 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net

 

=== START OF INFORMATION SECTION ===

Device Model:    WDC WD20EARS-00MVWB0

Serial Number:    WD-WMAZA1789360

Firmware Version: 51.0AB51

User Capacity:    2,000,398,934,016 bytes

Device is:        Not in smartctl database [for details use: -P showall]

ATA Version is:  8

ATA Standard is:  Exact ATA specification draft version not indicated

Local Time is:    Mon Dec 27 10:27:39 2010 Local time zone must be set--see zic m

SMART support is: Available - device has SMART capability.

SMART support is: Enabled

 

=== START OF READ SMART DATA SECTION ===

SMART overall-health self-assessment test result: PASSED

 

General SMART Values:

Offline data collection status:  (0x84) Offline data collection activity

was suspended by an interrupting command from host.

Auto Offline Data Collection: Enabled.

Self-test execution status:      (  0) The previous self-test routine completed

without error or no self-test has ever

been run.

Total time to complete Offline

data collection: (36480) seconds.

Offline data collection

capabilities: (0x7b) SMART execute Offline immediate.

Auto Offline data collection on/off support.

Suspend Offline collection upon new

command.

Offline surface scan supported.

Self-test supported.

Conveyance Self-test supported.

Selective Self-test supported.

SMART capabilities:            (0x0003) Saves SMART data before entering

power-saving mode.

Supports SMART auto save timer.

Error logging capability:        (0x01) Error logging supported.

General Purpose Logging supported.

Short self-test routine

recommended polling time: (  2) minutes.

Extended self-test routine

recommended polling time: ( 255) minutes.

Conveyance self-test routine

recommended polling time: (  5) minutes.

SCT capabilities:       (0x3035) SCT Status supported.

SCT Feature Control supported.

SCT Data Table supported.

 

SMART Attributes Data Structure revision number: 16

Vendor Specific SMART Attributes with Thresholds:

ID# ATTRIBUTE_NAME          FLAG    VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE

  1 Raw_Read_Error_Rate    0x002f  200  200  051    Pre-fail  Always      -      0

  3 Spin_Up_Time            0x0027  253  253  021    Pre-fail  Always      -      1041

  4 Start_Stop_Count        0x0032  100  100  000    Old_age  Always      -      10

  5 Reallocated_Sector_Ct  0x0033  200  200  140    Pre-fail  Always      -      0

  7 Seek_Error_Rate        0x002e  100  253  000    Old_age  Always      -      0

  9 Power_On_Hours          0x0032  100  100  000    Old_age  Always      -      8

10 Spin_Retry_Count        0x0032  100  253  000    Old_age  Always      -      0

11 Calibration_Retry_Count 0x0032  100  253  000    Old_age  Always      -      0

12 Power_Cycle_Count      0x0032  100  100  000    Old_age  Always      -      8

192 Power-Off_Retract_Count 0x0032  200  200  000    Old_age  Always      -      7

193 Load_Cycle_Count        0x0032  200  200  000    Old_age  Always      -      36

196 Reallocated_Event_Count 0x0032  200  200  000    Old_age  Always      -      0

197 Current_Pending_Sector  0x0032  200  200  000    Old_age  Always      -      0

198 Offline_Uncorrectable  0x0030  100  253  000    Old_age  Offline      -      0

199 UDMA_CRC_Error_Count    0x0032  200  200  000    Old_age  Always      -      0

200 Multi_Zone_Error_Rate  0x0008  100  253  000    Old_age  Offline      -      0

 

SMART Error Log Version: 1

No Errors Logged

 

SMART Self-test log structure revision number 1

No self-tests have been logged.  [To run self-tests, use: smartctl -t]

 

 

SMART Selective self-test log data structure revision number 1

SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS

    1        0        0  Not_testing

    2        0        0  Not_testing

    3        0        0  Not_testing

    4        0        0  Not_testing

    5        0        0  Not_testing

Selective self-test flags (0x0):

  After scanning selected spans, do NOT read-scan remainder of disk.

If Selective self-test is pending on power-up, resume after 0 minute delay.

 

 

[dwoods99]

You have 6 virtual consoles available if you attach a keyboard and monitor to your unRAID server.

 

ALT-F1 through ALT-F6 will switch between them.

 

You can run up to 6 pre-clear sessions, one on each virtual console.

 

The problem with this, as I discovered, is that you can't see the final 'diff' results because they scroll

above the screen. There's no way to page up to see what's been missed, and no log file to view.

 

It would be very helpful to create a final log of the results, per disk when it completes.

Even the ones that got pre-cleared, I have no way of checking for any output I should worry about.

 

Is there any way to re-run the script but only do the last part (after step 10 -- post-read)?

 

[Joe L.]

You have 6 virtual consoles available if you attach a keyboard and monitor to your unRAID server.

 

ALT-F1 through ALT-F6 will switch between them.

 

You can run up to 6 pre-clear sessions, one on each virtual console.

 

The problem with this, as I discovered, is that you can't see the final 'diff' results because they scroll

above the screen. There's no way to page up to see what's been missed, and no log file to view.

 

It would be very helpful to create a final log of the results, per disk when it completes.

Even the ones that got pre-cleared, I have no way of checking for any output I should worry about.

 

Is there any way to re-run the script but only do the last part (after step 10 -- post-read)?

 

I think shift-page-up will scroll upward on the virtual consoles

All the output is also logged to the system log.

 

Individual output files are in /tmp for each drive's pre and post smart report.

you can get the diff by typing

diff /tmp/filename1 /tmp/filename2

 

Joe L.

[dwoods99]

OK thanks for info on /tmp files, I got info from them although it's hard to tell which drive it belongs to.

It would be helpful to new users about syslog output if it was mentioned in the script's help output.

 

BTW, shift-PageUp etc does not work on console.  I only have 5.0b2 with nothing else installed.

 

[squirrellydw]

I should be getting my WD 2TB EARS drive today and I want to pre clear the drive 3 times, would this be correct?

 

cd /boot

preclear_disk.sh /dev/sdf-3