Re: preclear_disk.sh - a new utility to burn-in and pre-clear disks for quick add

[Thornwood]

Here is My log file. If i am correct it is showing many errors on this drive errors that the other drives dont have. So I am questioning if the drive might be starting to go bad. See Atached GIF

 

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: SMART Attributes Data Structure revision number: 16

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: Vendor Specific SMART Attributes with Thresholds:

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 3 Spin_Up_Time 0x0027 182 182 063 Pre-fail Always - 23933

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 4 Start_Stop_Count 0x0032 253 253 000 Old_age Always - 182

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 5 Reallocated_Sector_Ct 0x0033 253 253 063 Pre-fail Always - 1

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 6 Read_Channel_Margin 0x0001 253 253 100 Pre-fail Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 7 Seek_Error_Rate 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 8 Seek_Time_Performance 0x0027 252 240 187 Pre-fail Always - 35789

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 10 Spin_Retry_Count 0x002b 253 252 157 Pre-fail Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 11 Calibration_Retry_Count 0x002b 253 252 223 Pre-fail Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 12 Power_Cycle_Count 0x0032 253 253 000 Old_age Always - 197

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 192 Power-Off_Retract_Count 0x0032 253 253 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 193 Load_Cycle_Count 0x0032 253 253 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 195 Hardware_ECC_Recovered 0x000a 253 252 000 Old_age Always - 941

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 196 Reallocated_Event_Count 0x0008 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 197 Current_Pending_Sector 0x0008 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 198 Offline_Uncorrectable 0x0008 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 199 UDMA_CRC_Error_Count 0x0008 175 023 000 Old_age Offline - 173

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 200 Multi_Zone_Error_Rate 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 201 Soft_Read_Error_Rate 0x000a 253 236 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 202 TA_Increase_Count 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 203 Run_Out_Cancel 0x000b 253 252 180 Pre-fail Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 204 Shock_Count_Write_Opern 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 205 Shock_Rate_Write_Opern 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 207 Spin_High_Current 0x002a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 208 Spin_Buzz 0x002a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 209 Offline_Seek_Performnce 0x0024 192 191 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 99 Unknown_Attribute 0x0004 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 100 Unknown_Attribute 0x0004 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 101 Unknown_Attribute 0x0004 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]:

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: SMART Error Log Version: 1

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: Warning: ATA error count 352 inconsistent with error log pointer 5

[Joe L.]

Here is My log file. If i am correct it is showing many errors on this drive errors that the other drives dont have. So I am questioning if the drive might be starting to go bad. See Atached GIF

 

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: SMART Attributes Data Structure revision number: 16

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: Vendor Specific SMART Attributes with Thresholds:

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 3 Spin_Up_Time 0x0027 182 182 063 Pre-fail Always - 23933

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 4 Start_Stop_Count 0x0032 253 253 000 Old_age Always - 182

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 5 Reallocated_Sector_Ct 0x0033 253 253 063 Pre-fail Always - 1

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 6 Read_Channel_Margin 0x0001 253 253 100 Pre-fail Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 7 Seek_Error_Rate 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 8 Seek_Time_Performance 0x0027 252 240 187 Pre-fail Always - 35789

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 10 Spin_Retry_Count 0x002b 253 252 157 Pre-fail Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 11 Calibration_Retry_Count 0x002b 253 252 223 Pre-fail Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 12 Power_Cycle_Count 0x0032 253 253 000 Old_age Always - 197

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 192 Power-Off_Retract_Count 0x0032 253 253 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 193 Load_Cycle_Count 0x0032 253 253 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 195 Hardware_ECC_Recovered 0x000a 253 252 000 Old_age Always - 941

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 196 Reallocated_Event_Count 0x0008 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 197 Current_Pending_Sector 0x0008 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 198 Offline_Uncorrectable 0x0008 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 199 UDMA_CRC_Error_Count 0x0008 175 023 000 Old_age Offline - 173

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 200 Multi_Zone_Error_Rate 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 201 Soft_Read_Error_Rate 0x000a 253 236 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 202 TA_Increase_Count 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 203 Run_Out_Cancel 0x000b 253 252 180 Pre-fail Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 204 Shock_Count_Write_Opern 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 205 Shock_Rate_Write_Opern 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 207 Spin_High_Current 0x002a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 208 Spin_Buzz 0x002a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 209 Offline_Seek_Performnce 0x0024 192 191 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 99 Unknown_Attribute 0x0004 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 100 Unknown_Attribute 0x0004 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 101 Unknown_Attribute 0x0004 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]:

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: SMART Error Log Version: 1

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: Warning: ATA error count 352 inconsistent with error log pointer 5

 

All I see is the one re-allocated sector.  The others are meaningful only to the manufacturer.

 

You need to learn what the various columns mean.

 

If the VALUE is below the THRESH column, that attribute fails.  None of yours are even close to the failure threshold.

 

As far as the numbers... some manufacturers show them, some do not, all disks have internal retry counters and error correction. 

 

Of course, you can ignore what I've said and RMA a perfectly good drive.

 

Joe L. 

[Joe L.]

When doing a multi pass preclear I am wondering if there is any benifit to doing a preread for passes 2-n? Considering that a post read/verify just finished does doing another read pass before the write pass contribute to the processes? Running multi pass preclears on 2TB drives takes a long time.

 

Regardless, thanks Joe for a great utility. I added six precleared drives (2x1.5TB & 4x1TB) to my array with only a few minutes of down time. I also ran all six preclears at the same time. Worked perfectly.

 

If you change one line in the script it will do as you suggested.

At line 878, you'll find this "if"statment

 

if [ "$pre_read_flag" = "y" ]

  then

  pretmr=$(timer) #get preread start time

  read_entire_disk $theDisk preread display_progress

  fi

 

Add the text in blue below.  It will then not do a pre-read prior to a subsequent cycle if you use the -c option to specify multiple cycles since, as you noted, it just did a post-read.

if [ "$pre_read_flag" = "y"  -a  $cc = 1 ]

  then

  pretmr=$(timer) #get preread start time

  read_entire_disk $theDisk preread display_progress

  fi

 

[Thornwood]

Here is My log file. If i am correct it is showing many errors on this drive errors that the other drives dont have. So I am questioning if the drive might be starting to go bad. See Atached GIF

 

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: SMART Attributes Data Structure revision number: 16

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: Vendor Specific SMART Attributes with Thresholds:

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 3 Spin_Up_Time 0x0027 182 182 063 Pre-fail Always - 23933

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 4 Start_Stop_Count 0x0032 253 253 000 Old_age Always - 182

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 5 Reallocated_Sector_Ct 0x0033 253 253 063 Pre-fail Always - 1

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 6 Read_Channel_Margin 0x0001 253 253 100 Pre-fail Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 7 Seek_Error_Rate 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 8 Seek_Time_Performance 0x0027 252 240 187 Pre-fail Always - 35789

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 10 Spin_Retry_Count 0x002b 253 252 157 Pre-fail Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 11 Calibration_Retry_Count 0x002b 253 252 223 Pre-fail Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 12 Power_Cycle_Count 0x0032 253 253 000 Old_age Always - 197

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 192 Power-Off_Retract_Count 0x0032 253 253 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 193 Load_Cycle_Count 0x0032 253 253 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 195 Hardware_ECC_Recovered 0x000a 253 252 000 Old_age Always - 941

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 196 Reallocated_Event_Count 0x0008 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 197 Current_Pending_Sector 0x0008 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 198 Offline_Uncorrectable 0x0008 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 199 UDMA_CRC_Error_Count 0x0008 175 023 000 Old_age Offline - 173

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 200 Multi_Zone_Error_Rate 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 201 Soft_Read_Error_Rate 0x000a 253 236 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 202 TA_Increase_Count 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 203 Run_Out_Cancel 0x000b 253 252 180 Pre-fail Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 204 Shock_Count_Write_Opern 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 205 Shock_Rate_Write_Opern 0x000a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 207 Spin_High_Current 0x002a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 208 Spin_Buzz 0x002a 253 252 000 Old_age Always - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 209 Offline_Seek_Performnce 0x0024 192 191 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 99 Unknown_Attribute 0x0004 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 100 Unknown_Attribute 0x0004 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: 101 Unknown_Attribute 0x0004 253 253 000 Old_age Offline - 0

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]:

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: SMART Error Log Version: 1

Jun 17 06:45:29 Dragon-Tower preclear_disk-finish[16662]: Warning: ATA error count 352 inconsistent with error log pointer 5

 

All I see is the one re-allocated sector.   The others are meaningful only to the manufacturer.

 

You need to learn what the various columns mean.

 

If the VALUE is below the THRESH column, that attribute fails.   None of yours are even close to the failure threshold.

 

As far as the numbers... some manufacturers show them, some do not, all disks have internal retry counters and error correction. 

 

Of course, you can ignore what I've said and RMA a perfectly good drive.

 

Joe L.   

 

Thank you I was under the inpression that a crc error was a bad thing but I trust what you say thank you.

One thing on the screen shot why does it show the errors in red? just to make you aware?

I will add the drave as a data drive.

[Joe L.]

One thing on the screen shot why does it show the errors in red? just to make you aware?

In unMENU's syslog viewer ANY line with the word "error" in it will be highlighted in red.  Even if it said "No errors found"

 

The patterns matched and resulting colors are defined in syslog_match.conf

 

Joe L

[greybeard]

if [ "$pre_read_flag" = "y"  -a  $cc = 1 ]

 

Made the change using wordpad from Windows 7. Suspect I might have broken the script by adding CRs, will find out when I try to run it. One way or another I will get it working even if I have figure out how to edit the script using a UNIX editor. My two new 2TB drives arrived in the mail today so this will save quite a bit of time preclearing them.

 

Thanks!

Jerry

[Joe L.]

if [ "$pre_read_flag" = "y"  -a  $cc = 1 ]

 

Made the change using wordpad from Windows 7. Suspect I might have broken the script by adding CRs, will find out when I try to run it. One way or another I will get it working even if I have figure out how to edit the script using a UNIX editor. My two new 2TB drives arrived in the mail today so this will save quite a bit of time preclearing them.

 

Thanks!

Jerry

http://lime-technology.com/wiki/index.php?title=FAQ#Why_do_my_scripts_have_problems_with_end-of-lines.3F

[greybeard]

Made the change using wordpad from Windows 7. Suspect I might have broken the script

 

I did in fact breake the script. Wordpad did something bad to it. Downloaded and istalled Notepad2. Made the edit using a clean copy of the script and it is working now. Started 6 more preclears, four of them are in their 2nd or 3rd pass, the preread is being skipped after pass 1.

 

Joe: Thanks again for the update and link.

[barrygordon]

There is a free editor called Textpad.  It has many features that neither Notepad nor Wordpad have.  Basically it was built with programmers in mind.

 

One of its features is that it handles both kinds of line endings "PC which is (CR)(LF) and "unix" which is just (CR).  When it loads a file it notes what the line endings are and sets its state accordingly.  Do not know what happens if the file has mixed endings.  When you save the file it tells you wether it is using PC or Unix line endings.

 

If you have broken a Unix file in a PC editor, then Textpad will load it as a PC file. When you save it (Save AS) you just change the file mode to Unix and you are all fixed.

[Thornwood]

I use VIM 7.2 it has cool features like it makes backups of files and it shows files in color based on command routines. It can be used for all the UNIX and dos editing.

Free too. As a plus it has a gui that teaches you VI commands.

[Swampy]

Here is a newbie question.

 

If I have the Unregistered unRAID and have all three drives (sda,sdb & sdc) assigned in the array, can I still run the preclear_disk.sh from the console or putty for the sdd drive?

 

Thanks

 

[Joe L.]

Here is a newbie question.

 

If I have the Unregistered unRAID and have all three drives (sda,sdb & sdc) assigned in the array, can I still run the preclear_disk.sh from the console or putty for the sdd drive?

 

Thanks

 

Yes, no problem at all. 

 

Even with the un-registered version you can have as many drives connected outside of the array as your disk controller/power supply can handle.  You can only assign three to the unRAID array, one parity drive and two data drives in the un-registered version, but you can physically have many more outside of the protected array. 

 

As long as they are not logically "mounted" on a mount-point directory, you can clear them.

 

Joe L.

[Swampy]

Thanks Joe L.

 

I thought that may of been the case but wanted to know for sure.

[kizer]

I hope this doesn't sound stupid, but I'm going to ask anyways.

 

I ran the preclear earlier on a test drive and my real drives are on the way. I ran the preclear from console and I often see people posting their results. I'm guessing they are posting them from a Putty window or telnet.

 

How do you or can post a log from console?

[Joe L.]

I hope this doesn't sound stupid, but I'm going to ask anyways.

 

I ran the preclear earlier on a test drive and my real drives are on the way. I ran the preclear from console and I often see people posting their results. I'm guessing they are posting them from a Putty window or telnet.

 

How do you or can post a log from console?

Basically, you can't cut-paste from the console.

 

The results were also logged to your syslog.  You can find them there.

 

Unless you have already read the posts on how to interpret the results, in this and the other preclear-results thread, and your disk has something I've never seen before,  you really don't need to post anything. 

 

I really don't need to repeat what I've posted dozens of times.

Look for re-allocated sectors and sectors pending re-allocation.  Just about everything else is meaningful to the manufacturer, and not to me, or to you.

 

Joe L.

[WeeboTech]

Look for re-allocated sectors and sectors pending re-allocation.  Just about everything else is meaningful to the manufacturer, and not to me, or to you.

 

These are some of the most important values I've seen displayed by other nas units.

 

In working with my thecus 5200BR, temperature, re-allocated sectors and sectors pending re-allocation were the only smart status displayed.

[kizer]

Thanks guys. I try to read the threads looking for the obvious, but sometimes I don't catch it. Yeah I'll rip more through the results thread that I have been watching looking for things that stand out.

 

I figured it had to log it somewhere and obviously the syslog does sound like the logical location.

[comfox]

Is there way to "pause" this script once it is going?  I am finding that it is taking about 2.5 days to pre-clear a 2TB drive and during daylight hours I would like to use the NAS without that strain that this script is causing on the NAS.

[Joe L.]

Is there way to "pause" this script once it is going?  I am finding that it is taking about 2.5 days to pre-clear a 2TB drive and during daylight hours I would like to use the NAS without that strain that this script is causing on the NAS.

No... sorry.

 

2.5 days seems long.  I suspect something is up, unless you are really using older hardware.  There is no real strain on the server itself by the pre-clear, but it is putting a lot of strain on the drive being cleared.

 

What symptoms are you seeing when using the server otherwise?

 

Most 2TB drives will take 30 to 35 hours.

[comfox]

Thanks Joe, great script otherwise.  I am just seeing random pausing when accessing (either putting or getting) stuff fromt the server.  It could very much be another drive I added but I am in the process of clearing this 2TB drive so don't want to do too much testing.  It has been running for 24 hours now but is only 49% done on step 2 of 10 so I just extrapolated that to be 2.5 days.

[kizer]

What Joe L said. I precleared a 2TB drive while using the server for normal file acess. Seemed to work just fine. As soon as it was done, which I belive was around 36hours I added it to the array.

[slowrolling]

I was running preclear from a console connection, the console connection got reset overnight.  Is there an easy way to check if it is still running and or a log of results?

 

Or am I SOL and just need to start over?

 

[EDIT] Looks like I should read the thread first, I can it in syslog correct? [/EDIT]

[Joe L.]

I was running preclear from a console connection, the console connection got reset overnight.  Is there an easy way to check if it is still running and or a log of results?

 

Or am I SOL and just need to start over?

 

[EDIT] Looks like I should read the thread first, I can it in syslog correct? [/EDIT]

Yes, you can look in the syslog to see if the post-clear smart report is there.  Otherwise, best to start again.

 

You are in good company, we lost power here three times last night when thunderstorms went by.  Fortunately, the unRAID server is on a UPS, and I was not doing anything that got interrupted..

[slowrolling]

So here is my syslog... judging by my timestamps, and comparing them to my current time stamp something seems fishy

 

Jul 13 08:51:49 vault in.telnetd[1399]: connect from 192.168.1.101 (192.168.1.101)

Jul 13 08:51:55 vault login[1400]: ROOT LOGIN on `pts/1' from `192.168.1.101'

Jul 13 08:52:21 vault kernel: sdb: unknown partition table

Jul 13 08:53:35 vault preclear_disk-start[1561]: smartctl version 5.38 [i486-slackware-linux-gnu] Copyright © 2002-8 Bruce Allen

Jul 13 08:53:35 vault preclear_disk-start[1561]: Home page is http://smartmontools.sourceforge.net/

Jul 13 08:53:35 vault preclear_disk-start[1561]:

Jul 13 08:53:35 vault preclear_disk-start[1561]: === START OF INFORMATION SECTION ===

Jul 13 08:53:35 vault preclear_disk-start[1561]: Device Model: WDC WD20EARS-00MVWB0

Jul 13 08:53:35 vault preclear_disk-start[1561]: Serial Number: WD-WMAZ20198286

Jul 13 08:53:35 vault preclear_disk-start[1561]: Firmware Version: 50.0AB50

Jul 13 08:53:35 vault preclear_disk-start[1561]: User Capacity: 2,000,398,934,016 bytes

Jul 13 08:53:35 vault preclear_disk-start[1561]: Device is: Not in smartctl database [for details use: -P showall]

Jul 13 08:53:35 vault preclear_disk-start[1561]: ATA Version is: 8

Jul 13 08:53:35 vault preclear_disk-start[1561]: ATA Standard is: Exact ATA specification draft version not indicated

Jul 13 08:53:35 vault preclear_disk-start[1561]: Local Time is: Tue Jul 13 08:53:35 2010 GMT+8

Jul 13 08:53:35 vault preclear_disk-start[1561]: SMART support is: Available - device has SMART capability.

Jul 13 08:53:35 vault preclear_disk-start[1561]: SMART support is: Enabled

Jul 13 08:53:35 vault preclear_disk-start[1561]:

Jul 13 08:53:35 vault preclear_disk-start[1561]: === START OF READ SMART DATA SECTION ===

Jul 13 08:53:35 vault preclear_disk-start[1561]: SMART overall-health self-assessment test result: PASSED

Jul 13 08:53:35 vault preclear_disk-start[1561]:

Jul 13 08:53:35 vault preclear_disk-start[1561]: General SMART Values:

Jul 13 08:53:35 vault preclear_disk-start[1561]: Offline data collection status: (0x80)^IOffline data collection activity

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^Iwas never started.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^IAuto Offline Data Collection: Enabled.

Jul 13 08:53:35 vault preclear_disk-start[1561]: Self-test execution status: ( 0)^IThe previous self-test routine completed

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^Iwithout error or no self-test has ever

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^Ibeen run.

Jul 13 08:53:35 vault preclear_disk-start[1561]: Total time to complete Offline

Jul 13 08:53:35 vault preclear_disk-start[1561]: data collection: ^I^I (37800) seconds.

Jul 13 08:53:35 vault preclear_disk-start[1561]: Offline data collection

Jul 13 08:53:35 vault preclear_disk-start[1561]: capabilities: ^I^I^I (0x7b) SMART execute Offline immediate.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^IAuto Offline data collection on/off support.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^ISuspend Offline collection upon new

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^Icommand.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^IOffline surface scan supported.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^ISelf-test supported.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^IConveyance Self-test supported.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^ISelective Self-test supported.

Jul 13 08:53:35 vault preclear_disk-start[1561]: SMART capabilities: (0x0003)^ISaves SMART data before entering

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^Ipower-saving mode.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^ISupports SMART auto save timer.

Jul 13 08:53:35 vault preclear_disk-start[1561]: Error logging capability: (0x01)^IError logging supported.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^IGeneral Purpose Logging supported.

Jul 13 08:53:35 vault preclear_disk-start[1561]: Short self-test routine

Jul 13 08:53:35 vault preclear_disk-start[1561]: recommended polling time: ^I ( 2) minutes.

Jul 13 08:53:35 vault preclear_disk-start[1561]: Extended self-test routine

Jul 13 08:53:35 vault preclear_disk-start[1561]: recommended polling time: ^I ( 255) minutes.

Jul 13 08:53:35 vault preclear_disk-start[1561]: Conveyance self-test routine

Jul 13 08:53:35 vault preclear_disk-start[1561]: recommended polling time: ^I ( 5) minutes.

Jul 13 08:53:35 vault preclear_disk-start[1561]: SCT capabilities: ^I (0x3035)^ISCT Status supported.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^ISCT Feature Control supported.

Jul 13 08:53:35 vault preclear_disk-start[1561]: ^I^I^I^I^ISCT Data Table supported.

Jul 13 08:53:35 vault preclear_disk-start[1561]:

Jul 13 08:53:35 vault preclear_disk-start[1561]: SMART Attributes Data Structure revision number: 16

Jul 13 08:53:35 vault preclear_disk-start[1561]: Vendor Specific SMART Attributes with Thresholds:

Jul 13 08:53:35 vault preclear_disk-start[1561]: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE

Jul 13 08:53:35 vault preclear_disk-start[1561]: 1 Raw_Read_Error_Rate 0x002f 100 253 051 Pre-fail Always - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 3 Spin_Up_Time 0x0027 172 172 021 Pre-fail Always - 6391

Jul 13 08:53:35 vault preclear_disk-start[1561]: 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 10

Jul 13 08:53:35 vault preclear_disk-start[1561]: 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 8

Jul 13 08:53:35 vault preclear_disk-start[1561]: 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 7

Jul 13 08:53:35 vault preclear_disk-start[1561]: 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 8

Jul 13 08:53:35 vault preclear_disk-start[1561]: 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 199 UDMA_CRC_Error_Count 0x0032 200 253 000 Old_age Always - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]: 200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0

Jul 13 08:53:35 vault preclear_disk-start[1561]:

Jul 13 08:53:35 vault preclear_disk-start[1561]: SMART Error Log Version: 1

Jul 13 08:53:35 vault preclear_disk-start[1561]: No Errors Logged

Jul 13 08:53:35 vault preclear_disk-start[1561]:

Jul 13 08:53:35 vault preclear_disk-start[1561]: SMART Self-test log structure revision number 1

Jul 13 08:53:35 vault preclear_disk-start[1561]: No self-tests have been logged. [To run self-tests, use: smartctl -t]

Jul 13 08:53:35 vault preclear_disk-start[1561]:

Jul 13 08:53:35 vault preclear_disk-start[1561]:

Jul 13 08:53:35 vault preclear_disk-start[1561]: SMART Selective self-test log data structure revision number 1

Jul 13 08:53:35 vault preclear_disk-start[1561]: SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS

Jul 13 08:53:35 vault preclear_disk-start[1561]: 1 0 0 Not_testing

Jul 13 08:53:35 vault preclear_disk-start[1561]: 2 0 0 Not_testing

Jul 13 08:53:35 vault preclear_disk-start[1561]: 3 0 0 Not_testing

Jul 13 08:53:35 vault preclear_disk-start[1561]: 4 0 0 Not_testing

Jul 13 08:53:35 vault preclear_disk-start[1561]: 5 0 0 Not_testing

Jul 13 08:53:35 vault preclear_disk-start[1561]: Selective self-test flags (0x0):

Jul 13 08:53:35 vault preclear_disk-start[1561]: After scanning selected spans, do NOT read-scan remainder of disk.

Jul 13 08:53:35 vault preclear_disk-start[1561]: If Selective self-test is pending on power-up, resume after 0 minute delay.

Jul 13 08:53:35 vault preclear_disk-start[1561]:

Jul 13 09:42:33 vault kernel: mdcmd (30): spindown 1

Jul 13 09:42:33 vault kernel: mdcmd (31): spindown 2

Jul 13 20:46:29 vault dhcpcd[1213]: sending DHCP_REQUEST for 192.168.1.106 to 192.168.1.1

Jul 13 20:46:29 vault dhcpcd[1213]: dhcpIPaddrLeaseTime=86400 in DHCP server response.

Jul 13 20:46:29 vault dhcpcd[1213]: dhcpT1value is missing in DHCP server response. Assuming 43200 sec

Jul 13 20:46:29 vault dhcpcd[1213]: dhcpT2value is missing in DHCP server response. Assuming 75600 sec

Jul 13 20:46:29 vault dhcpcd[1213]: DHCP_ACK received from (192.168.1.1)

Jul 14 00:43:17 vault in.telnetd[5656]: connect from 192.168.1.107 (192.168.1.107)

Jul 14 00:43:21 vault login[5657]: ROOT LOGIN on `pts/1' from `192.168.1.107'

Jul 14 00:43:31 vault kernel: usb 2-1.4: new low speed USB device using uhci_hcd and address 6

Jul 14 00:43:31 vault kernel: usb 2-1.4: configuration #1 chosen from 1 choice

Jul 14 00:43:31 vault kernel: input: Microsoft Microsoft 3-Button Mouse with IntelliEye as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.0/input/input6

Jul 14 00:43:31 vault kernel: generic-usb 0003:045E:0040.0004: input,hidraw1: USB HID v1.10 Mouse [Microsoft Microsoft 3-Button Mouse with IntelliEye] on usb-0000:00:1d.0-1.4/input0

Jul 14 00:43:57 vault kernel: usb 2-1.4: USB disconnect, address 6

Jul 14 00:50:20 vault unmenu[1238]: gawk: ./08-unmenu-array_mgmt.awk:115: warning: escape sequence `\'' treated as plain `''

 

When I went to bed the pre-read stage was over 5 hours elapsed and about 80% complete... If the start time was 8:53 on Jul 13 then I should have some more entries correct?

 

This is a 2TB WD EARS (with jumper installed) it was getting 85-65 MB/s during the pre-read.

[Joe L.]

Sorry, but no.  The only items logged in the syslog are the initial SMART report, and the post-read SMART report, and the "diff" of the two of them.

 

No way to know where you were in the middle.