Re: preclear_disk.sh - a new utility to burn-in and pre-clear disks for quick add

[Superorb]

So I was curious about something. My server only has 2GB of RAM, would I need to set the block size to something smaller since the stock 2048 would consume 100% of my available memory?

[Gizmotoy]

So I was curious about something. My server only has 2GB of RAM, would I need to set the block size to something smaller since the stock 2048 would consume 100% of my available memory?

 

Maybe I misunderstand your question, but the default block size is 2048 bytes, not megabytes.  Even then, from what I recall you're only writing zeros, so you might not even have to store the whole block in RAM.  You can easily pre_clear drives with 2GB of RAM.  I was doing 3 at a time on my old machine.

[Superorb]

So I was curious about something. My server only has 2GB of RAM, would I need to set the block size to something smaller since the stock 2048 would consume 100% of my available memory?

 

Maybe I misunderstand your question, but the default block size is 2048 bytes, not megabytes.  Even then, from what I recall you're only writing zeros, so you might not even have to store the whole block in RAM.  You can easily pre_clear drives with 2GB of RAM.  I was doing 3 at a time on my old machine.

 

Crap. No, you didn't misunderstand, I did. Bytes =! megabytes Oops.

[electroglyph]

Hell yeah, good call on that one.  Doing 4 now @ 100MB/sec.

 

Six preclears running at the same time

[madburg]

Question, First page states preclear_disk is now at version 1.9 but when I down load it and run "preclear_disk -v" is states it is version 1.6, was this just missed (in the code to display 1.9) or is the link not updated with the new source?

 

Url is pointing to http://lime-technology.com/forum/index.php?action=dlattach;topic=2817.0;attach=6210 just in case this helps.

 

[Joe L.]

Question, First page states preclear_disk is now at version 1.9 but when I down load it and run "preclear_disk -v" is states it is version 1.6, was this just missed or is the link not updated?

 

Url is pointing to http://lime-technology.com/forum/index.php?action=dlattach;topic=2817.0;attach=6210 just in case this helps.

 

Clicking on your link and opening up the zip file shows this in the script itself:

# Version 1.6  - Fixed logic to prevent use on disk assigned to array

# Version 1.7  - Again fixed logic to deal with change in disk.cfg format.

# Version 1.8  - Changes to range of random blocks read to not read past last block on disk.

# Version 1.9  - fixed parse of default partition type from config

#                fixed parse of assigned disks

 

ver="1.9"

 

You must have an older version earlier in your search path and it is being invoked.  (or it is being cached by your browser)

Type

which preclear_disk.sh

to see where the older version is located on your server.  Replace it with the new version you download and un-zip.

 

 

[madburg]

Yeap, so sorry, I am an idiot. Thats what i get for rushing. Thank you as always!

[KYThrill]

I just pre-cleared a a new drive using the -A option.  I thought this was supposed to pre-clear with a starting sector at 64.  However, when I rean pre_clear with the -t option, it reports the drive is precleared with a starting sector of 63.

 

What exactly did I do wrong?  Is -A not the right switch?

[Joe L.]

I just pre-cleared a a new drive using the -A option.  I thought this was supposed to pre-clear with a starting sector at 64.  However, when I rean pre_clear with the -t option, it reports the drive is precleared with a starting sector of 63.

 

What exactly did I do wrong?  Is -A not the right switch?

which version of the preclear script did you use?

 

post the output of

fdisk -lu /dev/sdX

 

(where sdX = your disk)

[Joe L.]

I just pre-cleared a a new drive using the -A option.  I thought this was supposed to pre-clear with a starting sector at 64.  However, when I rean pre_clear with the -t option, it reports the drive is precleared with a starting sector of 63.

 

What exactly did I do wrong?  Is -A not the right switch?

-A = start on sector 64

-a = start on sector 63.

 

Did you use a capital "A" or lower case "a"?

[KYThrill]

I just pre-cleared a a new drive using the -A option.  I thought this was supposed to pre-clear with a starting sector at 64.  However, when I rean pre_clear with the -t option, it reports the drive is precleared with a starting sector of 63.

 

What exactly did I do wrong?  Is -A not the right switch?

which version of the preclear script did you use?

 

post the output of

fdisk -lu /dev/sdX

 

(where sdX = your disk)

 

I used capital -A as instructed in the original post.

 

root@Tower:~# fdisk -lu /dev/sdf

 

Disk /dev/sdf: 2000.3 GB, 2000398934016 bytes

255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors

Units = sectors of 1 * 512 = 512 bytes

Disk identifier: 0x00000000

 

  Device Boot      Start        End      Blocks  Id  System

/dev/sdf1              63  3907029167  1953514552+  0  Empty

Partition 1 does not end on cylinder boundary.

 

Which the -C 64 option did convert it, so no big deal I guess.  But strange that it didn't do it.  I have an EARS drive to do next, so I'll what happens the second time around.

 

PreClear unRAID Disk /dev/sdf

################################################################## 1.9

Device Model:    WDC WD2001FASS-00U0B0

Serial Number:    WD-WMAUR0293305

Firmware Version: 01.00101

User Capacity:    2,000,398,934,016 bytes

########################################################################

Converting existing pre-cleared disk to start partition on sector 64

========================================================================1.9

Step 1. Verifying existing pre-clear signature prior to conversion.  DONE

Step 2. converting existing pre-clear signature:  DONE

========================================================================1.9

==

== Conversion complete.

== DISK /dev/sdf is now PRECLEARED with a starting sector of 64

==

============================================================================

root@Tower:/boot#

 

[Joe L.]

I just pre-cleared a a new drive using the -A option.

 

Where on the command line did you put the "-A" option?

 

It should have been like this:

preclear_disk.sh -A /dev/sdX

and not like this:

preclear_disk.sh /dev/sdX -A

 

It converted easily enough using "-C 64", but I'm curious too.  Please keep me informed of the results on the next drive.

[KYThrill]

I just pre-cleared a a new drive using the -A option.

 

Where on the command line did you put the "-A" option?

 

It should have been like this:

preclear_disk.sh -A /dev/sdX

and not like this:

preclear_disk.sh /dev/sdX -A

 

It converted easily enough using "-C 64", but I'm curious too.  Please keep me informed of the results on the next drive.

 

The switch was in the right place.  The only thing I can think of is if I entered an extra space before or after the switch by accident.  But then I would think that if it ignored the switch, it would have ignored the dev entry too, and the command wouldn't have ran at all.

 

The pre-clear did do one thing weird.  Everything indicated that the pre-clear ran correctly.  However, at the end, it normally goes back to the command prompt.  However, I did this via telnet and it never showed the command prompt at the end of the pre-clear.  So I had to close telnet and reopen.  That is partly why I ran the -t option, to check and make sure the drive had precleared.

[joelones]

I'm not exactly sure why this is happening, possibly a throughput issue. When trying to preclear a wd20ears I noticed that streaming to my xbmc what being impacted big time. In fact to the point where the video would quit on me.

 

Both the drive I'm trying to preclear and movie I was watching are on the same controller, AOC-SASLP-MV8 .

 

Is there some way to limit the throughput or bandwidth during the preclear? Would changing the block size help?

 

thanks

[Joe L.]

I'm not exactly sure why this is happening, possibly a throughput issue. When trying to preclear a wd20ears I noticed that streaming to my xbmc what being impacted big time. In fact to the point where the video would quit on me.

 

Both the drive I'm trying to preclear and movie I was watching are on the same controller, AOC-SASLP-MV8 .

 

Is there some way to limit the throughput or bandwidth during the preclear? Would changing the block size help?

 

thanks

Sorry to hear your disk controller cannot handle the combined load at full speed.  (there is no bandwidth control)  Changing block size/number of blocks being read might help, I have absolutely no way to know.  You can certainly try.

 

Joe L.

[joelones]

I'm not exactly sure why this is happening, possibly a throughput issue. When trying to preclear a wd20ears I noticed that streaming to my xbmc what being impacted big time. In fact to the point where the video would quit on me.

 

Both the drive I'm trying to preclear and movie I was watching are on the same controller, AOC-SASLP-MV8 .

 

Is there some way to limit the throughput or bandwidth during the preclear? Would changing the block size help?

 

thanks

Sorry to hear your disk controller cannot handle the combined load at full speed.  (there is no bandwidth control)   Changing block size/number of blocks being read might help, I have absolutely no way to know.  You can certainly try.

 

Joe L.

 

I find it quite odd that I'm having I/O problems either preclearing or copying data into the array for that matter, while streaming relatively low bitrate clips. Like I said, xbmc just bails and quits in this case. Is this a normal occurrence?

 

I've got a gigabit ethernet setup (allbeit with cat5 cables, ethtool reports 1000Mb/s), an unRAID server with an i3 and 4GB of ram and a Supermicro AOC-SASLP-MV8. The problem seems definitely I/O related.

 

Any suggestions on how I might troubleshoot this further?

[BRiT]

How do you have the SATA controller setup in bios? Did you enable AHCI?

[joelones]

How do you have the SATA controller setup in bios? Did you enable AHCI?

 

Good question BRIT, have yet to poke around in the BIOS since putting together the server. What is the recommended setting? I guess AHCI is the preferable setting and could be changed at any time without affect the integrity of the array/data? Are you referring to the setting in the MV8 bios or motherboard's?

 

EDIT: Attempted to change AHCI in the mobo's bios and came up with some missing disks. Why? In any case, I just reassigned them. So by turning on AHCI on the motherboard directly impacts the disks on the AOC-SASLP-MV8, cause I didn't see a setting in the card's bios for AHCI, or does AHCI only work on the motherboard's SATA ports? In which case, I was having problems with the disks attached to the MV8.

 

http://imageshack.us/photo/my-images/683/screenshot20110507at121.png/

[Superorb]

^^ With my drives in cages and AHCI enabled, I don't see any drives in the BIOS. But, everything works fine in unRAID.

[Superorb]

^^ With my drives in cages and AHCI enabled, I don't see any drives in the BIOS. But, everything works fine in unRAID.

[purko]

How about having an option to write something other than zeros to the disk?

For example, if I want to do a pass of writing an AA(hex) pattern everywhere,

and then the post-read will verify that that is what was written?

 

[squirrellydw]

Is it OK to preclear a drive that will be used in windows just to test it for errors?

[Superorb]

Is it OK to preclear a drive that will be used in windows just to test it for errors?

 

Sure, you'll just have to reformat with a file system that Windows can use when you put it into the Windows machine.

[squirrellydw]

Can I keep the array online while I pre clear, my guess is no

 

Thanks

[BRiT]

Yes. Pre-clear was specifically made so there would be no array downtime. You can preclear drives while your main array is online.