Jump to content

Preclear.sh results - Questions about your results? Post them here.


Recommended Posts

Thank you.

 

I downloaded the newest one from the preclear thread but as it was running I thought something was off since it did not show the status in unmenu.  Turns out I had the old one too and when I ran it that's the one that ran.  Running it again with 1.1 now, hopefully with better results.

Link to comment

Hey Joe L., are you working on a new release of preclear_disk for 3TB drives? If so and you need a beta tester, I just received one today.

 

Yes, I have just finished one... I'll attach it to this post temporarily.  (Until it gets tested by a few brave users)

 

It has NEVER been tested on an actual 3TB drive... It should work.  The GPT signature it creates is identical to the one Tom at lime-technology described to me.

 

You'll have to give it a try and let me know how it goes.

 

It can not convert GPT partitions using the "-C" option, as they are handled entirely differently and never start on partition 63 or 64. 

Also, if your disk is over 2.2TB in size, it will set the preclear signature as it needs, completely ignoring any "-a" or "-A" option you might try to give it.

 

Joe L.

preclear_disk1.12beta.sh.zip

Link to comment

Is there a pretend mode or quick mode I can use? What I mean by that is something like have it read just a bit maybe write just a bit and then have it stamp (gpt, signature, etc). This way I can give u back if unRaid said it as a formatted drive already, etc... Knowing it will be pre-cleared fully at a later time, should the quick test work out.

Link to comment

Is there a pretend mode or quick mode I can use? What I mean by that is something like have it read just a bit maybe write just a bit and then have it stamp (gpt, signature, etc). This way I can give u back if unRaid said it as a formatted drive already, etc... Knowing it will be pre-cleared fully at a later time, should the quick test work out.

nope, no mode to fake a pre-clear signature.  That is by design.  Too likely to be mis-used and destroy somebody's parity integrity. 

Formatted does not mean pre-cleared, in fact, it is EXACTLY the opposite.  A formatted drive has tons of data on it already from the formatting process.

 

Only shortcut is to use the "-n" option to skip the pre and post read phases.  That will zero the drive and write the pre-clear signature, but it WILL NOT identify bad sectors that can cripple parity re-construction of a failed drive later, when you learn they are un-readable.  Use of the "-n" option will result in the drive being zeroed at about a 100MB/s rate on most modern hardware.  10 seconds for 1Gig, 3000Gig = 30000 seconds = roughly 8.33 hours.

 

When done, I have no idea if unRAID will tell you anything when you add the drive to an existing array to let you know it is pre-cleared (It would be nice if it did though).  It will either add the drive quickly, or it will proceed to clear the drive on its own, and take your array off-line for the same 8.33 hours while it does.

 

The "-t" option will tell you if the pre-clear script thinks the drive is pre-cleared.  I'd use it first, and if it says the drive is pre-cleared, you are probably ok.

 

Joe L.

Link to comment

Thanks, I guess what I meant to be more clear (just in case), I saw this: "short_test=0        # set to non-zero for short test for script testing - Leave 0 for normal test"

I was under the impresstion preclear_disk would at the end write something to the disk which is the same as what unRAID does at the end of a format. So that is why when we add a precleared drive in as a data drive it ready to go quickly. So what I was wondering is if there is a way to skip the pre-read/write/post read and just have the script do that last thing you do to the drive to make unRAID believe it formatted it, i guess. In this case we are testing the 3TB GPT sig you are applying (I think). I just wanted to get that done fast and results over to you. Then if theoutcome is what you expected. Go back, remove the drive from the array and run a true FULL preclear on it. Hope this make more sense.

Link to comment

Thanks, I guess what I meant to be more clear (just in case), I saw this: "short_test=0         # set to non-zero for short test for script testing - Leave 0 for normal test"

I was under the impresstion preclear_disk would at the end write something to the disk which is the same as what unRAID does at the end of a format. So that is why when we add a precleared drive in as a data drive it ready to go quickly. So what I was wondering is if there is a way to skip the pre-read/write/post read and just have the script do that last thing you do to the drive to make unRAID believe it formatted it, i guess. In this case we are testing the 3TB GPT sig you are applying (I think). I just wanted to get that done fast and results over to you. Then if theoutcome is what you expected. Go back, remove the drive from the array and run a true FULL preclear on it. Hope this make more sense.

Yes, there is a option to run a short-test.  However, it purposely does NOT write a valid pre-clear signature.  (again, to prevent somebody from attempting to pre-clear using it thinking it is doing what is needed to protect their parity data)

 

I'll send you a PM describing how to invoke it if you like...

 

After invoking it, and before assigning the drive to the unRAID array, you can post the results of this command to assist me in knowing if it worked as expected on the 3 TB drive:

dd if=/dev/sda count=1 2>/dev/null | od -x -A x

(substituting your disk for /dev/sda )

 

Link to comment

Hey Joe L., are you working on a new release of preclear_disk for 3TB drives? If so and you need a beta tester, I just received one today.

 

Yes, I have just finished one... I'll attach it to this post temporarily.  (Until it gets tested by a few brave users)

 

It has NEVER been tested on an actual 3TB drive... It should work.  The GPT signature it creates is identical to the one Tom at lime-technology described to me.

 

You'll have to give it a try and let me know how it goes.

 

It can not convert GPT partitions using the "-C" option, as they are handled entirely differently and never start on partition 63 or 64.  

Also, if your disk is over 2.2TB in size, it will set the preclear signature as it needs, completely ignoring any "-a" or "-A" option you might try to give it.

 

Joe L.

 

Running now, with the "-W" option (disk was precleared previously and has been running in 2.2T mode for a while).  Zeroing at 140 MB/sec :)

 

The initial screen is still saying stuff about partition alignment / offsets, etc.  It should probably say it is a >2TB disk and being prepared for a GPT structure.  My disk already had an MBR from the 2.2T mode (all I did was remove the HPA, reboot, and start this preclear).  So this should be a pretty good test of preclear's ability to preclear a 3T drive.

 

Link to comment

Hey Joe L., are you working on a new release of preclear_disk for 3TB drives? If so and you need a beta tester, I just received one today.

 

Yes, I have just finished one... I'll attach it to this post temporarily.  (Until it gets tested by a few brave users)

 

It has NEVER been tested on an actual 3TB drive... It should work.  The GPT signature it creates is identical to the one Tom at lime-technology described to me.

 

You'll have to give it a try and let me know how it goes.

 

It can not convert GPT partitions using the "-C" option, as they are handled entirely differently and never start on partition 63 or 64.  

Also, if your disk is over 2.2TB in size, it will set the preclear signature as it needs, completely ignoring any "-a" or "-A" option you might try to give it.

 

Joe L.

 

Running now, with the "-W" option (disk was precleared previously and has been running in 2.2T mode for a while).  Zeroing at 140 MB/sec :)

 

The initial screen is still saying stuff about partition alignment / offsets, etc.  It should probably say it is a >2TB disk and being prepared for a GPT structure.  My disk already had an MBR from the 2.2T mode (all I did was remove the HPA, reboot, and start this preclear).  So this should be a pretty good test of preclear's ability to preclear a 3T drive.

 

Oops... forgot to change the initial screen and what it says when a GPT preclear will occur.

It is actually a bit tricky, since the MBR partition would STILL be used by unRAID if the disk advertised its size in 4K blocks as opposed to advertising its size in 512 byte blocks.   Only if the size is >= 0xFFFFFFFF blocks(sectors) is a GPT partition used by unRAID.   The math work on the number of blocks, not the disk size in bytes.

 

I'll be curious to see how it does.  What kind of "write" speed is it showing?  (If I would only read I'd know.  Will be curious to see speed on inner cylinders)

 

Joe L.

Link to comment

I'll be curious to see how it does.  What kind of "write" speed is it showing?  (If I would only read I'd know.  Will be curious to see speed on inner cylinders)

 

Running at 143 MB/sec now @ 7% complete.

 

Not sure I'll be home when it gets to the inner cyl, but will have to see.

 

BTW, this is a 7200 RPM Hitachi drive.

Link to comment

I'll be curious to see how it does.  What kind of "write" speed is it showing?  (If I would only read I'd know.  Will be curious to see speed on inner cylinders)

 

Running at 143 MB/sec now @ 7% complete.

 

Not sure I'll be home when it gets to the inner cyl, but will have to see.

 

BTW, this is a 7200 RPM Hitachi drive.

If it can keep up that speed it will be 6 or 7 hours for the "Write" phase.  Figures though, since the rotational speed of the disk is the same, and the density on the platters higher, the write speed should also be higher.  I expect you'll be down to 100MB/s or less on inner cylinders... still very nice.
Link to comment

I'll be curious to see how it does.  What kind of "write" speed is it showing?  (If I would only read I'd know.  Will be curious to see speed on inner cylinders)

 

Running at 143 MB/sec now @ 7% complete.

 

Not sure I'll be home when it gets to the inner cyl, but will have to see.

 

BTW, this is a 7200 RPM Hitachi drive.

 

Write phase finished in about 7 1/2 hours.  I wasn't watching at the end, so don't know speed on inner tracks.

 

Working on post read now.

Link to comment

I realized since I was in the post read step, that the disk was already precleared, so decided to see if "gdisk" recognized it as a valid GPT disk.

 

See below.  Is this what I should be seeing?

 

I am hoping that running this command didn't mess up the preclear signature.  I did run the command a second time and got the exact same results, so am hopeful that the "converting MBR to GPT format" did not write anything back to the disk.  Does the "-t" option of preclear work for large disks in this beta version?

 

>gdisk -l /dev/sdd

GPT fdisk (gdisk) version 0.6.14

 

Partition table scan:

  MBR: MBR only

  BSD: not present

  APM: not present

  GPT: not present

 

 

***************************************************************

Found invalid GPT and valid MBR; converting MBR to GPT format.

***************************************************************

 

Disk /dev/sdd: 5860533168 sectors, 2.7 TiB

Logical sector size: 512 bytes

Disk identifier (GUID): A624A0B1-9F34-4185-8DEE-E6985C5EED53

Partition table holds up to 128 entries

First usable sector is 34, last usable sector is 5860533134

Partitions will be aligned on 2048-sector boundaries

Total free space is 5860533101 sectors (2.7 TiB)

 

Number  Start (sector)    End (sector)  Size      Code  Name

 

Link to comment

I realized since I was in the post read step, that the disk was already precleared, so decided to see if "gdisk" recognized it as a valid GPT disk.

 

See below.  Is this what I should be seeing?

 

I am hoping that running this command didn't mess up the preclear signature.  I did run the command a second time and got the exact same results, so am hopeful that the "converting MBR to GPT format" did not write anything back to the disk.  Does the "-t" option of preclear work for large disks in this beta version?

 

>gdisk -l /dev/sdd

GPT fdisk (gdisk) version 0.6.14

 

Partition table scan:

  MBR: MBR only

  BSD: not present

  APM: not present

  GPT: not present

 

 

***************************************************************

Found invalid GPT and valid MBR; converting MBR to GPT format.

***************************************************************

 

Disk /dev/sdd: 5860533168 sectors, 2.7 TiB

Logical sector size: 512 bytes

Disk identifier (GUID): A624A0B1-9F34-4185-8DEE-E6985C5EED53

Partition table holds up to 128 entries

First usable sector is 34, last usable sector is 5860533134

Partitions will be aligned on 2048-sector boundaries

Total free space is 5860533101 sectors (2.7 TiB)

 

Number  Start (sector)    End (sector)  Size       Code  Name

 

Yes, the -t should work on large disks.  (I'm guessing it is no longer a valid pre-clear signature if it "converted" it)

 

Link to comment

Yes, the -t should work on large disks.   (I'm guessing it is no longer a valid pre-clear signature if it "converted" it)

 

I ran preclear in a new window with the -t option (didn't think it would hurt the running post read).

 

Looks like everything is okay.  Guess won't know for sure until I try to add the disk to the array.

 

################################################################## 1.12

Device Model:    Hitachi HDS723030ALA640

Serial Number:    MK0311YHG2TWKA

Firmware Version: MKAOA3B0

User Capacity:    3,000,592,982,016 bytes

 

Disk /dev/sdd: 3000.6 GB, 3000592982016 bytes

255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x00000000

 

  Device Boot      Start        End      Blocks  Id  System

/dev/sdd1              1  4294967295  2147483647+  0  Empty

Partition 1 does not end on cylinder boundary.

########################################################################

========================================================================1.12

==

== DISK /dev/sdd IS PRECLEARED with a GPT Protective MBR

==

============================================================================

 

Link to comment

The preclear completed successfully, and the "-t" showed it as precleared.

 

But unfortunately it didn't work.  unRAID is now clearing the disk itself.

 

Not sure if a preclear problem or an unRAID problem.

 

Here is what I did:

 

- Stopped the array

- Added disk to next slot (disk12) (showed with blue ball)

- Clicked the checkbox to enable the start button

- Clicked start button

- unRAID did something for a few seconds

- unRAID screen came back, disk12 still showed with blue ball, array not started

- So again I clicked the checkbox to enable the start button

- And again clicked the start button

- unRAID began clearing the disk

 

Here is the relevant portion of the syslog:

 

Jun 18 09:40:46 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:40:47 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:41:25 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:41:25 Shark kernel: md: import disk12: [8,48] (sdd) Hitachi_HDS723030ALA640_MK0311YHG2TWKA size: 2930266532 (Drive related)

Jun 18 09:41:25 Shark kernel: md: disk12 new disk (unRAID engine)

Jun 18 09:41:49 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:41:49 Shark kernel: md: import disk12: [8,48] (sdd) Hitachi_HDS723030ALA640_MK0311YHG2TWKA size: 2930266532 (Drive related)

Jun 18 09:41:49 Shark kernel: md: disk12 new disk (unRAID engine)

Jun 18 09:41:50 Shark emhttp: shcmd (163): /usr/local/sbin/set_ncq sdd 1 >/dev/null (Drive related)

Jun 18 09:41:50 Shark emhttp: writing GPT on disk 12 (sdd) with partition 1 offset 64 (Drive related)

Jun 18 09:41:50 Shark emhttp: shcmd (165): sgdisk -o -a 64 -n 1:64:0 /dev/sdd 2>$stuff$1 |logger (Drive related)

Jun 18 09:41:50 Shark emhttp: re-reading (sdd) partition table (Drive related)

Jun 18 09:41:50 Shark kernel: sdd: unknown partition table (Drive related)

Jun 18 09:41:52 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:41:52 Shark kernel: md: import disk12: [8,48] (sdd) Hitachi_HDS723030ALA640_MK0311YHG2TWKA size: 2930266532 (Drive related)

Jun 18 09:41:52 Shark kernel: md: disk12 new disk (unRAID engine)

Jun 18 09:42:23 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:42:23 Shark kernel: md: import disk12: [8,48] (sdd) Hitachi_HDS723030ALA640_MK0311YHG2TWKA size: 2930266532 (Drive related)

Jun 18 09:42:23 Shark kernel: md: disk12 new disk (unRAID engine)

Jun 18 09:42:24 Shark emhttp: shcmd (194): /usr/local/sbin/set_ncq sdd 1 >/dev/null (Drive related)

Jun 18 09:42:24 Shark emhttp: writing GPT on disk 12 (sdd) with partition 1 offset 64 (Drive related)

Jun 18 09:42:24 Shark emhttp: shcmd (196): sgdisk -o -a 64 -n 1:64:0 /dev/sdd 2>$stuff$1 |logger (Drive related)

Jun 18 09:42:26 Shark kernel: sdd: sdd1 (Drive related)

Jun 18 09:42:26 Shark emhttp: re-reading (sdd) partition table (Drive related)

Jun 18 09:42:26 Shark kernel: sdd: sdd1 (Drive related)

Jun 18 09:42:27 Shark emhttp: clearing disk12... (Other emhttp)

 

Link to comment

The preclear completed successfully, and the "-t" showed it as precleared.

 

But unfortunately it didn't work.  unRAID is now clearing the disk itself.

 

Not sure if a preclear problem or an unRAID problem.

 

Here is what I did:

 

- Stopped the array

- Added disk to next slot (disk12) (showed with blue ball)

- Clicked the checkbox to enable the start button

- Clicked start button

- unRAID did something for a few seconds

- unRAID screen came back, disk12 still showed with blue ball, array not started

- So again I clicked the checkbox to enable the start button

- And again clicked the start button

- unRAID began clearing the disk

 

Here is the relevant portion of the syslog:

 

Jun 18 09:40:46 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:40:47 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:41:25 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:41:25 Shark kernel: md: import disk12: [8,48] (sdd) Hitachi_HDS723030ALA640_MK0311YHG2TWKA size: 2930266532 (Drive related)

Jun 18 09:41:25 Shark kernel: md: disk12 new disk (unRAID engine)

Jun 18 09:41:49 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:41:49 Shark kernel: md: import disk12: [8,48] (sdd) Hitachi_HDS723030ALA640_MK0311YHG2TWKA size: 2930266532 (Drive related)

Jun 18 09:41:49 Shark kernel: md: disk12 new disk (unRAID engine)

Jun 18 09:41:50 Shark emhttp: shcmd (163): /usr/local/sbin/set_ncq sdd 1 >/dev/null (Drive related)

Jun 18 09:41:50 Shark emhttp: writing GPT on disk 12 (sdd) with partition 1 offset 64 (Drive related)

Jun 18 09:41:50 Shark emhttp: shcmd (165): sgdisk -o -a 64 -n 1:64:0 /dev/sdd 2>$stuff$1 |logger (Drive related)

Jun 18 09:41:50 Shark emhttp: re-reading (sdd) partition table (Drive related)

Jun 18 09:41:50 Shark kernel: sdd: unknown partition table (Drive related)

Jun 18 09:41:52 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:41:52 Shark kernel: md: import disk12: [8,48] (sdd) Hitachi_HDS723030ALA640_MK0311YHG2TWKA size: 2930266532 (Drive related)

Jun 18 09:41:52 Shark kernel: md: disk12 new disk (unRAID engine)

Jun 18 09:42:23 Shark emhttp: Hitachi_HDS723030ALA640_MK0311YHG2TWKA (sdd) 2930266584 (Drive related)

Jun 18 09:42:23 Shark kernel: md: import disk12: [8,48] (sdd) Hitachi_HDS723030ALA640_MK0311YHG2TWKA size: 2930266532 (Drive related)

Jun 18 09:42:23 Shark kernel: md: disk12 new disk (unRAID engine)

Jun 18 09:42:24 Shark emhttp: shcmd (194): /usr/local/sbin/set_ncq sdd 1 >/dev/null (Drive related)

Jun 18 09:42:24 Shark emhttp: writing GPT on disk 12 (sdd) with partition 1 offset 64 (Drive related)

Jun 18 09:42:24 Shark emhttp: shcmd (196): sgdisk -o -a 64 -n 1:64:0 /dev/sdd 2>$stuff$1 |logger (Drive related)

Jun 18 09:42:26 Shark kernel: sdd: sdd1 (Drive related)

Jun 18 09:42:26 Shark emhttp: re-reading (sdd) partition table (Drive related)

Jun 18 09:42:26 Shark kernel: sdd: sdd1 (Drive related)

Jun 18 09:42:27 Shark emhttp: clearing disk12... (Other emhttp)

 

I cannot tell either... as I have no 3TB drive to test with.

 

Too bad you did not run this command before assigning the disk to the array:

After invoking it, and before assigning the drive to the unRAID array, you can post the results of this command to assist me in knowing if it worked as expected on the 3 TB drive:

dd if=/dev/sda count=1 2>/dev/null | od -x -A x

(substituting your disk for /dev/sda )

 

It should have returned

0000000 0000 0000 0000 0000 0000 0000 0000 0000

*

00001c0 0002 ff00 ffff 0001 0000 ffff ffff 0000

00001d0 0000 0000 0000 0000 0000 0000 0000 0000

*

00001f0 0000 0000 0000 0000 0000 0000 0000 aa55

Link to comment

Too bad you did not run this command before assigning the disk to the array:

After invoking it, and before assigning the drive to the unRAID array, you can post the results of this command to assist me in knowing if it worked as expected on the 3 TB drive:

dd if=/dev/sda count=1 2>/dev/null | od -x -A x

(substituting your disk for /dev/sda )

 

It should have returned

0000000 0000 0000 0000 0000 0000 0000 0000 0000

*

00001c0 0002 ff00 ffff 0001 0000 ffff ffff 0000

00001d0 0000 0000 0000 0000 0000 0000 0000 0000

*

00001f0 0000 0000 0000 0000 0000 0000 0000 aa55

 

NOW he tells me ;).

 

I am going to wait for the clear to finish, but I do have another 3T disk I can experiment with later.

Link to comment

I was thinking the "-t" would verify the signature.

 

Will run an abbreviated test tonight and give you the results of the command.

 

I suspect unRaid is having a problem with the add vs a preclear signature problem, but we'll see.

Link to comment

I was thinking the "-t" would verify the signature.

 

Will run an abbreviated test tonight and give you the results of the command.

 

I suspect unRaid is having a problem with the add vs a preclear signature problem, but we'll see.

You might be right... and I could only test the -t on a simulated disk here, so it too could be an issue.. but not as likely.
Link to comment

I am running the 12 beta on a brand new 3TB Hitachi green HD.

 

There was a 6 to 11 MB/s difference in the speed reported from the console (higher) than the myMain at one point. This difference has grown now to 40 MB/s during the post-read.

myMain speed is given as 51 MB/s while the console is showing around 91 MB/s ( at the 58% done post-read -around 28h:30min mark)

 

Will report later once it is done.

Link to comment

I am running the 12 beta on a brand new 3TB Hitachi green HD.

 

There was a 6 to 11 MB/s difference in the speed reported from the console (higher) than the myMain at one point. This difference has grown now to 40 MB/s during the post-read.

myMain speed is given as 51 MB/s while the console is showing around 91 MB/s ( at the 58% done post-read -around 28h:30min mark)

 

Will report later once it is done.

After it is done, and before assigning the drive to the unRAID array, you can post the results of this command to assist me in knowing if it worked as expected on your 3 TB drive:

dd if=/dev/sda count=1 2>/dev/null | od -x -A x

(substituting your disk for /dev/sda )

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...