Re: preclear_disk.sh - a new utility to burn-in and pre-clear disks for quick add


Recommended Posts

I am currently preclearing 3 drives at once from the console.  I used control-c to get a new prompt each time. Is there anyway to monitor all 3 preclear sessions from the console, webgui or telnet?  The console currently shows me the last session I started.

 

Sent from my SM-N900V using Xparent BlueTapatalk 2

 

 

Link to comment

I am currently preclearing 3 drives at once from the console.  I used control-c to get a new prompt each time. Is there anyway to monitor all 3 preclear sessions from the console, webgui or telnet?  The console currently shows me the last session I started.

 

Sent from my SM-N900V using Xparent BlueTapatalk 2

 

IF you used a control-c, you killed the process that was running in that shell!  SO you can only run one preclear function per shell.

 

You can run multiple shells, you use alt-F1, Alt-F2, and alt-F3 to get to the first three shells.  (You can run up to six shells.)  Start a preclear session on each disk in a separate shell.  You then switch between the shells with alt-Fx key sequence.

 

Hope this helps...

Link to comment

Edit: Also, anything in version 1.14* that might keep all drives in the array spinning? I don't recall having seen this behavior in the past, but I can't identify anything else going on  'round here that would be keeping every drive busy.

The writing to a drive will use most of your free memory in the disk buffer cache, displacing anything previously cached.  Any other activity on the server will therefore need to spin up the physical disks to read their contents.

 

It could therefore be anything, even a PC on your lan attempting to update its directory listings.  (don't sweat it)

 

Joe L.

 

Ah. I have Cache Dirs running, so every 15 min or so, it's scanning the full directory tree to keep it in the (temporarily) non-existent cache memory. Guess that'll do it. Odd, don't remember having noticed that before, but no big deal.

Link to comment

I am currently preclearing 3 drives at once from the console.  I used control-c to get a new prompt each time. Is there anyway to monitor all 3 preclear sessions from the console, webgui or telnet?  The console currently shows me the last session I started.

 

Sent from my SM-N900V using Xparent BlueTapatalk 2

 

IF you used a control-c, you killed the process that was running in that shell!  SO you can only run one preclear function per shell.

 

You can run multiple shells, you use alt-F1, Alt-F2, and alt-F3 to get to the first three shells.  (You can run up to six shells.)  Start a preclear session on each disk in a separate shell.  You then switch between the shells with alt-Fx key sequence.

 

Hope this helps...

 

Oops. That perfect.  Thank you!!

 

 

 

Sent from my SM-N900V using Xparent BlueTapatalk 2

 

 

Link to comment

Awesome... I have 3 drives up and preclearing through screen now.  Thank you!!

 

Question.  i have a 4th drive in the machine.  but preclear -l does not list it as eligible.  I previously used it as a snap drive I think.  Anyway I want to clear it.  Is there anything I can do to convince preclear to let me clear it?  format it or soemthing?

 

 

Link to comment

Awesome... I have 3 drives up and preclearing through screen now.  Thank you!!

 

Question.  i have a 4th drive in the machine.  but preclear -l does not list it as eligible.  I previously used it as a snap drive I think.  Anyway I want to clear it.  Is there anything I can do to convince preclear to let me clear it?  format it or soemthing?

It must NOT be assigned to the array, and NOT be mounted, and NOT be currently in use by some other program.

 

Formatting it will not do anything to help... if the preclear script thinks it is part of your array, or already mounted, or in use in some way, it will not let you shoot yourself in the foot by writing zeros to it.

Link to comment

I got it Joe.  But this machine has no Array.  It's just being setup now and I am using it to preclear all the drives I have lying around the house.  The 2TB in question was taken from my production unraid machine.  I previously used it as a SNAP drive before I had a PRO license.  So maybe it has some data on it that makes Preclear think its in the array.  But there is no array.  So is there anything I can do that will let me preclear this disk?  It's serial 1211 in the attached pics.

preclear1.PNG.58ad518044f1b576e26641aef7f51f9f.PNG

preclear2.jpg.dd0700c26ac1b1890b46ce83b2bdb899.jpg

Link to comment

Solved.

 

Strange.  Preclear -l did not show my 2TB WD EARS drive as eligible for preclear.  But I knew it should be and I successfully started the preclear.  I just ignored preclear -l

 

I wonder why the drive was not in the -l list.

 

Anyway.. preclearing away!  thanks again Joe.

 

preclear3.jpg.e6adbfae6a384dabdb22027f0bd9ff1d.jpg

Link to comment

It must NOT be assigned to the array, and NOT be mounted, and NOT be currently in use by some other program.

 

Formatting it will not do anything to help... if the preclear script thinks it is part of your array, or already mounted, or in use in some way, it will not let you shoot yourself in the foot by writing zeros to it.

 

I'm running into the same issue as thegizzard. I'm finishing up a parity check after replacing my 3TB parity drive with a new 4TB parity drive. The 3TB is not assigned to the array, but preclear_disk.sh is reporting "no un-assigned disks detected". I know the disk is not in use, I know it's assigned as /dev/sdm after the last reboot, is there anything I should do to attempt to get preclear to recognize it as not in the array, or should I just fire off the preclear at my own risk?

 

TIA

FreeMan

Unassigned_disk_issue.PNG.134e2b95e38240ce41def03c149e223a.PNG

Link to comment

I'm going to go ahead and run the preclear, but I get this:

Disk /dev/sdm: 3000.6 GB, 3000592982016 bytes
256 heads, 63 sectors/track, 363376 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdm1               1  4294967295  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.
########################################################################
invoked as  ./preclear_disk.sh -n /dev/sdm
########################################################################
(MBR 4k-aligned set. Partition will start on sector 64 for disks <= 2.2TB and sector 1 for disks > 2.2TB)

 

Note that this was previously a parity disk. Is Partition 1 does not start on physical sector boundary anything to worry about considering (MBR 4k-aligned set. Partition will start on sector 64 for disks <= 2.2TB and sector 1 for disks > 2.2TB)?

Link to comment

I'm going to go ahead and run the preclear, but I get this:

Disk /dev/sdm: 3000.6 GB, 3000592982016 bytes
256 heads, 63 sectors/track, 363376 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdm1               1  4294967295  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.
########################################################################
invoked as  ./preclear_disk.sh -n /dev/sdm
########################################################################
(MBR 4k-aligned set. Partition will start on sector 64 for disks <= 2.2TB and sector 1 for disks > 2.2TB)

 

Note that this was previously a parity disk. Is Partition 1 does not start on physical sector boundary anything to worry about considering (MBR 4k-aligned set. Partition will start on sector 64 for disks <= 2.2TB and sector 1 for disks > 2.2TB)?

nothing to worry about.
Link to comment
  • 1 month later...

Joe,

 

I am also having an issue where a a drive that was formerly part of the array is not shown by "preclear_disk.sh -l".  When I execute "strings /boot/config/super.dat | grep 2604" (the disk's serial number ends in 2604), I get "A5812604", which is the end of the disk's serial number.  Further commands make the preclear script think the disk is in the array.

 

The preclear script will clear the drive as other mention above.

 

I am using preclear v1.15 of and unraid v5.0.5.

Link to comment

Joe,

 

I am also having an issue where a a drive that was formerly part of the array is not shown by "preclear_disk.sh -l".  When I execute "strings /boot/config/super.dat | grep 2604" (the disk's serial number ends in 2604), I get "A5812604", which is the end of the disk's serial number.  Further commands make the preclear script think the disk is in the array.

 

The preclear script will clear the drive as other mention above.

 

I am using preclear v1.15 of and unraid v5.0.5.

I have no way to help you there.  Since unRAID has no definitive list of assigned disks, the -l option of the preclear script looks in super.dat too.  You assigned it, so it is there, and apparently the value has not been overwritten by another disk's serial number.

 

If you are sure of the disk, it will probably let you pre-clear it.    It is just being VERY cautious.

Link to comment

 

You assigned it, so it is there, and apparently the value has not been overwritten by another disk's serial number.

 

 

At least we know why it is happening.  I'm also not sure how to handle this without the possibility of listing a disk that is in the array.

On older arrays, lime-tech used disk.cfg.  It was easy at that point.

 

Newer versions of unRAID no longer used disk.cfg in the same way.  It could not be relied upon to determine assigned disks.

 

With the newer cache-pool, etc, I've no idea what is the definitive list of assigned devices.  Best to be overly cautious.

 

At some point, I expect a native preclear to exist.  At that point, it will be Tom's job to determine which disks are assigned, and which are not.

Joe L.

Link to comment
  • 1 month later...

Quick question about preclearing a drive which will be used to replace an existing smaller drive (basically an upgrade from 350GB to 4TB).

If I preclear the 4TB drive, will the rebuild phase be quicker compared to rebuilding immediately with no pre-clearing phase?

Pre-clearing does not affect rebuild time.    What it does is provide a confidence check that the disk is in a good condition before you commit your data to it.

 

The one time when pre-clearing DOES save time is when you are adding a new disk to an existing parity protected array.  In such a scenario it avoids the array being offline for a significant length of time while unRAID does the equivalent of the preclear script..

Link to comment

Pre-clearing does not affect rebuild time.    What it does is provide a confidence check that the disk is in a good condition before you commit your data to it.

 

The one time when pre-clearing DOES save time is when you are adding a new disk to an existing parity protected array.  In such a scenario it avoids the array being offline for a significant length of time while unRAID does the equivalent of the preclear script..

 

Thanks for the confirmation. I am using the latest version of Unraid, v5.0.5. Do you still have to download the smartctl missing library separately?

Link to comment
  • 5 weeks later...

I had my preclear interrupted (power outage) during the post read stage. I verified it does have the preclear signature, and then initiated just a post read session. Is this "okay"? I would just do the preclear over again but it's a 6 TB drive, takes a while, I have 3 more to preclear and I have some hardware changes to make this weekend. Got some new drive cages + cables to max out my SASLP so I can preclear more than one drive at a time, only had one slot left, and less than 500 GB of 12 TB left. 

Link to comment

I had my preclear interrupted (power outage) during the post read stage. I verified it does have the preclear signature, and then initiated just a post read session. Is this "okay"? I would just do the preclear over again but it's a 6 TB drive, takes a while, I have 3 more to preclear and I have some hardware changes to make this weekend. Got some new drive cages + cables to max out my SASLP so I can preclear more than one drive at a time, only had one slot left, and less than 500 GB of 12 TB left.

 

Since it's already got the appropriate signature, simply redoing the post read is fine -- it will complete the final check that it was in the process of doing anyway.  What I suspect you won't see is the complete report that shows any changes in the SMART data during the process (not sure what might have already been saved by the pre-clear utility) ... but as long as the SMART data all looks good, you should be good to go.

 

Link to comment

II had my preclear interrupted (power outage) during the post read stage. I verified it does have the preclear signature, and then initiated just a post read session. Is this "okay"? I would just do the preclear over again but it's a 6 TB drive, takes a while,

 

Most of my pending sector issues have come during unclean shutdowns. Since this is a 'new' drive and you've verified the signature, I would also do a SMART long test on the drive.  The firmware will read every sector and mark it in the self test log.  Verify the smart log for pending sectors.

 

The smart long test and post read are similar however the smart long test is done by the firmware and you will have a line in the sand marked internally within the hard drive.  I (personally) would not skip this step even though it will take a long time. It's much harder to recover from bad sectors after data is on the drive.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.