Jump to content
Biggy2872

Re: preclear_disk.sh - a new utility to burn-in and pre-clear disks for quick add

1381 posts in this topic Last Reply

Recommended Posts

Latest update:  I tried to Pre-Clear the disk once more on my new Server with nothing else attached and it failed.  I then added it to my current array (older Server - Parity not verified) and ran it through the Pre-Clear process again with success! 

 

It has been quite a process w/new drives especially those that have been taken out of External units, but if you are willing to put up with some additional effort it can be rewarding :D!

 

Good Luck to others who might find this a successful and less expensive option especially with Black Friday Deals over the next few weeks!

 

Dave

Share this post


Link to post

Latest update:  I tried to Pre-Clear the disk once more on my new Server with nothing else attached and it failed.  I then added it to my current array (older Server - Parity not verified) and ran it through the Pre-Clear process again with success! 

 

It has been quite a process w/new drives especially those that have been taken out of External units, but if you are willing to put up with some additional effort it can be rewarding :D!

 

Good Luck to others who might find this a successful and less expensive option especially with Black Friday Deals over the next few weeks!

 

Dave

 

You do find the size of the disk in Unraid is now the entire disk.  In other words a 3tb disk is 3tb etc

 

Share this post


Link to post

Hi all new user and got the preclear plugin installed,but now whne i want to preclear my disks it says that the script isn't present how to go from here?

 

Share this post


Link to post

Hi all new user and got the preclear plugin installed,but now whne i want to preclear my disks it says that the script isn't present how to go from here?

 

This plugin is a GUI for the script so the script is installed separately by the user. The OP has complete instructions and a link to one version of the script.

 

Edit: I think you're referring to this plugin. Read the OP in that thread.

Share this post


Link to post

The preclear_disk.sh script is failing on v6.2 betas because the sfdisk -R is no longer supported.  Preclear_disk.sh is reporting the disk as busy and will not preclear it.  It looks like 'blockdev --rereadpt' is the replacement according to the sfdisk man-pages here http://man7.org/linux/man-pages/man8/sfdisk.8.html.

 

"      Since version 2.26 sfdisk no longer provides the -R or --re-read

      option to force the kernel to reread the partition table.  Use

      blockdev --rereadpt instead."

 

EDIT: There is also an issue with reads failing.  I changed the following:

 

read_entire_disk( ) {
  # Get the disk geometry (cylinders, heads, sectors)
  fgeometry=`fdisk -l $1 2>/dev/null`
  units=`echo "$fgeometry" | grep Units | awk '{ print $9 }'`

 

to

 

read_entire_disk( ) {
  # Get the disk geometry (cylinders, heads, sectors)
  fgeometry=`fdisk -l $1 2>/dev/null`
  units=`echo "$fgeometry" | grep Units | awk '{ print $8 }'`

 

and the reads will work.

 

Joe L. - Can we get an official fix and an update from you?

Share this post


Link to post

The preclear_disk.sh script is failing on v6.2 betas because the sfdisk -R is no longer supported.  Preclear_disk.sh is reporting the disk as busy and will not preclear it.  It looks like 'blockdev --rereadpt' is the replacement according to the sfdisk man-pages here http://man7.org/linux/man-pages/man8/sfdisk.8.html.

 

"      Since version 2.26 sfdisk no longer provides the -R or --re-read

      option to force the kernel to reread the partition table.  Use

      blockdev --rereadpt instead."

 

EDIT: There is also an issue with reads failing.  I changed the following:

 

read_entire_disk( ) {
  # Get the disk geometry (cylinders, heads, sectors)
  fgeometry=`fdisk -l $1 2>/dev/null`
  units=`echo "$fgeometry" | grep Units | awk '{ print $9 }'`

 

to

 

read_entire_disk( ) {
  # Get the disk geometry (cylinders, heads, sectors)
  fgeometry=`fdisk -l $1 2>/dev/null`
  units=`echo "$fgeometry" | grep Units | awk '{ print $8 }'`

 

and the reads will work.

 

Joe L. - Can we get an official fix and an update from you?

I only (very) recently put 6.2 beta on my server.

 

I did not have any issue pre-clearing the second parity disk I have just added to my array.

 

The fix will need to wait until I add/replace one of the existing disks with a larger one.

(Otherwise, I have no way to test the process. )

 

Whatever the fix might be, it must be backwards compatible with the older releases of unRAID.

 

In the interim, you can type this command to "patch" the preclear_disk.sh command

First change directory to the directory holding the preclear_disk.sh command.  For most, it will be

cd /boot

then type (or copy from here and paste) the following:

sed -i -e "s/print \$9 /print \$8 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

 

Your preclear disk script will be edited and should work with the two changes you mentioned.  (actually, each occurs in two places, so there are a total of 4 lines changed)

 

Joe L.

Share this post


Link to post

sed -i -e "s/print \$8 /print \$9 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh doesn;t work for me.

 

I keep getting an error on line 1107

 

/preclear_disk.sh: line 1107: 0+( 2245937793)%() : syntax error: operand expected (error token is ") ")

 

it shows 0% complete but lower down it shows that it was completed successfully. It is a 6 Tb WD red drive. What am I doing wrong?

Share this post


Link to post

$8 and $9 are in the wrong order. eg:

sed -i -e "s/print \$9 /print \$8 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

 

sed -i -e "s/print \$8 /print \$9 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh doesn;t work for me.

 

I keep getting an error on line 1107

 

/preclear_disk.sh: line 1107: 0+( 2245937793)%() : syntax error: operand expected (error token is ") ")

 

it shows 0% complete but lower down it shows that it was completed successfully. It is a 6 Tb WD red drive. What am I doing wrong?

Share this post


Link to post

I only (very) recently put 6.2 beta on my server.

 

I did not have any issue pre-clearing the second parity disk I have just added to my array.

 

The fix will need to wait until I add/replace one of the existing disks with a larger one.

(Otherwise, I have no way to test the process. )

 

Whatever the fix might be, it must be backwards compatible with the older releases of unRAID.

 

In the interim, you can type this command to "patch" the preclear_disk.sh command

First change directory to the directory holding the preclear_disk.sh command.  For most, it will be

cd /boot

then type (or copy from here and paste) the following:

sed -i -e "s/print \$8 /print \$9 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

 

Your preclear disk script will be edited and should work with the two changes you mentioned.  (actually, each occurs in two places, so there are a total of 4 lines changed)

 

Joe L.

 

That worked for me, thanks 6.2.0-beta23

Share this post


Link to post

The preclear_disk.sh script is failing on v6.2 betas because the sfdisk -R is no longer supported.  Preclear_disk.sh is reporting the disk as busy and will not preclear it.  It looks like 'blockdev --rereadpt' is the replacement according to the sfdisk man-pages here http://man7.org/linux/man-pages/man8/sfdisk.8.html.

 

"      Since version 2.26 sfdisk no longer provides the -R or --re-read

      option to force the kernel to reread the partition table.  Use

      blockdev --rereadpt instead."

 

EDIT: There is also an issue with reads failing.  I changed the following:

 

read_entire_disk( ) {
  # Get the disk geometry (cylinders, heads, sectors)
  fgeometry=`fdisk -l $1 2>/dev/null`
  units=`echo "$fgeometry" | grep Units | awk '{ print $9 }'`

 

to

 

read_entire_disk( ) {
  # Get the disk geometry (cylinders, heads, sectors)
  fgeometry=`fdisk -l $1 2>/dev/null`
  units=`echo "$fgeometry" | grep Units | awk '{ print $8 }'`

 

and the reads will work.

 

Joe L. - Can we get an official fix and an update from you?

I only (very) recently put 6.2 beta on my server.

 

I did not have any issue pre-clearing the second parity disk I have just added to my array.

 

The fix will need to wait until I add/replace one of the existing disks with a larger one.

(Otherwise, I have no way to test the process. )

 

Whatever the fix might be, it must be backwards compatible with the older releases of unRAID.

 

In the interim, you can type this command to "patch" the preclear_disk.sh command

First change directory to the directory holding the preclear_disk.sh command.  For most, it will be

cd /boot

then type (or copy from here and paste) the following:

sed -i -e "s/print \$8 /print \$9 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

 

Your preclear disk script will be edited and should work with the two changes you mentioned.  (actually, each occurs in two places, so there are a total of 4 lines changed)

 

Joe L.

 

Joe L.    Shouldn't this command be:

 

sed -i -e "s/print \$9 /print \$8 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

 

(It has been some forty years since I last used sed so I might be mistaken....    :o    )

Share this post


Link to post

Joe L.    Shouldn't this command be:

 

sed -i -e "s/print \$9 /print \$8 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

 

(It has been some forty years since I last used sed so I might be mistaken....    :o    )

 

Yes, and I fixed my original post.  (and I've used "sed" for over 40 years)

Share this post


Link to post

Bizarre situation here:

 

PreClear user for 4+ years, 6tb PreClearer for 2+ years, and I've never encountered anything like this.

 

I know dangerously little about non-Windows/Mac commands beyond what I've read on this board in specific Unraid support situations.

 

I've been getting refurbished 6tb drives through Amazon lately because they're dirt cheap and I have plenty of time to run PreClear on each one multiple times before throwing a new one in the array.  So far, this has worked well: 4 drives, 1 bad (immediately), 3 tests each. 7200rpm drives, about 60 hours total (didn't record these successful PreClears).

 

I've read that the pre and post read segments can slow down as the PreClear hits the edges of each disc (or I'm misremembering that, but that there's some part which "naturally" slows).  Just got another refurbished 7200rpm 6tb.  Over the course of a couple of days on each PreClear, I noticed that slowdown (dramatically) towards the end of each post read, as well as towards the end of each pre read. Total (successful, no noticeable red flags to me) drive time per pre-clear: ~110 hours.  All of these 6tb PreClear sessions have been run on a Rosewill eSATA external dock.

 

Something must be wrong with this drive, right?  I ran both instances of PreClear on two separate .screens, which I hope will allow me to pull whatever diagnostic data anyone would need at this point to help me figure out whether something strange is going on with this drive or if the speed drop off is just a quirk of PreClear and my system.

 

Hardware setup:

CASE: Fractal Define R4

MOBO: Supermicro MBD-X10SL7-F-O ATX LGA 1150 Intel C222, DDR3

CPU: Xeon E3-1271v3

RAM: 2x 8GB G.SKILL Ripjaws X ECC

PSU: Corsair RM-650

INTERNAL CAGE: 4-Drive

UnRaid 5.0.5

 

Thanks for any guidance!

Share this post


Link to post

Joe L.    Shouldn't this command be:

 

sed -i -e "s/print \$9 /print \$8 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

 

(It has been some forty years since I last used sed so I might be mistaken....    :o    )

 

Yes, and I fixed my original post.  (and I've used "sed" for over 40 years)

 

Any idea where else on the disk the preclear_disk.sh file could be? I did cd /boot and then typed the sed command, but got a comment "no such file or folder". Any thoughts?

Share this post


Link to post

Joe L.    Shouldn't this command be:

 

sed -i -e "s/print \$9 /print \$8 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

 

(It has been some forty years since I last used sed so I might be mistaken....    :o    )

 

Yes, and I fixed my original post.  (and I've used "sed" for over 40 years)

 

Any idea where else on the disk the preclear_disk.sh file could be? I did cd /boot and then typed the sed command, but got a comment "no such file or folder". Any thoughts?

Could be wrong, but I believe you still have to download and install it onto flash. Lots of changes lately so it's possible this has changed.

Share this post


Link to post

Thanks for your quick reply. I have the preclear plugin and used to be able to use it under 6.1.9. I am now using 6.2.0RC3 and it no longer works. Installed the plugin-beta, but also no success. Does it still require to install something else? If so, how to download and what to do?

Share this post


Link to post

Thanks for your quick reply. I have the preclear plugin and used to be able to use it under 6.1.9. I am now using 6.2.0RC3 and it no longer works. Installed the plugin-beta, but also no success. Does it still require to install something else? If so, how to download and what to do?

I haven't used the beta but iirc it uses a different script. Where did you install preclear_disk.sh? Mine's at the root of the flash drive. Perhaps you need to download and install the file again.

Share this post


Link to post

Joe L.    Shouldn't this command be:

 

sed -i -e "s/print \$9 /print \$8 /" -e "s/sfdisk -R /blockdev --rereadpt /" preclear_disk.sh

 

(It has been some forty years since I last used sed so I might be mistaken....    :o    )

 

Yes, and I fixed my original post.  (and I've used "sed" for over 40 years)

 

Any idea where else on the disk the preclear_disk.sh file could be? I did cd /boot and then typed the sed command, but got a comment "no such file or folder". Any thoughts?

 

Well, mine is in:

 

        /boot/config/plugins/preclear.disk

 

That is the Linux path that you will use with the  cd  command to get you to that directory.  You then can execute the sed command string.  (If I remember correctly, it is silent when the string works.) 

 

While I have installed rc3 on my Test Bed, I have not tried either of the preclear GUI's with it as I don't have a disk to test it with at this time.  I am hearing some noise that there may be come other issues...

 

 

Share this post


Link to post

Thanks. It is somewhat working now, but you are right that there may be some more issues.

 

I am using RC3 and now went back to the "normal" plugin (not beta), copied the script in the respective folder, and did the edit with sed a specified here.

 

I started a preclear with 3 cycles. The first time, it started running, but after 2 days (and around 50% progress), it just showed the disk again with the option to "start preclear". I started the preclear process again and it now shows 97% after 52 hours. But this is still the first cycle, so I anticipated preclearing will be completed in a week. I will keep everyone updated whether the preclear cycles indeed complete.

Share this post


Link to post

I can't get the plugin or the script to work with 6.3.0rc2 - it just says 'disk is busy' when I run manually and gets stuck on 'please wait retrieving information!' with the plugin.

 

 

Help please!

Have you patched the script as mentioned in recent posts in this thread?

Share this post


Link to post

I can't get the plugin or the script to work with 6.3.0rc2 - it just says 'disk is busy' when I run manually and gets stuck on 'please wait retrieving information!' with the plugin.

 

 

Help please!

Have you patched the script as mentioned in recent posts in this thread?

 

 

Ahh that worked.  Sorry, I thought that was a fix for the 6.2 betas as everything worked fine for me (I've only been using unRAID since the last 6.2 RC) so I assumed this would be a new bug.

Share this post


Link to post

I've seen a lot of discussion regarding the patches that must be applied to the script, but honestly I'm still a bit unclear on a couple of things:

 

Is it just JoeL's script that requires the patch?

 

What versions of unRaid require the script to be patched?

Share this post


Link to post

I've seen a lot of discussion regarding the patches that must be applied to the script, but honestly I'm still a bit unclear on a couple of things:

 

Is it just JoeL's script that requires the patch?

 

What versions of unRaid require the script to be patched?

 

I believe JoeL explained the current situation pretty well in this post regarding his script:

 

        https://lime-technology.com/forum/index.php?topic=13054.msg464653#msg464653

 

However, he has not been able to provide proper support for his script for some time because of other commitments on his Time (life!).  Apparently, the need to patch the script developed somewhere in the 6.2 beta releases.  For versions earlier than that, you use the original script.  For later versions, you will have to patch.

 

You may realize that there is also a plugin to do preclear and that plugin can utilize JoeL's script.  And his script has to be patched for the version of unRAID that is installed on the server.  (If you are referring to some issues with the plugin or plugin/script combination, let's not cluttered up this thread with any discussion on those types of issues!)

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.