Preclear plugin


3042 posts in this topic Last Reply

Recommended Posts

When I go to preclear a drive it will pop up the preclear options page. I leave the defaults and click start. It pauses for quite a long time and then goes pack to the page with the list of drives and the drive says "starting" indefinitely. When I click on the preview button I get this:

 

/boot/config/plugins/preclear.disk/preclear_disk.sh  -c 1 /dev/sdh 2>/tmp/preclear.log
root@Tower:/usr/local/emhttp# /boot/config/plugins/preclear.disk/preclear_disk.sh  -c 1 /dev/sdh 2>/tmp/preclear.log
Sorry: Device /dev/sdh is busy.: 1
root@Tower:/usr/local/emhttp#

 

I've tried rebooting several times. No idea what could be keeping it busy.

 

Do you have the 6.2 beta? The line in the preclear script that triggers the "Device X is busy" calls sfdisk -R, but the version of sfdisk in 6.2 doesn't have the -R option.

 

Yes, I do. That must be the problem. I needed dual parity. I looked around for a bit but didn't have time to dig through the whole thread for the cause. The beta version of the plugin works fine though. Are there any major differences between the normal version and beta version scripts? I looked through them and they are obviously different scripts but didn't go through the functions to see what the actual differences were, task wise.

 

Make a simple modification to the preclear_disk.sh script.  Change the 'sfdisk -R $theDisk' to 'blockdev --rereadpt $theDisk' in two places and it should work.  Do this at your own risk as I do not know enough about the preclear script to know if this is the right thing to do.

Link to post
  • Replies 3k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Hi guys, this is a simple plugin that allows users to clear their disks before add them to the array.   The main characteristics of this plugin are: Modularity: can be used standalon

I think Thanks should be directed at all those members that have been dealing with the mess the last few months. @dlandon and @Squid have been working behind the scenes to try and clean up, and o

Dontlookoverthere over on the unRAID subreddit figured out how to get the plugin running again with a simple edit to the plg:   https://www.reddit.com/r/unRAID/comments/7wjpus/preclear_worki

Posted Images

Preclear Beta 2016.03.24a on Unraid 6.2 beta 21

 

basename missing operand error.

 

I removed each plugin one by one and rebooted until this message went away.  So I know it is tied to this plugin.  I did a diagnostics, but when I searched for basename nothing came up.

 

Let me know if more information is needed to track down this issue.

preclear-beta-basename-error.png.999dcf3f71ae7f724d65d901bf484f52.png

Link to post

When I go to preclear a drive it will pop up the preclear options page. I leave the defaults and click start. It pauses for quite a long time and then goes pack to the page with the list of drives and the drive says "starting" indefinitely. When I click on the preview button I get this:

 

/boot/config/plugins/preclear.disk/preclear_disk.sh  -c 1 /dev/sdh 2>/tmp/preclear.log
root@Tower:/usr/local/emhttp# /boot/config/plugins/preclear.disk/preclear_disk.sh  -c 1 /dev/sdh 2>/tmp/preclear.log
Sorry: Device /dev/sdh is busy.: 1
root@Tower:/usr/local/emhttp#

 

I've tried rebooting several times. No idea what could be keeping it busy.

 

Do you have the 6.2 beta? The line in the preclear script that triggers the "Device X is busy" calls sfdisk -R, but the version of sfdisk in 6.2 doesn't have the -R option.

 

Yes, I do. That must be the problem. I needed dual parity. I looked around for a bit but didn't have time to dig through the whole thread for the cause. The beta version of the plugin works fine though. Are there any major differences between the normal version and beta version scripts? I looked through them and they are obviously different scripts but didn't go through the functions to see what the actual differences were, task wise.

 

Make a simple modification to the preclear_disk.sh script.  Change the 'sfdisk -R $theDisk' to 'blockdev --rereadpt $theDisk' in two places and it should work.  Do this at your own risk as I do not know enough about the preclear script to know if this is the right thing to do.

 

This got the write of zeroes working, but the pre-read is failing.  Joe L. needs to work on the script for 6.2.

Link to post

Preclear Beta 2016.03.24a on Unraid 6.2 beta 21

 

basename missing operand error.

 

I removed each plugin one by one and rebooted until this message went away.  So I know it is tied to this plugin.  I did a diagnostics, but when I searched for basename nothing came up.

 

Let me know if more information is needed to track down this issue.

 

It's harmless and will be squashed in the next release.

Link to post

Preclear Beta 2016.03.24a on Unraid 6.2 beta 21

 

basename missing operand error.

 

I removed each plugin one by one and rebooted until this message went away.  So I know it is tied to this plugin.  I did a diagnostics, but when I searched for basename nothing came up.

 

Let me know if more information is needed to track down this issue.

 

It's harmless and will be squashed in the next release.

 

Thanks, did not seem to cause any issues, but wanted to let you know.

Link to post

When I go to preclear a drive it will pop up the preclear options page. I leave the defaults and click start. It pauses for quite a long time and then goes pack to the page with the list of drives and the drive says "starting" indefinitely. When I click on the preview button I get this:

 

/boot/config/plugins/preclear.disk/preclear_disk.sh  -c 1 /dev/sdh 2>/tmp/preclear.log
root@Tower:/usr/local/emhttp# /boot/config/plugins/preclear.disk/preclear_disk.sh  -c 1 /dev/sdh 2>/tmp/preclear.log
Sorry: Device /dev/sdh is busy.: 1
root@Tower:/usr/local/emhttp#

 

I've tried rebooting several times. No idea what could be keeping it busy.

 

Do you have the 6.2 beta? The line in the preclear script that triggers the "Device X is busy" calls sfdisk -R, but the version of sfdisk in 6.2 doesn't have the -R option.

 

Yes, I do. That must be the problem. I needed dual parity. I looked around for a bit but didn't have time to dig through the whole thread for the cause. The beta version of the plugin works fine though. Are there any major differences between the normal version and beta version scripts? I looked through them and they are obviously different scripts but didn't go through the functions to see what the actual differences were, task wise.

 

Make a simple modification to the preclear_disk.sh script.  Change the 'sfdisk -R $theDisk' to 'blockdev --rereadpt $theDisk' in two places and it should work.  Do this at your own risk as I do not know enough about the preclear script to know if this is the right thing to do.

 

I found the issue with reading of the disk.  Change the following:

 

read_entire_disk( ) {
  # Get the disk geometry (cylinders, heads, sectors)
  fgeometry=`fdisk -l $1 2>/dev/null`
  units=`echo "$fgeometry" | grep Units | awk '{ print $9 }'`

 

to

 

read_entire_disk( ) {
  # Get the disk geometry (cylinders, heads, sectors)
  fgeometry=`fdisk -l $1 2>/dev/null`
  units=`echo "$fgeometry" | grep Units | awk '{ print $8 }'`

 

and the pre and post reads will work.

Link to post

An unRAID user friend of mine using unRAID 6.1.9 and the newest version of this plugin reported something to me and I was curious if it was possibly a bug? Being a new unRAID user, he had already tried to create his array without preclearing drives. I advised him to install the script and this plugin, and do the pre-clear. Unbeknownst to me, he had tried to perform a pre-clear on a drive already in an array. After several hours of being stuck at "Starting" from the status screen he checked the log to see the command invoked, and the response from unRAID saying it would not perform the pre-clear as the drive was already in the array. Is there a reason that the plugin itself doesn't try to stop or warn you when trying to pre-clear a currently in the array drive? I definitely appreciate that it stops it from happening as you'd lose all your data, but should there not be a warning letting a user know this is not a preferred method or that the command was not completed instead of it just sticking endlessly at "started" ?

Link to post

I am just running the Plugin for the first time and have experienced something that seems strange : On average I am getting around 50 MB/s for the zeroing step but most of the drives have now moved onto the "Post-read" step and the speed has dropped (for the Post-read drives) to around 1.5 MB/s. Is this normal ?

 

On a separate note, I started all 12 drives at the same time (well as fast as I could click through the pre-clear dialogues) and the drives are identical. During the zeroing I noticed several drives take a lot longer to move to the next step even though the average transfer speed stayed the same as the other drives. Does this indicate a problem with the drives that are lagging or perhaps a problem with the controller ?

 

Thank you

Link to post

Not sure which discussion i need to post this in, Preclear or Unassigned_Devices but feel free to MOVE as appropriate ...

 

Scenario ... i'm getting an unassigned device (via usb enclosure (inateck) showing in the Main screen.

 

I then choose 'Preclear' and get 'Starting' displayed but then it hangs.

 

Looking at the preclear log via the eye icon it is waiting for an input of 'Y'es as it would if you were clearing outside of the array and able to input via keyboard.

. there is an error logged 'smartctl may not be able to run on /dev/sdf with the -d ata option' which it initially gets a response to continue and then hanges on the next similar response needed ...

 

So, how does this ever get satisfied to enable the preclear to run??

 

I have version 1.15 of the preclear script installed, is this correct or not, i suspect not in the circumstances but can't seem to see any other?

 

BTW ... it's version 6.1.9 i'm running.

 

What am i doing wrong?

 

<edit>

 

Here's the full log as it currently stands inactive ...

 

Pre-Clear unRAID Disk /dev/sdf

################################################################## 1.15

smartctl may not be able to run on /dev/sdf with the -d ata option.

however this should not affect the clearing of a disk.

smartctl exit status = 2

smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.1.18-unRAID] (local build)

Copyright © 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

Read Device Identity failed: Invalid argument

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.$

Do you wish to continue?

(Answer Yes to continue. Capital 'Y', lower case 'es'): Yes

Disk /dev/sdf: 8001.6 GB, 8001563222016 bytes

255 heads, 63 sectors/track, 972801 cylinders, total 15628053168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x00000000

  Device Boot      Start        End      Blocks  Id  System

/dev/sdf1              1  4294967295  2147483647+  0  Empty

########################################################################

invoked as  /boot/config/plugins/preclear.disk/preclear_disk.sh -c 1 -W /dev/sdf

########################################################################

(MBR 4k-aligned set. Partition will start on sector 64 for disks <= 2.2TB and sector 1 for disks > 2.2TB)

Are you absolutely sure you want to clear this drive?

(Answer Yes to continue. Capital 'Y', lower case 'es'):

 

Link to post

2 days 6 hours later I have finished pre-clearing 12 drives simultaneously. Turned out the low Post Read was just a temporary slow down, once it got going the average read was 45MB/s.

 

Big question now though, how do I see the results of the Pre-clear ? When I click on the "eye" icon against any of the drives I just get a blank window popup (whereas before I got the log). Thx

Link to post

2 days 6 hours later I have finished pre-clearing 12 drives simultaneously. Turned out the low Post Read was just a temporary slow down, once it got going the average read was 45MB/s.

 

Big question now though, how do I see the results of the Pre-clear ? When I click on the "eye" icon against any of the drives I just get a blank window popup (whereas before I got the log). Thx

 

One thing to check is, if you are using an ad blocker, that you have 'whitelisted' the unRAID server. 

Link to post

It looks like I have lost my http connection to the Server.

 

I will navigate around the console, could someone kindly direct me to where the Results files will be located from the Pre-clear I have just run.

 

Also, are these Results Files written to the USB ? Thx

Link to post

Found the problem with my lost http. Both on board Ethernet ports overheated from the hot air exhaust coming off the server due to running at 95% for over two days to complete the Preclearing !

 

I found the logs in the USB. Tried opening them on a Mac but it looks like they are a script rather than a text log

 

How can I view the results of the Preclearing through the command line on the console ?

 

 

 

 

Sent from my iPhone using Tapatalk

Link to post

Found the problem with my lost http. Both on board Ethernet ports overheated from the hot air exhaust coming off the server due to running at 95% for over two days to complete the Preclearing !

 

I found the logs in the USB. Tried opening them on a Mac but it looks like they are a script rather than a text log

 

How can I view the results of the Preclearing through the command line on the console ?

 

Sent from my iPhone using Tapatalk

 

Try this command:

 

 

grep preclear /var/log/syslog | todos >> /boot/preclear_results.txt

 

It will use the more standard lf/cr format for line endings

Link to post
  • 2 weeks later...

Not really a big deal but the speeds are not correct, write speed should be lower than the average as the disks goes to the inner tracks, they are always similar, it looks to me like the current write speed is the wrong one.

 

############################################################################################################################
#                                                                                                                          #
#                                        unRAID Server Pre-Clear of disk /dev/sdf                                          #
#                                       Cycle 1 of 1, partition start on sector 64.                                        #
#                                                                                                                          #
#                                                                                                                          #
#   Step 1 of 3 - Zeroing in progress:                                                                       (30% Done)    #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#   ** Time elapsed: 3:04:35 | Write speed: 166 MB/s | Average speed: 166 MB/s                                             #
#                                                                                                                          #
############################################################################################################################
#                                   Cycle elapsed time: 3:04:38 | Total elapsed time: 3:04:39                              #
############################################################################################################################

Link to post

Initial speed was 175MB/s, now @ 50%, disk write speed at this point is <150MB/s.

 

############################################################################################################################
#                                                                                                                          #
#                                        unRAID Server Pre-Clear of disk /dev/sdf                                          #
#                                       Cycle 1 of 1, partition start on sector 64.                                        #
#                                                                                                                          #
#                                                                                                                          #
#   Step 1 of 3 - Zeroing in progress:                                                                       (48% Done)    #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#   ** Time elapsed: 5:06:58 | Write speed: 159 MB/s | Average speed: 159 MB/s                                             #
#                                                                                                                          #
############################################################################################################################
#                                 Cycle elapsed time: 5:07:01 | Total elapsed time: 5:07:02                                #
############################################################################################################################

Link to post

Initial speed was 175MB/s, now @ 50%, disk write speed at this point is <150MB/s.

 

############################################################################################################################
#                                                                                                                          #
#                                        unRAID Server Pre-Clear of disk /dev/sdf                                          #
#                                       Cycle 1 of 1, partition start on sector 64.                                        #
#                                                                                                                          #
#                                                                                                                          #
#   Step 1 of 3 - Zeroing in progress:                                                                       (48% Done)    #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#   ** Time elapsed: 5:06:58 | Write speed: 159 MB/s | Average speed: 159 MB/s                                             #
#                                                                                                                          #
############################################################################################################################
#                                 Cycle elapsed time: 5:07:01 | Total elapsed time: 5:07:02                                #
############################################################################################################################

 

Write speed is obtained from dd output, and Average speed is calculated by dividing the written blocks by the elapsed time. Is common that they coincide until the very end of the disk, where the speed drop is more tangible.

Link to post

I believe DD output is also average speed:

 

############################################################################################################################
#                                                                                                                          #
#                                        unRAID Server Pre-Clear of disk /dev/sdf                                          #
#                                       Cycle 1 of 1, partition start on sector 64.                                        #
#                                                                                                                          #
#                                                                                                                          #
#   Step 1 of 3 - Zeroing in progress:                                                                       (58% Done)    #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#   ** Time elapsed: 6:19:25 | Write speed: 155 MB/s | Average speed: 154 MB/s                                             #
#                                                                                                                          #
############################################################################################################################
#                                 Cycle elapsed time: 6:19:29 | Total elapsed time: 6:19:29                                #
############################################################################################################################

 

Current disk speed is <140MB/s

 

pc.png.727f5570cbd9b40e0a83b1be41c9550e.png

Link to post

Hi guys, new unraid user here. I installed the preclear plugin and kicked it off on a brand new 4TB WD red about two hours ago, and the plugin is still showing status as "Starting". When I look at the logs I only see this:

 

/boot/config/plugins/preclear.disk/preclear_disk.sh  -c 3 /dev/sdb 2>/tmp/preclear.log
root@Tower:/usr/local/emhttp# /boot/config/plugins/preclear.disk/preclear_disk.sh  -c 3 /dev/sdb 2>/tmp/preclear.log

 

Should the status have changed to something other than starting by now, or should there be something else in the logs?

Link to post

Hi guys, new unraid user here. I installed the preclear plugin and kicked it off on a brand new 4TB WD red about two hours ago, and the plugin is still showing status as "Starting". When I look at the logs I only see this:

 

/boot/config/plugins/preclear.disk/preclear_disk.sh  -c 3 /dev/sdb 2>/tmp/preclear.log
root@Tower:/usr/local/emhttp# /boot/config/plugins/preclear.disk/preclear_disk.sh  -c 3 /dev/sdb 2>/tmp/preclear.log

 

Should the status have changed to something other than starting by now, or should there be something else in the logs?

Are you running version 6.2 beta? If so make sure you either use the beta preclear plug in or patch the preclear script to support the new version.

Link to post
  • trurl featured, unfeatured and pinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.