Unofficial Faster Preclear


Recommended Posts

Where/how do I specify that I want the fast preclear if I am using the preclear plugin (http://lime-technology.com/forum/index.php?topic=39985.0)?

The thread you linked has all your answers, and as threads go around here it is not really that long. ;D

 

If you install the fast preclear script the plugin GUI will give you a check box for fast.

 

*edit* The plugin will not install the script for you. You have to put it in the plugin folder yourself.

Link to comment

*edit* The plugin will not install the script for you. You have to put it in the plugin folder yourself.

Will not install which script? My preclear plugin works, but I don't see the fast option. I installed the fast option by copying preclear_bjp.sh and the correct readvz to the root of my flash drive (according to the readme.txt). Need I do more?

 

Thanks, Richard

Link to comment

*edit* The plugin will not install the script for you. You have to put it in the plugin folder yourself.

Will not install which script? My preclear plugin works, but I don't see the fast option. I installed the fast option by copying preclear_bjp.sh and the correct readvz to the root of my flash drive (according to the readme.txt). Need I do more?

 

Thanks, Richard

Did you read the thread you linked like I suggested? Never mind, I just reread it for you. Here

 

Since you may have additional issues or questions about this please read the thread.

Link to comment
  • 4 weeks later...

Just used faster preclear on my first 8TB drive:

 

root@Tower:/boot/preclear_reports# cat preclear_rpt_Z8408YNX_2015-11-17.txt
========================================================================1.15
== invoked as: /boot/config/plugins/preclear.disk/preclear_disk.sh -M 3 -o 1 -c 1 -f -J /dev/sdb
== ST8000AS0002-1NA17Z   Z8408YNX
== Disk /dev/sdb has been successfully precleared
== with a starting sector of 1
== Ran 1 cycle
==
== Using :Read block size = 1000448 Bytes
== Last Cycle's Pre Read Time  : 19:43:27 (112 MB/s)
== Last Cycle's Zeroing time   : 19:02:46 (116 MB/s)
== Last Cycle's Post Read Time : 23:49:56 (93 MB/s)
== Last Cycle's Total Time     : 62:37:15
==
== Total Elapsed Time 62:37:15

 

During the post read, I noticed it started about 200 MB/s for most of the first half, and then the speed dropped progressively lower as it made it through the last half...

 

 

Link to comment
  • 1 month later...
  • 3 weeks later...

Hey!

Since it seems like it got missed can you take a look at my original question please?  :)

 

Hey!

Does this include the changes made to the "official" preclear script by Joe L.?

 

# Version 1.15  - Added export of LC_CTYPE=C to prevent unicode use on 64 bit printf.

 

Your changelog is not clear about this  :)

 

As an additional request: The script should look for the readvz file in the same folder as the actual script is, for people who dont like their /boot/ to be filled ;)

Link to comment
  • 1 month later...

Note sure if you are still tracking metrics, but recently did preclear on an HGST 5TB:

 

Joe's 1.15 script:

========================================================================1.15
== invoked as: /boot/config/plugins/preclear.disk/preclear_disk.sh -c 1 /dev/sdk
== HGSTHDN726050ALE610   NAG53JLN
== Disk /dev/sdk has been successfully precleared
== with a starting sector of 1 
== Ran 1 cycle
==
== Using :Read block size = 1000448 Bytes
== Last Cycle's Pre Read Time  : 12:19:47 (112 MB/s)
== Last Cycle's Zeroing time   : 9:01:56 (153 MB/s)
== Last Cycle's Post Read Time : 17:28:55 (79 MB/s)
== Last Cycle's Total Time     : 38:51:38
==
== Total Elapsed Time 38:51:38
==
== Disk Start Temperature: 30C
==
== Current Disk Temperature: 35C

 

bjp999 1.15b mod:

========================================================================1.15b
== invoked as: /boot/config/plugins/preclear.disk/preclear_disk.sh -c 1 -f -J /dev/sdk
== HGSTHDN726050ALE610   NAG53JLN
== Disk /dev/sdk has been successfully precleared
== with a starting sector of 1 
== Ran 1 cycle
==
== Using :Read block size = 1000448 Bytes
== Last Cycle's Pre Read Time  : 12:19:34 (112 MB/s)
== Last Cycle's Zeroing time   : 9:02:52 (153 MB/s)
== Last Cycle's Post Read Time : 16:57:56 (81 MB/s)
== Last Cycle's Total Time     : 38:21:23
==
== Total Elapsed Time 38:21:23
==
== Disk Start Temperature: 33C
==
== Current Disk Temperature: 33C

 

x64 version of readvz was in /boot and did chmod +x on it.  Not sure what the '-J' is as I used the preclear plugin and simply swapped scripts out for each run.  These speeds are with the drive negotiating at 3Gb/s vs. 6Gb/s SATA link speed (different slots/controllers made no difference so not sure what going on there).

Link to comment
  • 2 weeks later...

The script does not run on a brand new UNRAID install where no drives were configured. Have to run the "New config" link prior to preclearing drives in such a situation.

 

I don't find this. I recently set up a new install and run the script on 5 old drives on my test server. I did use the Preclear Plugin though.

Link to comment

The script does not run on a brand new UNRAID install where no drives were configured. Have to run the "New config" link prior to preclearing drives in such a situation.

 

I don't find this. I recently set up a new install and run the script on 5 old drives on my test server. I did use the Preclear Plugin though.

 

If you at any point in the past had a drive configured it will run, but on a new installation disk.cfg does not exist and the plugin won’t run without it, easiest way it to hit new config like Jbartlett pointed out.

Link to comment

The script does not run on a brand new UNRAID install where no drives were configured. Have to run the "New config" link prior to preclearing drives in such a situation.

This is a known issue.    This script expects there to be disk.cfg file present, and this is created either when you start your array with at least one disk configured, or when you use the new Config option. 

 

Since its original purpose was to speed up adding disks to an existing parity protected array then this was not an unreasonable assumption.  Since this provides a safety check against wiping the wrong disk by accident it is debatable as to what should happen if the disk.cfg file is not found.  Maybe simply giving a good warning about what this means and what to do is the right thing to do?

Link to comment
  • 2 weeks later...

Is there any limit to the number of preclears that can be run in parallel?

I'm sure I read on another thread that someone had memory issues when running too many (can't find the thread now).

 

I'm currently running 5 on Seagate 8TB drives, but have another 4 and would like to kick those off now rather than waiting for the other drives to finish.

 

The server is a decent spec (2 x E5-2670, 64GB RAM) and the only thing that has moved during the current preclears is the CPU usage, which occasionally reaches 20%. Memory usage sitting at 1%.

 

Is there anything else I should be looking for before starting the other 4?

The current 5 have been running for more than a day and I don't really want to break those.

 

Thanks

 

 

Link to comment

Interesting plan.

The hardware (SATA plugs) should be OK.

 

But is the OS (device driver) really ready for hotplugging?

Can't imagine that linux will do the detecting and initializing stuff.

On the other hand, I'm not really a linux guy.

I'm very interested if this is possible in the meantime.

Link to comment

Thanks. I shall give it a go then.

 

Edit: Seems to be running OK with 9 disks preclearing. The only thing I have noticed is that the CPU now spikes just above 30%.

 

Edit 2: I think I might have been pushing it doing 9 at once, as I think I may have found the limitations of the HBA and expander backplane. When I was doing 5 drives all of them were roughly reading at 100 MB/s, now with 9 they have all dropped to 60-70 MB/s.

Link to comment

Good evening,

 

I'm having an issue running preclear on 2 new 4TB Seagate NAS drives.

 

The drives show when I do a preclear_disk.sh -l.  When I run preclear_disk.sh /dev/sdx (x=drive slot) I get a message that the drive is busy.

 

I thought it might be a power issue so I moved the connectors to different power cables.  It's a 750W supply, which should be good for the 10-11 or so drives I

have. No effect.

 

Using 6.2 beta 18.

 

I've included the output of the preclear below.  Syslog attached.

 

Thanks!

 

 

root@Tower:/boot# preclear_disk.sh -l
====================================1.15
Disks not assigned to the unRAID array
  (potential candidates for clearing)
========================================
     /dev/sdg = ata-ST4000VN000-1H4168_Z305RYB1
     /dev/sdf = ata-ST4000VN000-1H4168_Z305RYKG
root@Tower:/boot# preclear_disk.sh /dev/sdf
sfdisk: invalid option -- 'R'

Usage:
sfdisk [options] <dev> [[-N] <part>]
sfdisk [options] <command>

Display or manipulate a disk partition table.

Commands:
-A, --activate <dev> [<part> ...] list or set bootable MBR partitions
-d, --dump <dev>                  dump partition table (usable for later input)
-J, --json <dev>                  dump partition table in JSON format
-g, --show-geometry [<dev> ...]   list geometry of all or specified devices
-l, --list [<dev> ...]            list partitions of each device
-F, --list-free [<dev> ...]       list unpartitions free areas of each device
-s, --show-size [<dev> ...]       list sizes of all or specified devices
-T, --list-types                  print the recognized types (see -X)
-V, --verify [<dev> ...]          test whether partitions seem correct

--part-label <dev> <part> [<str>] print or change partition label
--part-type <dev> <part> [<type>] print or change partition type
--part-uuid <dev> <part> [<uuid>] print or change partition uuid
--part-attrs <dev> <part> [<str>] print or change partition attributes

<dev>                     device (usually disk) path
<part>                    partition number
<type>                    partition type, GUID for GPT, hex for MBR

Options:
-a, --append              append partitions to existing partition table
-b, --backup              backup partition table sectors (see -O)
     --bytes               print SIZE in bytes rather than in human readable for                                                                                                                     mat
-f, --force               disable all consistency checking
     --color[=<when>]      colorize output (auto, always or never)
                             colors are enabled by default
-N, --partno <num>        specify partition number
-n, --no-act              do everything except write to device
     --no-reread           do not check whether the device is in use
-O, --backup-file <path>  override default backup file name
-o, --output <list>       output columns
-q, --quiet               suppress extra info messages
-X, --label <name>        specify label type (dos, gpt, ...)
-Y, --label-nested <name> specify nested label type (dos, bsd)

-L, --Linux               deprecated, only for backward compatibility
-u, --unit S              deprecated, only sector unit is supported

-h, --help     display this help and exit
-v, --version  output version information and exit

Available columns (for -o):
gpt: Device Start End Sectors Size Type Type-UUID Attrs Name UUID
dos: Device Start End Sectors Cylinders Size Type Id Attrs Boot End-C/H/S
      Start-C/H/S
bsd: Slice Start End Sectors Cylinders Size Type Bsize Cpg Fsize
sgi: Device Start End Sectors Cylinders Size Type Id Attrs
sun: Device Start End Sectors Cylinders Size Type Id Flags

For more details see sfdisk(.
Sorry: Device /dev/sdf is busy.: 1

syslog.txt

Link to comment

I'm having an issue running preclear on 2 new 4TB Seagate NAS drives.

 

The drives show when I do a preclear_disk.sh -l.  When I run preclear_disk.sh /dev/sdx (x=drive slot) I get a message that the drive is busy.

 

I thought it might be a power issue so I moved the connectors to different power cables.  It's a 750W supply, which should be good for the 10-11 or so drives I

have. No effect.

 

Using 6.2 beta 18.

 

root@Tower:/boot# preclear_disk.sh -l
====================================1.15
Disks not assigned to the unRAID array
  (potential candidates for clearing)
========================================
     /dev/sdg = ata-ST4000VN000-1H4168_Z305RYB1
     /dev/sdf = ata-ST4000VN000-1H4168_Z305RYKG
root@Tower:/boot# preclear_disk.sh /dev/sdf
sfdisk: invalid option -- 'R'
...
Sorry: Device /dev/sdf is busy.: 1

You are using the new beta and the included sfdisk command does not support the R option.  I'm not sure there's a Preclear version that's compatible yet, possibly the Preclear beta plugin.  You may need to do your Preclearing back in v6.1.9, for now.

Link to comment

Thanks.  I just browsed the man page for sfdisk.  -R has been depracated in 2.26/

 

"Since version 2.26 sfdisk no longer provides the -R or --re-read option to force the kernel to reread the partition table.  Use blockdev --rereadpt instead"

 

edit:  I modified the two instances of sfdisk -R in the preclear script with blockdev --rereadpt and the preclear is now running.  I'll update as to the status when done in case there are any gotchas.

 

edit 2: I knew it wouldn't be that easy! It starts scanning and throws an error.  I'll wait for a script update.  I don't have spare hardware to run preclear on.

Link to comment

Thanks.  I just browsed the man page for sfdisk.  -R has been depracated in 2.26/

 

"Since version 2.26 sfdisk no longer provides the -R or --re-read option to force the kernel to reread the partition table.  Use blockdev --rereadpt instead"

 

edit:  I modified the two instances of sfdisk -R in the preclear script with blockdev --rereadpt and the preclear is now running.  I'll update as to the status when done in case there are any gotchas.

 

edit 2: I knew it wouldn't be that easy! It starts scanning and throws an error.  I'll wait for a script update.  I don't have spare hardware to run preclear on.

 

You can use the new beta plugin, works with v6.2-beta

 

http://lime-technology.com/forum/index.php?topic=39985.msg453938#msg453938

Link to comment
  • 1 month later...

Could I please ask a stupid question, I am new to unRaid 6 and in 5 when I ran the script when it finished it just kind of ended...

 

Has mine finished or is it still going?

 

 

WDC_WD40EFRX-68WT0N0_WD-WCC4E1LTP3XT

Preclear Successful

... Total time 20:14:20

... Zeroing time 9:13:46 (120 MB/s)

 

Only reason I ask is because under FS in Unassigned devices it is showing as precleared but I am thinking it is still going as I have the red x and an eye next to it, just the gui isn't changing at all and the log (clicking on the eye) looks like when a script would have ended....

 

0 sectors were pending re-allocation before the start of the preclear.

0 sectors were pending re-allocation after zero of disk in cycle 1 of 1.

0 sectors are pending re-allocation at the end of the preclear,

    the number of sectors pending re-allocation did not change.

0 sectors had been re-allocated before the start of the preclear.

0 sectors are re-allocated at the end of the preclear,

    the number of sectors re-allocated did not change.

root@Tower:/usr/local/emhttp#

 

 

***** It had finished, I got brave enough to click the X, now in my array, 4TB precleared in 20hours, not too shabby  :)

Link to comment
  • 2 weeks later...

Hmm, preclear finished fine, but got an error, it couldn't create the preclear results files.

 

cp: cannot create regular file '/boot/preclear_reports/preclear_rpt

 

There is a 0 kb file with no extension named preclear_reports, when I attempt to open it, it says it doesn't exist. Thumbdrive going bad?

Link to comment
  • 3 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.