Preclear plugin


3041 posts in this topic Last Reply

Recommended Posts

Just finished preclearing a 3TB WD Green, this disk was previously used on another server so skipped pre and post reads, but failed verifying signature:

 

Good to know it's not only me  :)

 

Thank you both for your testing! I just tested on <2TB disks, so it's likely something is wrong with larger disks. Well, the good part is that we know that verification part is working fine  ;D

 

I'll keep you posted when I got this fixed, ok?

 

Cheers.

Link to post
  • Replies 3k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Hi guys, this is a simple plugin that allows users to clear their disks before add them to the array.   The main characteristics of this plugin are: Modularity: can be used standalon

I think Thanks should be directed at all those members that have been dealing with the mess the last few months. @dlandon and @Squid have been working behind the scenes to try and clean up, and o

Dontlookoverthere over on the unRAID subreddit figured out how to get the plugin running again with a simple edit to the plg:   https://www.reddit.com/r/unRAID/comments/7wjpus/preclear_worki

Posted Images

Just finished preclearing a 3TB WD Green, this disk was previously used on another server so skipped pre and post reads, but failed verifying signature:

 

Good to know it's not only me  :)

 

Thank you both for your testing! I just tested on <2TB disks, so it's likely something is wrong with larger disks. Well, the good part is that we know that verification part is working fine  ;D

 

I'll keep you posted when I got this fixed, ok?

 

Cheers.

I'm going to Germany until Friday, so doesn't have much time to play with the beta version, so just take your time  :)

I'll connect remotely if I'm able to and have some time in the evenings.

Link to post

 

I'm going to Germany until Friday, so doesn't have much time to play with the beta version, so just take your time  :)

I'll connect remotely if I'm able to and have some time in the evenings.

 

 

#############################################################################################
#                                                                                           #
#                         unRAID Server Pre-Clear of disk /dev/sdi                          #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#   Model Family:   Seagate Desktop HDD.15                                                  #
#   Device Model:   ST4000DM000-1F2168                                                      #
#   Serial Number:  S301HZ0B                                                                #
#   User Capacity:  4,000,787,030,016 bytes [4.00 TB]                                       #
#   Firmware:       CC54                                                                    #
#   Disk Device:    /dev/sdi                                                                #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#   Type Yes to proceed:                                                                    #
#                                                                                           #
#                                                                                           #
#############################################################################################
#                                                                                           #
#############################################################################################

#############################################################################################
#                                                                                           #
#                         unRAID Server Pre-Clear of disk /dev/sdi                          #
#                        Cycle 1 of 1, partition start on sector 64.                        #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#   Step 1 of 4 - Zeroing the disk:                                    [@ 140 MB/s] SUCCESS #
#   Step 2 of 4 - Writing unRAID's Preclear signature:                              SUCCESS #
#   Step 3 of 4 - Verifying unRAID's Preclear signature:                            SUCCESS #
#   Step 4 of 4 - Post-Read verification:                              [@ 135 MB/s] SUCCESS #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#############################################################################################
#                Cycle elapsed time: 16:06:17 | Total elapsed time: 16:06:17                #
#############################################################################################

 

The good news is that apparently I found that bug preventing >2TB disks to be precleared. Spent 16 hours for a 4TB disk, not bad. With a pre-read step, it would take 24 hours per cycle.

 

Plugin updated.

Link to post

Is there a way to preclear USB attached drives? I like to "burn in" new disks with preclear even if I don't plan to use them in unRAID. I've got a new external 2 TB 2,5" USB disk and I would like to see if it will survive the first  X hours preclearing before I'll put real data on it.

Link to post

Is there a way to preclear USB attached drives? I like to "burn in" new disks with preclear even if I don't plan to use them in unRAID. I've got a new external 2 TB 2,5" USB disk and I would like to see if it will survive the first  X hours preclearing before I'll put real data on it.

I have not tried it with the plugin, but I know using the preclear script you can clear USB drives.  There are a couple of caveats:

  • If it is USB 2 then it is very slow.  USB 3 is similar in speed to SATA connected drives
  • Many USB drives/enclosures do not report their SMART attributes.  In such a case although the preclear will operate you get limited useful information from the preclear reports as it cannot tell you about changes to reallocated and pending sectors.

Guess I need to dig out a USB drive and try it with the plugin to see if it is OK as well :)  I think I can find USB enclosures that report SMART attributes and also one that does not to test both cases.

Link to post

Thanks for the quick reply. It seems the plugin is not able to use USB drives, the attached device is not listet as a valid disk to preclear the plugin just reports: No unassigned disks available.

 

My USB drive seems to report SMART attributes, CrystalDiskInfo is able to deliver the same data as shown on internal disks. So it seems SMART report is not in issue.

 

I'll check if I'm able to start a "manuell" preclear using screen and command line.

 

Link to post

 

I'm going to Germany until Friday, so doesn't have much time to play with the beta version, so just take your time  :)

I'll connect remotely if I'm able to and have some time in the evenings.

 

 

#############################################################################################
#                                                                                           #
#                         unRAID Server Pre-Clear of disk /dev/sdi                          #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#   Model Family:   Seagate Desktop HDD.15                                                  #
#   Device Model:   ST4000DM000-1F2168                                                      #
#   Serial Number:  S301HZ0B                                                                #
#   User Capacity:  4,000,787,030,016 bytes [4.00 TB]                                       #
#   Firmware:       CC54                                                                    #
#   Disk Device:    /dev/sdi                                                                #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#   Type Yes to proceed:                                                                    #
#                                                                                           #
#                                                                                           #
#############################################################################################
#                                                                                           #
#############################################################################################

#############################################################################################
#                                                                                           #
#                         unRAID Server Pre-Clear of disk /dev/sdi                          #
#                        Cycle 1 of 1, partition start on sector 64.                        #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#   Step 1 of 4 - Zeroing the disk:                                    [@ 140 MB/s] SUCCESS #
#   Step 2 of 4 - Writing unRAID's Preclear signature:                              SUCCESS #
#   Step 3 of 4 - Verifying unRAID's Preclear signature:                            SUCCESS #
#   Step 4 of 4 - Post-Read verification:                              [@ 135 MB/s] SUCCESS #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#                                                                                           #
#############################################################################################
#                Cycle elapsed time: 16:06:17 | Total elapsed time: 16:06:17                #
#############################################################################################

 

The good news is that apparently I found that bug preventing >2TB disks to be precleared. Spent 16 hours for a 4TB disk, not bad. With a pre-read step, it would take 24 hours per cycle.

 

Plugin updated.

This time it worked  :D Well done!

Link to post

So I've tried to use the command line to preclear my USB disk but it seems it is not that easy. If I start the preclear script the following error appears:

Pre-Clear unRAID Disk /dev/sda

################################################################## 1.15c

 

smartctl may not be able to run on /dev/sda with the -d ata option.

however this should not affect the clearing of a disk.

smartctl exit status = 2

smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.1.18-unRAID] (local build)

Copyright © 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

 

Read Device Identity failed: Invalid argument

 

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.$

Do you wish to continue?

(Answer Yes to continue. Capital 'Y', lower case 'es'):

 

Of course I could prelcear the disk anyway but the script won't be able to grab the SMART data. I think the issue is the "-d ata" option. The USB device needs a "-d sat" command to read the SMART attributes.

Maybe I could modify the preclear script to use the "-d sat" option instead of "-d ata" but the script is very complex and I don't think I'm able to modify it correctly  :-\

 

Edit: okay, I confess I'm just stupid and didn't recognize that the script already allows to set a custom device type for SMART. My bad  :-X:-\

Link to post

I recently added the Preclear Plugin Beta and did not read further on the first page where it said to add the script  preclear_disk.sh.

The original Preclear Plugin was already installed and the script was in that plugin folder.

I ran the Beta Preclear plugin and it ran without having the script in the  config/plugins/preclear.disk/ directory.

 

Is there a reason it ran OK ?

 

When the Preclear is finished is there way to recall the html log that shows the finished completion ?

unraid.PNG.dff2134569ac7f3e4816807def547c65.PNG

Link to post

I recently added the Preclear Plugin Beta and did not read further on the first page where it said to add the script  preclear_disk.sh.

The original Preclear Plugin was already installed and the script was in that plugin folder.

I ran the Beta Preclear plugin and it ran without having the script in the  config/plugins/preclear.disk/ directory.

 

Is there a reason it ran OK ?

 

When the Preclear is finished is there way to recall the html log that shows the finished completion ?

The beta version has a new script written by gfjardim. It is a beta version, so don't expect all the same functionality as the other preclear plugin yet.

Link to post

I recently added the Preclear Plugin Beta and did not read further on the first page where it said to add the script  preclear_disk.sh.

The original Preclear Plugin was already installed and the script was in that plugin folder.

I ran the Beta Preclear plugin and it ran without having the script in the  config/plugins/preclear.disk/ directory.

 

Is there a reason it ran OK ?

 

When the Preclear is finished is there way to recall the html log that shows the finished completion ?

The beta version has a new script written by gfjardim. It is a beta version, so don't expect all the same functionality as the other preclear plugin yet.

 

Thanks for the prompt response.

I understand he new script will be lacking until gfjardim has an opportunity to fine tune all the necessary functionality.

 

That doesn't explain my main question which may be a bug.

 

  I ran the Beta Preclear plugin and it ran without having the preclear_disk.sh script in the  config/plugins/preclear.disk/ directory

Link to post

I recently added the Preclear Plugin Beta and did not read further on the first page where it said to add the script  preclear_disk.sh.

The original Preclear Plugin was already installed and the script was in that plugin folder.

I ran the Beta Preclear plugin and it ran without having the script in the  config/plugins/preclear.disk/ directory.

 

Is there a reason it ran OK ?

 

When the Preclear is finished is there way to recall the html log that shows the finished completion ?

The beta version has a new script written by gfjardim. It is a beta version, so don't expect all the same functionality as the other preclear plugin yet.

 

Thanks for the prompt response.

I understand he new script will be lacking until gfjardim has an opportunity to fine tune all the necessary functionality.

 

That doesn't explain my main question which may be a bug.

 

  I ran the Beta Preclear plugin and it ran without having the preclear_disk.sh script in the  config/plugins/preclear.disk/ directory

As I already said, the beta has a new script built in. So it is no bug that it ran without copying the script  ;)

Link to post

Uraid version 6.1.8

Disk: Hitachi 4 TB 7200 rpm

Preclear Disk Beta version 0.3

 

[pre]#############################################################################################

#                                                                                          #

#                        unRAID Server Pre-Clear of disk /dev/sdi                          #

#                        Cycle 1 of 1, partition start on sector 64.                        #

#                                                                                          #

#                                                                                          #

#                                                                                          #

#  Step 1 of 5 - Pre-read verification:                      [9:34:09 @ 116 MB/s] SUCCESS #

#  Step 2 of 5 - Zeroing the disk:                            [8:20:13 @ 133 MB/s] SUCCESS #

#  Step 3 of 5 - Writing unRAID's Preclear signature:                              SUCCESS #

#  Step 4 of 5 - Verifying unRAID's Preclear signature:                            SUCCESS #

#  Step 5 of 5 - Post-Read verification:                    [10:41:20 @ 103 MB/s] SUCCESS #

#                                                                                          #

#                                                                                          #

#                                                                                          #

#                                                                                          #

#                                                                                          #

#                                                                                          #

#############################################################################################

#                  Cycle elapsed time: 28:36:27 | Total elapsed time: 28:36:27            #

#############################################################################################[/pre]

Link to post

Because of dual parity I’ve been reorganizing my servers and doing a lot of preclearing, the plugin has been working great, used on disks of various sizes, 3, 4 and 8TB.

 

One thing I noticed, starting an array after starting a preclear makes the WebGUI non responsive, I bet it will respond when the preclears finish, but until then I can’t do anything, is this a known limitation?

 

Link to post

JB ... exactly the same  problems i'm having when trying to clear a number of 8TB drives. Whether i kick off a preclear from the plugin before or after starting the array any attempts at changing its status either stop or start results in total unresponsiveness of the gui and each time i have had to power cycle the server. Not ideal with an 8TB drive taking days to complete but i just havent got the time to wait and see if the server will come back to life. SO .. the only is to run a preclear on its own therefore the array is unuseable for days until i can see it end successfully. Any way around this would be appreciated.

 

Link to post

Because of dual parity I’ve been reorganizing my servers and doing a lot of preclearing, the plugin has been working great, used on disks of various sizes, 3, 4 and 8TB.

 

One thing I noticed, starting an array after starting a preclear makes the WebGUI non responsive, I bet it will respond when the preclears finish, but until then I can’t do anything, is this a known limitation?

 

 

JB ... exactly the same  problems i'm having when trying to clear a number of 8TB drives. Whether i kick off a preclear from the plugin before or after starting the array any attempts at changing its status either stop or start results in total unresponsiveness of the gui and each time i have had to power cycle the server. Not ideal with an 8TB drive taking days to complete but i just havent got the time to wait and see if the server will come back to life. SO .. the only is to run a preclear on its own therefore the array is unuseable for days until i can see it end successfully. Any way around this would be appreciated.

 

I assume you all have Unassigned Devices installed. UD caches very little of the information it presents. When you start a clear operation, the hard drive starts to be hammered very hard, and some ops, like SMART probing, takes a lot to be processed.

 

The workaround is UD start to cache most of it's info, but the last time I tried, some things got borked. It's not a trivial job.

Link to post

I assume you all have Unassigned Devices installed.

 

Don't have UD installed.

 

hum, that's new. Every time i saw this behavior, was due to UD probing preclearing disks. Preclear Disk was designed to probe display info only once and temperature every 5 minutes.

 

I'll update the beta plugin to be more verbose about execution timing. Let's see what is hanging the webui. I'll let you know when and how to test it, ok?

Link to post

... Anything to save days if not weeks of lost time will be appreciated. I don't have UD installed, should i have? Also, my scenario is slightly different inasmuch as i am preclearing an unassigned disk not yet allocated to the array. I am assuming that this is ok and that i dont need to assign it until AFTER it has been successfully cleared. J BB, looks like yours is preclearing AFTER you've assigned it, yes? Got to admit i'm only anle to view your jpeg from my ohone at present and my eyes are not good

 

 

Link to post

Thanks ... anything to save days if not weeks of lost time will be appreciated. I don't have UD installed, should i have? Also, my scenario is slightly different inasmuch as i am preclearing an unassigned disk not yet part of the array. I am assuming that this is ok and that i dont need to assign it until AFTER it has been successfully cleared. J BB, looks like yours is preclearing AFTER you've assigned it, yes? Got to admit i'm only anle to view your jpeg from my ohone at present and my eyes are not good

 

You can only preclear unassigned disks, I'm preclearing 3 disks and trying to start the array with another disk.

Link to post

Just updated to a more verbose logging.

 

Please stop the current preclear operations, using this command:

 

tmux kill-session -t "preclear_disk_sdX"

 

Start a new preclear session, wait until it hangs the webui and then send me a copy of /var/log/preclear.disk.beta.log file.

Link to post
  • trurl featured, unfeatured and pinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.