Preclear plugin


3042 posts in this topic Last Reply

Recommended Posts

1 hour ago, dabl said:

 

So gfjardim, can you please comment on these these PHP warnings?  

 

Is this something you can fix?

 

 

 

You can install the Tips and Tweaks  plugin and on the 'Tweaks' tab, set the "Turn on PHP warnings?" to 'No'   and that will suppress the    PHP Warning:  entries.  

Link to post
  • Replies 3k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Hi guys, this is a simple plugin that allows users to clear their disks before add them to the array.   The main characteristics of this plugin are: Modularity: can be used standalon

I think Thanks should be directed at all those members that have been dealing with the mess the last few months. @dlandon and @Squid have been working behind the scenes to try and clean up, and o

Dontlookoverthere over on the unRAID subreddit figured out how to get the plugin running again with a simple edit to the plg:   https://www.reddit.com/r/unRAID/comments/7wjpus/preclear_worki

Posted Images

1 hour ago, Frank1940 said:

 

You can install the Tips and Tweaks  plugin and on the 'Tweaks' tab, set the "Turn on PHP warnings?" to 'No'   and that will suppress the    PHP Warning:  entries.  

 

Thanks for the reply and info, appreciated.

Link to post

Tried to clear one of my new 6TB WD Red drives. This is a used drive I got off of eBay. I'm out of SATA ports, so I had to use my USB connection to do this. It got 83% through the pre-clear read and that stopped. When I found it this morning, the drive was already spun down and idle. Nothing logged.

 

Suggestions other than trying it again? Takes forever over USB (days) for a single pass. I'm going to try and start it again now and I'm headed out of town for a few days. If it fails again, I may fire up a spare PC and use DBAN to verify the disk.

Link to post

Double check the smart report on this drive and see what it reports.  You said the drive was used.  What are the power-on hours?  Did the seller give any reason for selling?

 

By the way, if you are using USB for preclearing, you want to use a UBS3 port on the server and a UBS3 housing for the drive.  UBS2 will be very slow...

Link to post
On 4/14/2018 at 12:45 PM, snowboardjoe said:

Tried to clear one of my new 6TB WD Red drives. This is a used drive I got off of eBay. I'm out of SATA ports, so I had to use my USB connection to do this. It got 83% through the pre-clear read and that stopped. When I found it this morning, the drive was already spun down and idle. Nothing logged.

 

Suggestions other than trying it again? Takes forever over USB (days) for a single pass. I'm going to try and start it again now and I'm headed out of town for a few days. If it fails again, I may fire up a spare PC and use DBAN to verify the disk.

 

It's also worth noting if you read (way) back through the thread I experienced exactly this kind of behavior with this kind of drive with the plugin using the default script but success with the modified Joe L script run from the command line.  I posted a preclear log and the plugin was updated but there was never a discussion/diagnosis/explanation of exactly what may have happened or what may have been fixed.  Or not fixed.  Having been here from time time the plugin existed my observation is the WD Red drives have a higher than average number of reported incidents with the plugin but it's possible I'm reading more into it than is there.  The developer with all the data/reports never participates in the discussion so who knows.

 

I've seen more than one incident with the WD Reds I've tried where for whatever reason the plugin halts somewhere most of the way through but the original script run from the command line completes without incident.  I'm definitely leaning towards returning to the use of screen and the original (modified for v6) script run from the command line and abandoning the plugin despite it being nice to use and convenient.  If it fails all the time and requires a re-run it's not very useful in the end.  

Link to post
3 minutes ago, FrozenGamer said:

If i install preclear plugin as per instructions on post 1, using gfjardim, will it work on 6.5 unraid?  Or are there still problems?

It should work. There have been and still are isolated issues being reported. If you experience any issues after installing, try uninstalling the plugin to see if that corrects them.

Link to post
On 4/7/2018 at 12:08 PM, dabl said:

 

So gfjardim, can you please comment on these these PHP warnings?  

 

Is this something you can fix?

 

 

 

These are harmless, but I'll suppress them on the next version.

Link to post
21 hours ago, dabl said:

 

It's also worth noting if you read (way) back through the thread I experienced exactly this kind of behavior with this kind of drive with the plugin using the default script but success with the modified Joe L script run from the command line.  I posted a preclear log and the plugin was updated but there was never a discussion/diagnosis/explanation of exactly what may have happened or what may have been fixed.  Or not fixed.  Having been here from time time the plugin existed my observation is the WD Red drives have a higher than average number of reported incidents with the plugin but it's possible I'm reading more into it than is there.  The developer with all the data/reports never participates in the discussion so who knows.

 

I've seen more than one incident with the WD Reds I've tried where for whatever reason the plugin halts somewhere most of the way through but the original script run from the command line completes without incident.  I'm definitely leaning towards returning to the use of screen and the original (modified for v6) script run from the command line and abandoning the plugin despite it being nice to use and convenient.  If it fails all the time and requires a re-run it's not very useful in the end.  

 

I have a theory about this problem but could not fix it without a Preclear log (please see the original post of this thread).

Link to post
7 hours ago, gfjardim said:

 

I have a theory about this problem but could not fix it without a Preclear log (please see the original post of this thread).

 

As I mentioned, in all cases I posted a preclear log.

Link to post

Issue fixed. I don't think it could deal with a USB2 connection. Both drives kept hanging on me over and over again. Spun up some spare hardware where I could plugin both hard drives to a real SATA interface. Preclear blasted right though and almost done on the post-read now.

Link to post

Is this the right place to post the results and ask questions on the clear itself?  I remember there used to be a thread for that before this plugin so wasn't sure.

 

Was clearing an older drive using "erase and clear" option, got the following error which stopped the process.

 

Apr 25 17:11:46 preclear_disk_9XK0AKZH_24453: Command: /usr/local/emhttp/plugins/preclear.disk/script/preclear_disk.sh --erase-clear --notify 4 --frequency 4 --cycles 1 --no-prompt /dev/sdb
Apr 25 17:11:46 preclear_disk_9XK0AKZH_24453: Preclear Disk Version: 0.9.5-beta
Apr 25 17:11:46 preclear_disk_9XK0AKZH_24453: S.M.A.R.T. info type: default
Apr 25 17:11:47 preclear_disk_9XK0AKZH_24453: Pre-Read: dd if=/dev/sdb of=/dev/null bs=2097152 skip=2097152 count=3000590884864 conv=notrunc iflag=nocache,sync,count_bytes,skip_bytes
Apr 25 17:11:47 preclear_disk_9XK0AKZH_24453: Pre-Read: dd pid [25616]
Apr 25 18:34:22 preclear_disk_9XK0AKZH_24453: dd[25616]: pausing (sync command issued)
Apr 25 18:34:54 preclear_disk_9XK0AKZH_24453: dd[25616]: resumed
Apr 26 00:21:41 preclear_disk_9XK0AKZH_24453: Pre-Read: dd - read 3000592982016 of 3000592982016.
Apr 26 00:21:41 preclear_disk_9XK0AKZH_24453: Pre-Read: dd exit code - 0
Apr 26 00:21:41 preclear_disk_9XK0AKZH_24453: Erasing: openssl enc -aes-256-ctr -pass pass:'oKM2mDN6kqiPry0mztZP3xtE9uQYyMy7pgsjCZEjsAVkcHSJZwPBXjXi9fn26Z7a03BkSRqAsebJYeIRyzKppVg+XEyWK+U/WNi+D0riDiF+Zq36MXHL2v3K2W1WiFUpAy7hNGXNq5aWQbClWKltpTCXXLHEMVHpndMRSCyKdTk=' -nosalt < /dev/zero 2>/dev/null | dd of=/dev/sdb bs=2097152 seek=2097152 count=3000590884864 conv=notrunc iflag=count_bytes,nocache oflag=seek_bytes iflag=fullblock 2>/tmp/.preclear/sdb/dd_output
Apr 26 00:21:41 preclear_disk_9XK0AKZH_24453: Erasing: dd pid [13610]
Apr 26 07:23:55 preclear_disk_9XK0AKZH_24453: smartctl exec_time: 7s
Apr 26 07:30:11 preclear_disk_9XK0AKZH_24453: smartctl exec_time: 7s
Apr 26 07:39:02 preclear_disk_9XK0AKZH_24453: smartctl exec_time: 1s
Apr 26 07:39:09 preclear_disk_9XK0AKZH_24453: smartctl exec_time: 8s
Apr 26 07:55:47 preclear_disk_9XK0AKZH_24453: dd process hung at 3000577818624, killing....
Apr 26 07:55:47 preclear_disk_9XK0AKZH_24453: Continuing disk write on byte 3000575721472
Apr 26 07:56:01 preclear_disk_9XK0AKZH_24453: /usr/local/emhttp/plugins/preclear.disk/script/preclear_disk.sh: line 446: 13609 Exit 1                  openssl enc -aes-256-ctr -pass pass:'oKM2mDN6kqiPry0mztZP3xtE9uQYyMy7pgsjCZEjsAVkcHSJZwPBXjXi9fn26Z7a03BkSRqAsebJYeIRyzKppVg+XEyWK+U/WNi+D0riDiF+Zq36MXHL2v3K2W1WiFUpAy7hNGXNq5aWQbClWKltpTCXXLHEMVHpndMRSCyKdTk=' -nosalt < /dev/zero 2> /dev/null
Apr 26 07:56:01 preclear_disk_9XK0AKZH_24453: 13610 Killed                  | dd of=/dev/sdb bs=2097152 seek=2097152 count=3000590884864 conv=notrunc iflag=count_bytes,nocache oflag=seek_bytes iflag=fullblock 2> /tmp/.preclear/sdb/dd_output
Apr 26 07:56:01 preclear_disk_9XK0AKZH_24453: Erasing: openssl enc -aes-256-ctr -pass pass:'vhk+SaVdyeTevJEBVabhQeoj4g5qT1XuobgHBZJspcEN+iY84H6/Hl1ZBwJgrIaCCWziEn9HsE8iWI2/8dQzfWBbeY0Dsxl/KD6Yvslw/WxJOhXKnMZDAKT4oqH52+tX7WL3pQgtahw8YH8XhdYjYbNof45P5WTlroTf6LKP1g0=' -nosalt < /dev/zero 2>/dev/null | dd of=/dev/sdb bs=2097152 seek=3000575721472 count=17260544 conv=notrunc iflag=count_bytes,nocache oflag=seek_bytes iflag=fullblock 2>/tmp/.preclear/sdb/dd_output
Apr 26 07:56:01 preclear_disk_9XK0AKZH_24453: Erasing: dd pid [16010]
Apr 26 07:56:02 preclear_disk_9XK0AKZH_24453: Erasing: dd command failed -> 8+1 records in
8+1 records out
17260544 bytes (17 MB, 16 MiB) copied, 0.0164792 s, 1.0 GB/s

 

Link to post
1 hour ago, wgstarks said:

 

That is the thread but I think his case belongs on this thread since his use of the plugin seems to have failed before producing a result.

 

Sean M.

Are you using the latest version of the plugin? Are you using one of the other scripts?

Link to post
4 hours ago, trurl said:

 

That is the thread but I think his case belongs on this thread since his use of the plugin seems to have failed before producing a result.

 

Sean M.

Are you using the latest version of the plugin? Are you using one of the other scripts?

 

Just recently re-installed the plugin.

 

Plugin Version: 018.04.24

Script: gfjardim - 0.9.5-beta

Operation: Erase and Clear the disk

 

Should I just skip the erase part and clear it?  In the past I've only used new drives and thus just cleared but this is being re-purposed from an old desktop.

 

Link to post
3 minutes ago, Sean M. said:

 

Just recently re-installed the plugin.

 

Plugin Version: 018.04.24

Script: gfjardim - 0.9.5-beta

Operation: Erase and Clear the disk

 

Should I just skip the erase part and clear it?  In the past I've only used new drives and thus just cleared but this is being re-purposed from an old desktop.

 

 

I think the purpose of erase is to prepare a disk so nothing can be recovered from it by anybody you might pass it on to. It writes random data to the disk before it clears it.

Link to post
6 minutes ago, Sean M. said:

Should I just skip the erase part and clear it? 

 

Just now, trurl said:

 

I think the purpose of erase is to prepare a disk so nothing can be recovered from it by anybody you might pass it on to. It writes random data to the disk before it clears it.

 

+1.  If you have a disk with data and it and you want to dispose of it by giving it away, selling it or tossing it in the trash, you probably want to remove any data that might be recoverable from the disk before doing so.  The "erase' and the 'erase and clear' are the two ways provided to do exactly that.  ('Erase and clear' is like 'belt and suspenders'...) 

Link to post
20 hours ago, trurl said:

 

I think the purpose of erase is to prepare a disk so nothing can be recovered from it by anybody you might pass it on to. It writes random data to the disk before it clears it.

 

20 hours ago, Frank1940 said:

 

 

+1.  If you have a disk with data and it and you want to dispose of it by giving it away, selling it or tossing it in the trash, you probably want to remove any data that might be recoverable from the disk before doing so.  The "erase' and the 'erase and clear' are the two ways provided to do exactly that.  ('Erase and clear' is like 'belt and suspenders'...) 

 

Right so since I'm just re-using my own drive, I skipped the erase.

 

It aborted in the post-read verification

 

Plugin Version: 018.04.24

Script: gfjardim - 0.9.5-beta

Operation: Clear

 

Preclear_Disk_Log: 

 

Apr 26 16:24:38 preclear_disk_9XK0AKZH_29218: Command: /usr/local/emhttp/plugins/preclear.disk/script/preclear_disk.sh --notify 4 --frequency 4 --cycles 1 --skip-preread --no-prompt /dev/sdb
Apr 26 16:24:38 preclear_disk_9XK0AKZH_29218: Preclear Disk Version: 0.9.5-beta
Apr 26 16:24:38 preclear_disk_9XK0AKZH_29218: S.M.A.R.T. info type: default
Apr 26 16:24:39 preclear_disk_9XK0AKZH_29218: Zeroing: dd if=/dev/zero of=/dev/sdb bs=2097152 seek=2097152 count=3000590884864 conv=notrunc iflag=count_bytes,nocache oflag=seek_bytes 2>/tmp/.preclear/sdb/dd_output
Apr 26 16:24:39 preclear_disk_9XK0AKZH_29218: Zeroing: dd pid [30378]
Apr 26 17:39:55 preclear_disk_9XK0AKZH_29218: smartctl exec_time: 1s
Apr 26 23:30:13 preclear_disk_9XK0AKZH_29218: smartctl exec_time: 8s
Apr 26 23:56:51 preclear_disk_9XK0AKZH_29218: smartctl exec_time: 7s
Apr 26 23:58:59 preclear_disk_9XK0AKZH_29218: Zeroing: dd - wrote 3000560017408 of 3000592982016.
Apr 26 23:59:00 preclear_disk_9XK0AKZH_29218: Zeroing: dd exit code - 0
Apr 26 23:59:01 preclear_disk_9XK0AKZH_29218: Post-Read: verifying the beggining of the disk.
Apr 26 23:59:01 preclear_disk_9XK0AKZH_29218: Post-Read: dd if=/dev/sdb bs=512 count=4095 skip=1 conv=notrunc iflag=direct  2>/tmp/.preclear/sdb/dd_output | cmp - /dev/zero &>/tmp/.preclear/sdb/cmp_out
Apr 26 23:59:01 preclear_disk_9XK0AKZH_29218: Post-Read: dd pid [1437]
Apr 26 23:59:01 preclear_disk_9XK0AKZH_29218: Post-Read: verifying the rest of the disk.
Apr 26 23:59:01 preclear_disk_9XK0AKZH_29218: Post-Read: dd if=/dev/sdb bs=2097152 skip=2097152 count=3000590884864 conv=notrunc iflag=nocache,sync,count_bytes,skip_bytes 2>/tmp/.preclear/sdb/dd_output | cmp - /dev/zero &>/tmp/.preclear/sdb/cmp_out
Apr 26 23:59:01 preclear_disk_9XK0AKZH_29218: Post-Read: dd pid [1459]
Apr 27 07:08:15 preclear_disk_9XK0AKZH_29218: Post-Read: dd - read 3000523292672 of 3000592982016.
Apr 27 07:08:15 preclear_disk_9XK0AKZH_29218: Post-Read: dd command failed, exit code [141].
Apr 27 07:08:15 preclear_disk_9XK0AKZH_29218: Post-Read: dd output -> 434+0 records in

 

Link to post
On 4/16/2018 at 7:33 PM, gfjardim said:

 

These are harmless, but I'll suppress them on the next version.

Revision history suggests that you've attempted to resolve this, but I'm still getting the PHP warnings, even after installing plugin version 2018.04.27 and rebooting.

Link to post

I am a long time user of preclear since since the late unRAID v4 days.

Personally I've always used the script, as when I started to first use the plugin, I never was able to see the output of disks being processed, and the reports when finished never provided the detail I was used to from the original script, (Not to mention they were blank.)

 

I've always just stuck to the scripts, and used the notification options built in.

 

I am however in the same boat, that I use preclear to fully stress test new disks whether they will end up in the array or not.  I also run an I.T. consultation business, where I again utilize the script on new PC HDD's, before installing them in client machines.  3 Passes of preclear with an extended SMART test, sure does give the required piece of mind, and has helped us to put fires out before there was even a sign of smoke.  (All done while Family and Friends enjoy streaming PLEX on the same server)

 

My Question(s):

 

Since I just built the newest server, it's been a while since I've updated the script I use, (Current Server on unRAID v6.1.9, New Server on v6.5.0) , and of course I want to bring the script I've grown to depend on.

  • What version of the script is recommended if I plan to only use the script?
    • I would assume the one that is being maintained, @gfjardim's, however I don't see anywhere I can get just the script?

(Forgive me if I missed it, I only picked up on this thread about 15 pages back, note: I do have my settings to show the maximum post per page, and it was quite a lot of posts!)

  • Should I even be worried about using the @gfjardim script, does the script itself have revisions/updated for later versions of unRAID, or is it merely updated for plugin purpose?
    • If I only really need @Joe L.'s or @SSD, (I assume this is now bjp999?), Should I be grabbing the originals or @gfjardim updated notification script?
      • Same question again: Does the notification updated script have any updates to the base script outside of notifications for the plugin?
  • Lastly: If it is recommended to get one of the base @Joe L., or @SSD scripts, With or without the notification update, is the patch Detailed in these posts required?

I do want to thank @Joe L., @SSD, @gfjardim, @dlandon, @Squid, @Frank1940, and @CHBMB.  I've used this script for years now. Both in, and out of unRAID, on systems destine to become workstations, servers, Security/Surveillance systems, laptops, and personal computers for both personal, and professional business clientele.  Anything with a Hard Drive, really.  This script has become one of, if not the most, valuable tools in our arsenal for new setup, system reload, and HDD reliability diagnostic.  There my be better tools, but I've yet to be failed by this.  Again, my hat off, head bowed, and knee bent, ...  Thank you.

Edited by Drider
Link to post
1 hour ago, Drider said:

Since I just built the newest server, it's been a while since I've updated the script I use, (Current Server on unRAID v6.1.9, New Server on v6.5.0) , and of course I want to bring the script I've grown to depend on.

 

Here is a link to the update required for @Joe L.'s script to work with unRAID versions after 6.2.0.   I have tested and verified that this modified script works several times but not with the most recent releases.  The last time that I tested the script was with V6.5.0.  

 

        https://lime-technology.com/forums/topic/12391-re-preclear_disksh-a-new-utility-to-burn-in-and-pre-clear-disks-for-quick-add/?page=53#comment-460592

 

Hope this helps...

Link to post
On 4/27/2018 at 2:19 PM, trurl said:

Are you sure the disk is OK? Go to Tools - Diagnostics and post complete zip.

 

Not 100% sure the disk is okay, part of the reason for the pre-clear.  It came from an old desktop that had been decommissioned for a bit, last I remember it was working fine though.

 

Full copy of diagnostics and pre-clear logs attached. 

 

The drive in question is: ST33000651AS_9XK0AKZH

Preclear.disk-20180429-0756.zip

Diagnostics-20180429-0756.zip

Link to post
11 minutes ago, Sean M. said:

Not 100% sure the disk is okay, part of the reason for the pre-clear.  It came from an old desktop that had been decommissioned for a bit, last I remember it was working fine though.

 

A disk that was working fine in an other computer may have surface defects (in an unused area of that disk) that would render it unusable for unRAID!  

 

For some reason, the SMART attributes were not reported for this disk.  It appears that their reporting was turned off for some reason. Perhaps, someone else might have some inslight into what your next course of action might be.

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.