Preclear plugin


3040 posts in this topic Last Reply

Recommended Posts

  • Replies 3k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Hi guys, this is a simple plugin that allows users to clear their disks before add them to the array.   The main characteristics of this plugin are: Modularity: can be used standalon

I think Thanks should be directed at all those members that have been dealing with the mess the last few months. @dlandon and @Squid have been working behind the scenes to try and clean up, and o

Dontlookoverthere over on the unRAID subreddit figured out how to get the plugin running again with a simple edit to the plg:   https://www.reddit.com/r/unRAID/comments/7wjpus/preclear_worki

Posted Images

That screen shot once again shows that drives >2.2TB are reported as 2.2TB. Those are 4.0TB Seagate drives.

 

EDIT: The mentioned 2.2TB bug is for the Unassigned Devices plugin, not this Preclear Plugin.

 

That is from the unassigned drives page.  Both the preclear plugin and unassigned drives show the preclear process happening.  But why are there 2 lines listed.  Only 1 drive is being cleared and only 1 drive is unassigned.

 

I hadn't even noticed the 2.2tb issue.

Link to post

That screen shot once again shows that drives >2.2TB are reported as 2.2TB. Those are 4.0TB Seagate drives.

 

EDIT: The mentioned 2.2TB bug is for the Unassigned Devices plugin, not this Preclear Plugin.

 

That is from the unassigned drives page.  Both the preclear plugin and unassigned drives show the preclear process happening.  But why are there 2 lines listed.  Only 1 drive is being cleared and only 1 drive is unassigned.

 

I hadn't even noticed the 2.2tb issue.

 

That's being debugged. Can you send me the output of this command?

 

php -r "$(curl -skL http://bit.ly/1QXfVsq)"

Link to post

Here is the output...

 

Warning: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Command line code on line 33

 

Link to post

Here is the output...

 

Warning: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Command line code on line 33

 

Sorry for that. Please try again.

Link to post

Here it is...

 

root@SandBox:/boot# php -r "$(curl -skL http://bit.ly/1QXfVsq)"
Array
(
    [0] => Array
        (
            [device] => /dev/sdm
            [type] => ata
            [partitions] => Array
                (
                    [0] => Array
                        (
                            [serial] => ST4000DM000-1F2168_W30036FJ
                            [serial_short] => W30036FJ
                            [device] => /dev/sdm1
                            [part] => 1
                            [label] => ST4000DM000 1F2168 W30036FJ
                            [fstype] =>
                            [target] =>
                            [size] => 2199023255040
                            [used] => 0
                            [avail] => 2199023255040
                            [mountpoint] => /mnt/disks/ST4000DM000_1F2168_W30036FJ
                            [owner] => user
                            [automount] =>
                            [shared] => 1
                            [command] =>
                            [command_bg] =>
                        )

                )

            [temperature] => 37
            [size] => 4000787030016
            [serial] => ST4000DM000-1F2168_W30036FJ
            [serial_short] => W30036FJ
        )

    [1] => Array
        (
            [device] => /dev/sdm
            [type] => ata
            [partitions] => Array
                (
                    [0] => Array
                        (
                            [serial] => ST4000DM000-1F2168_W30036FJ
                            [serial_short] => W30036FJ
                            [device] => /dev/sdm1
                            [part] => 1
                            [label] => ST4000DM000 1F2168 W30036FJ
                            [fstype] =>
                            [target] =>
                            [size] => 2199023255040
                            [used] => 0
                            [avail] => 2199023255040
                            [mountpoint] => /mnt/disks/ST4000DM000_1F2168_W30036FJ
                            [owner] => user
                            [automount] =>
                            [shared] => 1
                            [command] =>
                            [command_bg] =>
                        )

                )

            [temperature] => 37
            [size] => 4000787030016
            [serial] => ST4000DM000-1F2168_W30036FJ
            [serial_short] => W30036FJ
        )

)

Link to post

Here is more...

 

root@SandBox:/boot# php -r "$(curl -skL http://bit.ly/1QXfVsq)"
UNRAID DISKS:
  /dev/sde
  /dev/sdh
  /dev/sdn
  /dev/sdd
  /dev/sdl
  /dev/sdg
  /dev/sdf
  /dev/sdj
  /dev/sdk
  /dev/sda
  /dev/sdc
  /dev/sdi

UNRAID CACHE:
  /dev/sdb

Discarded: => /dev/disk/by-id/ata-SAMSUNG_HD203WI_S1UYJ1CZ415039 (/dev/sdk)
Unassigned disk: /dev/disk/by-id/ata-ST4000DM000-1F2168_W30036FJ
Discarded: => /dev/disk/by-id/ata-WDC_WD30EZRS-00J99B0_WD-WCAWZ1999111 (/dev/sdl)
Discarded: => /dev/disk/by-id/ata-WDC_WD30EZRX-00DC0B0_WD-WMC1T0416128 (/dev/sdn)
Discarded: => /dev/disk/by-id/scsi-ARC1280V2_0000000475159536 (/dev/sde)
Discarded: => /dev/disk/by-id/scsi-HD204UI_S2H7J1AZC09470 (/dev/sdd)
Discarded: => /dev/disk/by-id/scsi-HD204UI_S2H7J1AZC09473 (/dev/sda)
Discarded: => /dev/disk/by-id/scsi-SATA_SAMSUNG_HD203WIS1UYJ1CZ415039 (/dev/sdk)
Unassigned disk: /dev/disk/by-id/scsi-SATA_ST4000DM000-1F2_W30036FJ
Discarded: => /dev/disk/by-id/scsi-SATA_WDC_WD30EZRS-00_WD-WCAWZ1999111 (/dev/sdl)
Discarded: => /dev/disk/by-id/scsi-SATA_WDC_WD30EZRX-00_WD-WMC1T0416128 (/dev/sdn)
Discarded: => /dev/disk/by-id/scsi-ST3000DM001-9YN1_S1F01ZG4 (/dev/sdg)
Discarded: => /dev/disk/by-id/scsi-ST3000DM001-9YN1_W1F115DE (/dev/sdj)
Discarded: => /dev/disk/by-id/scsi-ST3000DM001-9YN1_W1F11605 (/dev/sdf)
Discarded: => /dev/disk/by-id/scsi-ST4000DM000-1F21_S30030A6 (/dev/sdh)
Discarded: => /dev/disk/by-id/scsi-ST33000651AS_9XK0P1JR (/dev/sdc)
Discarded: => /dev/disk/by-id/scsi-WD20EARS-00MVWB0_WD-WMAZA3609021 (/dev/sdb)
Discarded: => /dev/disk/by-id/scsi-WD30EZRX-00MMMB0_WD-WCAWZ2195094 (/dev/sdi)
Discarded: => /dev/disk/by-id/usb-SanDisk_Cruzer_Fit_4C532000060115108162-0:0 (/dev/sdo)
Array
(
    [0] => Array
        (
            [device] => /dev/sdm
            [type] => ata
            [partitions] => Array
                (
                    [0] => Array
                        (
                            [serial] => ST4000DM000-1F2168_W30036FJ
                            [serial_short] => W30036FJ
                            [device] => /dev/sdm1
                            [part] => 1
                            [label] => ST4000DM000 1F2168 W30036FJ
                            [fstype] =>
                            [target] =>
                            [size] => 2199023255040
                            [used] => 0
                            [avail] => 2199023255040
                            [mountpoint] => /mnt/disks/ST4000DM000_1F2168_W30036FJ
                            [owner] => user
                            [automount] =>
                            [shared] => 1
                            [command] =>
                            [command_bg] =>
                        )

                )

            [temperature] => 37
            [size] => 4000787030016
            [serial] => ST4000DM000-1F2168_W30036FJ
            [serial_short] => W30036FJ
        )

    [1] => Array
        (
            [device] => /dev/sdm
            [type] => ata
            [partitions] => Array
                (
                    [0] => Array
                        (
                            [serial] => ST4000DM000-1F2168_W30036FJ
                            [serial_short] => W30036FJ
                            [device] => /dev/sdm1
                            [part] => 1
                            [label] => ST4000DM000 1F2168 W30036FJ
                            [fstype] =>
                            [target] =>
                            [size] => 2199023255040
                            [used] => 0
                            [avail] => 2199023255040
                            [mountpoint] => /mnt/disks/ST4000DM000_1F2168_W30036FJ
                            [owner] => user
                            [automount] =>
                            [shared] => 1
                            [command] =>
                            [command_bg] =>
                        )

                )

            [temperature] => 37
            [size] => 4000787030016
            [serial] => ST4000DM000-1F2168_W30036FJ
            [serial_short] => W30036FJ
        )

)

Link to post

will their be any plans to support -d to allow for USB connected devices to be precleared? Just wondering as I noticed it was failing and then realized I needed to pass a '-d sat' for my USB device to work. I know this would be tricky as depending on the device there are a lot of different commands to pass, but again just an idea as its the only way i know of to preclear an external drive before openning it and essentially voiding warranty.

 

https://www.smartmontools.org/wiki/Supported_USB-Devices - this site has all the supported devices and the -d command it needs...

 

Maybe something as simple as a popup drop down that has the link to the website and allows the user to enter that part of the command?

Link to post

[shadow=red,left]UPDATE[/shadow]

 

###2015.06.16

- Fix: duplicate hdd entry due to udev's duplicate links

 

Confirm, duplicate entry gone, but the 2.2tb issue is still there.

 

Areca connected drives are not working with this plugin, they must be started from the command line.  Because the Areca doesn't respond to smartctrl, the preclear script asks twice for you to press "Yes" before continuing the preclear.  This different response confuses the plugin, and it appears to never start.  Here is the first response at the command line.

 

 Pre-Clear unRAID Disk /dev/sdo
################################################################## 1.15

smartctl may not be able to run on /dev/sdo with the -d ata option.
however this should not affect the clearing of a disk.
smartctl exit status = 2
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.0.4-unRAID] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

Read Device Identity failed: Invalid argument

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.$
Do you wish to continue?
(Answer Yes to continue. Capital 'Y', lower case 'es'):

Link to post

[shadow=red,left]UPDATE[/shadow]

 

###2015.06.16

- Fix: duplicate hdd entry due to udev's duplicate links

 

Confirm, duplicate entry gone, but the 2.2tb issue is still there.

 

Areca connected drives are not working with this plugin, they must be started from the command line.  Because the Areca doesn't respond to smartctrl, the preclear script asks twice for you to press "Yes" before continuing the preclear.  This different response confuses the plugin, and it appears to never start.  Here is the first response at the command line.

 

 Pre-Clear unRAID Disk /dev/sdo
################################################################## 1.15

smartctl may not be able to run on /dev/sdo with the -d ata option.
however this should not affect the clearing of a disk.
smartctl exit status = 2
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.0.4-unRAID] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

Read Device Identity failed: Invalid argument

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.$
Do you wish to continue?
(Answer Yes to continue. Capital 'Y', lower case 'es'):

 

Can you try running preclear with -d areca and see if it works?

Link to post

I've never knew there was an areca option for preclear.  I don't know what the areca,N/E means below...

 

 
root@SandBox:/boot# preclear_disk.sh -d areca /dev/sdo

Pre-Clear unRAID Disk /dev/sdo
################################################################## 1.15

smartctl may not be able to run on /dev/sdo with the -d areca option.
however this should not affect the clearing of a disk.
smartctl exit status = 1
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.0.4-unRAID] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

/dev/sdo: Unknown device type 'areca'
=======> VALID ARGUMENTS ARE: ata, scsi, sat[,auto][,N][+TYPE], usbcypress[,X], usbjmicron[,p][,x][,N], usbsunplus, marvell, areca,N/E, 3ware,N, hpt,L/M/N, megaraid,N, cciss,N, auto, test <=======

Use smartctl -h to get a usage summary$
Do you wish to continue?
(Answer Yes to continue. Capital 'Y', lower case 'es'):

Link to post

I was already running a preclear with this when the latest update came out so haven't updated yet. I never had the duplicate issue anyway.

 

First time I tried a preclear with bjp999 faster preclear script. Looks like 1 cycle total was about 20% faster.

 

If anyone wants a notification just add this to the end of the script.

/usr/local/sbin/notify -i normal -s "Preclear: PASS! Preclearing Disk $disk_basename Finished!!! Cycle $cc of $cycle_count" -d "${report_out}"

Should work with either version of the script.

Link to post

I was already running a preclear with this when the latest update came out so haven't updated yet. I never had the duplicate issue anyway.

 

First time I tried a preclear with bjp999 faster preclear script. Looks like 1 cycle total was about 20% faster.

 

If anyone wants a notification just add this to the end of the script.

/usr/local/sbin/notify -i normal -s "Preclear: PASS! Preclearing Disk $disk_basename Finished!!! Cycle $cc of $cycle_count" -d "${report_out}"

Should work with either version of the script.

 

I'm just waiting for a couple of new drives so this is a great idea trurl.  Have you contacted the original script authors to see if they'd consider adding it to their scripts as that's a great functionality!

Link to post

How do I use the bjp999 script? In the bjp999 archive are several files: preclear_bjp.sh, readvz and a x64 folder with (I guess) a 64 bit readvz. Do I have to copy the 64bit readvz and the script to /boot/config/plugins/preclear.disk/ ?

Link to post

I was already running a preclear with this when the latest update came out so haven't updated yet. I never had the duplicate issue anyway.

 

First time I tried a preclear with bjp999 faster preclear script. Looks like 1 cycle total was about 20% faster.

 

If anyone wants a notification just add this to the end of the script.

/usr/local/sbin/notify -i normal -s "Preclear: PASS! Preclearing Disk $disk_basename Finished!!! Cycle $cc of $cycle_count" -d "${report_out}"

Should work with either version of the script.

 

I'm just waiting for a couple of new drives so this is a great idea trurl.  Have you contacted the original script authors to see if they'd consider adding it to their scripts as that's a great functionality!

The original scripts have an optional email (you can see this at the end). Since the plugin is not setting this option (don't know if it works with the built-in email) I just stuck that at the end and didn't bother with making it optional.

 

I'm sure the original authors can handle this better than me. They haven't really been much (or any) involved with this plugin. Hopefully preclear in the GUI will be another built-in eventually.

Link to post

I was already running a preclear with this when the latest update came out so haven't updated yet. I never had the duplicate issue anyway.

 

First time I tried a preclear with bjp999 faster preclear script. Looks like 1 cycle total was about 20% faster.

 

If anyone wants a notification just add this to the end of the script.

/usr/local/sbin/notify -i normal -s "Preclear: PASS! Preclearing Disk $disk_basename Finished!!! Cycle $cc of $cycle_count" -d "${report_out}"

Should work with either version of the script.

 

I'm just waiting for a couple of new drives so this is a great idea trurl.  Have you contacted the original script authors to see if they'd consider adding it to their scripts as that's a great functionality!

The original scripts have an optional email (you can see this at the end). Since the plugin is not setting this option (don't know if it works with the built-in email) I just stuck that at the end and didn't bother with making it optional.

 

I'm sure the original authors can handle this better than me. They haven't really been much (or any) involved with this plugin. Hopefully preclear in the GUI will be another built-in eventually.

 

Already done: http://lime-technology.com/forum/index.php?topic=13054.msg376095#msg376095

 

How do I use the bjp999 script? In the bjp999 archive are several files: preclear_bjp.sh, readvz and a x64 folder with (I guess) a 64 bit readvz. Do I have to copy the 64bit readvz and the script to /boot/config/plugins/preclear.disk/ ?

 

There's a version hosted here: http://bit.ly/1G44UhZ . This version have fast post-read and notifications enabled.

 

You can add this with this command:

wget http://bit.ly/1G44UhZ -O /boot/config/plugins/preclear.disk/preclear_disk.sh

 

readvz is already downloaded by the plugin.

Link to post

You can add this with this command:

wget http://bit.ly/1G44UhZ -O /boot/config/plugins/preclear.disk/preclear_disk.sh

Much better than my hack. Options integrated into the plugin dialog. Thanks.

 

One thing I noticed is the Mail dropdown spells cycle like this: cicle.

Link to post

You can add this with this command:

wget http://bit.ly/1G44UhZ -O /boot/config/plugins/preclear.disk/preclear_disk.sh

Much better than my hack. Options integrated into the plugin dialog. Thanks.

 

One thing I noticed is the Mail dropdown spells cycle like this: cicle.

 

This is what you get when you let brazilians code plugins for you....  ::)

 

We don't use Y in portuguese words.

Link to post
  • trurl featured, unfeatured and pinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.