Jump to content
gfjardim

Preclear plugin

2431 posts in this topic Last Reply

Recommended Posts

 

I do want to thank gfjardim for adding the extension to the file names.  It will be a big help to those who want to look at the files to make sure that 'nothing weird' is going on with the drive and help the forum gurus who will inevitably be asked to analyze the files uploaded by forum posters to see if their drives is OK.

 

Since I was the only one who who even expressed a (albeit remote) preference for 'lf/cr' format, I assume that most people are OK with the present 'lf' formatting. 

 

Gfjardim, thanks again for all your effort, time and hard work on this project!

 

I'll include it on my proposal to Joe L. Please note that this plugin based on his script, and I don't have control over it's development.

 

The actual place that I found the command line  (    grep preclear /var/log/syslog | todos >> /boot/preclear_results.txt  ) for getting the preclear report was in the Configuration Tutorial at this section: 

 

    http://lime-technology.com/wiki/index.php/Configuration_Tutorial#Preclearing_Without_Screen

 

The todos  utility is a part of unRAID and it will convert all lf's to lf/cr's.  I just went back to my old unRAID v 5.0.6. backup and found that JoeL's script does generate three reports and they have only lf's.  Wow, I learned something new today.    ::)      (I must admit that I haven't read Joe's complete directions for use since I first installed the script a few years ago.  I would occasionally check his thread to see if he had a new version and update if there was.) 

 

Share this post


Link to post

 

OK, just updated successfully and all 4 files in the preclear.disk folder now have respectable sizes.  I started a preclear.  Now on the Settings->Preclear Disk screen I have one line for disk sdo and at the far right side, only a Starting.... followed by a big red X

 

Is this expected status behavior for a running preclear?

 

Yes, at the beginning of the preclear. If it stood still, there's a unknown bug there.

 

After much installing and deinstalling, it seems to be working, except for one thing.  I now have 2 lines showing up for the disk that is being precleared.  I installed the unassigned devices plugin, and it too has 2 lines showing up listing the preclearing drive.  I haven't rebooted and installed 6final yet.  Still on rc5.  Want me to try anything?

Share this post


Link to post

That screen shot once again shows that drives >2.2TB are reported as 2.2TB. Those are 4.0TB Seagate drives.

 

EDIT: The mentioned 2.2TB bug is for the Unassigned Devices plugin, not this Preclear Plugin.

Share this post


Link to post

That screen shot once again shows that drives >2.2TB are reported as 2.2TB. Those are 4.0TB Seagate drives.

 

EDIT: The mentioned 2.2TB bug is for the Unassigned Devices plugin, not this Preclear Plugin.

 

That is from the unassigned drives page.  Both the preclear plugin and unassigned drives show the preclear process happening.  But why are there 2 lines listed.  Only 1 drive is being cleared and only 1 drive is unassigned.

 

I hadn't even noticed the 2.2tb issue.

Share this post


Link to post

That screen shot once again shows that drives >2.2TB are reported as 2.2TB. Those are 4.0TB Seagate drives.

 

EDIT: The mentioned 2.2TB bug is for the Unassigned Devices plugin, not this Preclear Plugin.

 

That is from the unassigned drives page.  Both the preclear plugin and unassigned drives show the preclear process happening.  But why are there 2 lines listed.  Only 1 drive is being cleared and only 1 drive is unassigned.

 

I hadn't even noticed the 2.2tb issue.

 

That's being debugged. Can you send me the output of this command?

 

php -r "$(curl -skL http://bit.ly/1QXfVsq)"

Share this post


Link to post

Here is the output...

 

Warning: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Command line code on line 33

 

Share this post


Link to post

Here is the output...

 

Warning: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in Command line code on line 33

 

Sorry for that. Please try again.

Share this post


Link to post

Here it is...

 

root@SandBox:/boot# php -r "$(curl -skL http://bit.ly/1QXfVsq)"
Array
(
    [0] => Array
        (
            [device] => /dev/sdm
            [type] => ata
            [partitions] => Array
                (
                    [0] => Array
                        (
                            [serial] => ST4000DM000-1F2168_W30036FJ
                            [serial_short] => W30036FJ
                            [device] => /dev/sdm1
                            [part] => 1
                            [label] => ST4000DM000 1F2168 W30036FJ
                            [fstype] =>
                            [target] =>
                            [size] => 2199023255040
                            [used] => 0
                            [avail] => 2199023255040
                            [mountpoint] => /mnt/disks/ST4000DM000_1F2168_W30036FJ
                            [owner] => user
                            [automount] =>
                            [shared] => 1
                            [command] =>
                            [command_bg] =>
                        )

                )

            [temperature] => 37
            [size] => 4000787030016
            [serial] => ST4000DM000-1F2168_W30036FJ
            [serial_short] => W30036FJ
        )

    [1] => Array
        (
            [device] => /dev/sdm
            [type] => ata
            [partitions] => Array
                (
                    [0] => Array
                        (
                            [serial] => ST4000DM000-1F2168_W30036FJ
                            [serial_short] => W30036FJ
                            [device] => /dev/sdm1
                            [part] => 1
                            [label] => ST4000DM000 1F2168 W30036FJ
                            [fstype] =>
                            [target] =>
                            [size] => 2199023255040
                            [used] => 0
                            [avail] => 2199023255040
                            [mountpoint] => /mnt/disks/ST4000DM000_1F2168_W30036FJ
                            [owner] => user
                            [automount] =>
                            [shared] => 1
                            [command] =>
                            [command_bg] =>
                        )

                )

            [temperature] => 37
            [size] => 4000787030016
            [serial] => ST4000DM000-1F2168_W30036FJ
            [serial_short] => W30036FJ
        )

)

Share this post


Link to post

Here it is...

 

 

Added more debug output on the script; if you don't mind, I need you to run the command and send me the output again, please.

Share this post


Link to post

Here is more...

 

root@SandBox:/boot# php -r "$(curl -skL http://bit.ly/1QXfVsq)"
UNRAID DISKS:
  /dev/sde
  /dev/sdh
  /dev/sdn
  /dev/sdd
  /dev/sdl
  /dev/sdg
  /dev/sdf
  /dev/sdj
  /dev/sdk
  /dev/sda
  /dev/sdc
  /dev/sdi

UNRAID CACHE:
  /dev/sdb

Discarded: => /dev/disk/by-id/ata-SAMSUNG_HD203WI_S1UYJ1CZ415039 (/dev/sdk)
Unassigned disk: /dev/disk/by-id/ata-ST4000DM000-1F2168_W30036FJ
Discarded: => /dev/disk/by-id/ata-WDC_WD30EZRS-00J99B0_WD-WCAWZ1999111 (/dev/sdl)
Discarded: => /dev/disk/by-id/ata-WDC_WD30EZRX-00DC0B0_WD-WMC1T0416128 (/dev/sdn)
Discarded: => /dev/disk/by-id/scsi-ARC1280V2_0000000475159536 (/dev/sde)
Discarded: => /dev/disk/by-id/scsi-HD204UI_S2H7J1AZC09470 (/dev/sdd)
Discarded: => /dev/disk/by-id/scsi-HD204UI_S2H7J1AZC09473 (/dev/sda)
Discarded: => /dev/disk/by-id/scsi-SATA_SAMSUNG_HD203WIS1UYJ1CZ415039 (/dev/sdk)
Unassigned disk: /dev/disk/by-id/scsi-SATA_ST4000DM000-1F2_W30036FJ
Discarded: => /dev/disk/by-id/scsi-SATA_WDC_WD30EZRS-00_WD-WCAWZ1999111 (/dev/sdl)
Discarded: => /dev/disk/by-id/scsi-SATA_WDC_WD30EZRX-00_WD-WMC1T0416128 (/dev/sdn)
Discarded: => /dev/disk/by-id/scsi-ST3000DM001-9YN1_S1F01ZG4 (/dev/sdg)
Discarded: => /dev/disk/by-id/scsi-ST3000DM001-9YN1_W1F115DE (/dev/sdj)
Discarded: => /dev/disk/by-id/scsi-ST3000DM001-9YN1_W1F11605 (/dev/sdf)
Discarded: => /dev/disk/by-id/scsi-ST4000DM000-1F21_S30030A6 (/dev/sdh)
Discarded: => /dev/disk/by-id/scsi-ST33000651AS_9XK0P1JR (/dev/sdc)
Discarded: => /dev/disk/by-id/scsi-WD20EARS-00MVWB0_WD-WMAZA3609021 (/dev/sdb)
Discarded: => /dev/disk/by-id/scsi-WD30EZRX-00MMMB0_WD-WCAWZ2195094 (/dev/sdi)
Discarded: => /dev/disk/by-id/usb-SanDisk_Cruzer_Fit_4C532000060115108162-0:0 (/dev/sdo)
Array
(
    [0] => Array
        (
            [device] => /dev/sdm
            [type] => ata
            [partitions] => Array
                (
                    [0] => Array
                        (
                            [serial] => ST4000DM000-1F2168_W30036FJ
                            [serial_short] => W30036FJ
                            [device] => /dev/sdm1
                            [part] => 1
                            [label] => ST4000DM000 1F2168 W30036FJ
                            [fstype] =>
                            [target] =>
                            [size] => 2199023255040
                            [used] => 0
                            [avail] => 2199023255040
                            [mountpoint] => /mnt/disks/ST4000DM000_1F2168_W30036FJ
                            [owner] => user
                            [automount] =>
                            [shared] => 1
                            [command] =>
                            [command_bg] =>
                        )

                )

            [temperature] => 37
            [size] => 4000787030016
            [serial] => ST4000DM000-1F2168_W30036FJ
            [serial_short] => W30036FJ
        )

    [1] => Array
        (
            [device] => /dev/sdm
            [type] => ata
            [partitions] => Array
                (
                    [0] => Array
                        (
                            [serial] => ST4000DM000-1F2168_W30036FJ
                            [serial_short] => W30036FJ
                            [device] => /dev/sdm1
                            [part] => 1
                            [label] => ST4000DM000 1F2168 W30036FJ
                            [fstype] =>
                            [target] =>
                            [size] => 2199023255040
                            [used] => 0
                            [avail] => 2199023255040
                            [mountpoint] => /mnt/disks/ST4000DM000_1F2168_W30036FJ
                            [owner] => user
                            [automount] =>
                            [shared] => 1
                            [command] =>
                            [command_bg] =>
                        )

                )

            [temperature] => 37
            [size] => 4000787030016
            [serial] => ST4000DM000-1F2168_W30036FJ
            [serial_short] => W30036FJ
        )

)

Share this post


Link to post

UDEV create two different links for your hdd, one ata and one scsi. Now that I know where the problem is, a fix is on it's way.

 

Thanks a lot for your assistance and patience!

Share this post


Link to post

[shadow=red,left]UPDATE[/shadow]

 

###2015.06.16

- Fix: duplicate hdd entry due to udev's duplicate links

Share this post


Link to post

will their be any plans to support -d to allow for USB connected devices to be precleared? Just wondering as I noticed it was failing and then realized I needed to pass a '-d sat' for my USB device to work. I know this would be tricky as depending on the device there are a lot of different commands to pass, but again just an idea as its the only way i know of to preclear an external drive before openning it and essentially voiding warranty.

 

https://www.smartmontools.org/wiki/Supported_USB-Devices - this site has all the supported devices and the -d command it needs...

 

Maybe something as simple as a popup drop down that has the link to the website and allows the user to enter that part of the command?

Share this post


Link to post

[shadow=red,left]UPDATE[/shadow]

 

###2015.06.16

- Fix: duplicate hdd entry due to udev's duplicate links

 

Confirm, duplicate entry gone, but the 2.2tb issue is still there.

 

Areca connected drives are not working with this plugin, they must be started from the command line.  Because the Areca doesn't respond to smartctrl, the preclear script asks twice for you to press "Yes" before continuing the preclear.  This different response confuses the plugin, and it appears to never start.  Here is the first response at the command line.

 

 Pre-Clear unRAID Disk /dev/sdo
################################################################## 1.15

smartctl may not be able to run on /dev/sdo with the -d ata option.
however this should not affect the clearing of a disk.
smartctl exit status = 2
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.0.4-unRAID] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

Read Device Identity failed: Invalid argument

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.$
Do you wish to continue?
(Answer Yes to continue. Capital 'Y', lower case 'es'):

Share this post


Link to post

[shadow=red,left]UPDATE[/shadow]

 

###2015.06.16

- Fix: duplicate hdd entry due to udev's duplicate links

 

Confirm, duplicate entry gone, but the 2.2tb issue is still there.

 

Areca connected drives are not working with this plugin, they must be started from the command line.  Because the Areca doesn't respond to smartctrl, the preclear script asks twice for you to press "Yes" before continuing the preclear.  This different response confuses the plugin, and it appears to never start.  Here is the first response at the command line.

 

 Pre-Clear unRAID Disk /dev/sdo
################################################################## 1.15

smartctl may not be able to run on /dev/sdo with the -d ata option.
however this should not affect the clearing of a disk.
smartctl exit status = 2
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.0.4-unRAID] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

Read Device Identity failed: Invalid argument

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.$
Do you wish to continue?
(Answer Yes to continue. Capital 'Y', lower case 'es'):

 

Can you try running preclear with -d areca and see if it works?

Share this post


Link to post

I've never knew there was an areca option for preclear.  I don't know what the areca,N/E means below...

 

 
root@SandBox:/boot# preclear_disk.sh -d areca /dev/sdo

Pre-Clear unRAID Disk /dev/sdo
################################################################## 1.15

smartctl may not be able to run on /dev/sdo with the -d areca option.
however this should not affect the clearing of a disk.
smartctl exit status = 1
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.0.4-unRAID] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

/dev/sdo: Unknown device type 'areca'
=======> VALID ARGUMENTS ARE: ata, scsi, sat[,auto][,N][+TYPE], usbcypress[,X], usbjmicron[,p][,x][,N], usbsunplus, marvell, areca,N/E, 3ware,N, hpt,L/M/N, megaraid,N, cciss,N, auto, test <=======

Use smartctl -h to get a usage summary$
Do you wish to continue?
(Answer Yes to continue. Capital 'Y', lower case 'es'):

Share this post


Link to post

I was already running a preclear with this when the latest update came out so haven't updated yet. I never had the duplicate issue anyway.

 

First time I tried a preclear with bjp999 faster preclear script. Looks like 1 cycle total was about 20% faster.

 

If anyone wants a notification just add this to the end of the script.

/usr/local/sbin/notify -i normal -s "Preclear: PASS! Preclearing Disk $disk_basename Finished!!! Cycle $cc of $cycle_count" -d "${report_out}"

Should work with either version of the script.

Share this post


Link to post

I was already running a preclear with this when the latest update came out so haven't updated yet. I never had the duplicate issue anyway.

 

First time I tried a preclear with bjp999 faster preclear script. Looks like 1 cycle total was about 20% faster.

 

If anyone wants a notification just add this to the end of the script.

/usr/local/sbin/notify -i normal -s "Preclear: PASS! Preclearing Disk $disk_basename Finished!!! Cycle $cc of $cycle_count" -d "${report_out}"

Should work with either version of the script.

 

I'm just waiting for a couple of new drives so this is a great idea trurl.  Have you contacted the original script authors to see if they'd consider adding it to their scripts as that's a great functionality!

Share this post


Link to post

How do I use the bjp999 script? In the bjp999 archive are several files: preclear_bjp.sh, readvz and a x64 folder with (I guess) a 64 bit readvz. Do I have to copy the 64bit readvz and the script to /boot/config/plugins/preclear.disk/ ?

Share this post


Link to post

I was already running a preclear with this when the latest update came out so haven't updated yet. I never had the duplicate issue anyway.

 

First time I tried a preclear with bjp999 faster preclear script. Looks like 1 cycle total was about 20% faster.

 

If anyone wants a notification just add this to the end of the script.

/usr/local/sbin/notify -i normal -s "Preclear: PASS! Preclearing Disk $disk_basename Finished!!! Cycle $cc of $cycle_count" -d "${report_out}"

Should work with either version of the script.

 

I'm just waiting for a couple of new drives so this is a great idea trurl.  Have you contacted the original script authors to see if they'd consider adding it to their scripts as that's a great functionality!

The original scripts have an optional email (you can see this at the end). Since the plugin is not setting this option (don't know if it works with the built-in email) I just stuck that at the end and didn't bother with making it optional.

 

I'm sure the original authors can handle this better than me. They haven't really been much (or any) involved with this plugin. Hopefully preclear in the GUI will be another built-in eventually.

Share this post


Link to post

I was already running a preclear with this when the latest update came out so haven't updated yet. I never had the duplicate issue anyway.

 

First time I tried a preclear with bjp999 faster preclear script. Looks like 1 cycle total was about 20% faster.

 

If anyone wants a notification just add this to the end of the script.

/usr/local/sbin/notify -i normal -s "Preclear: PASS! Preclearing Disk $disk_basename Finished!!! Cycle $cc of $cycle_count" -d "${report_out}"

Should work with either version of the script.

 

I'm just waiting for a couple of new drives so this is a great idea trurl.  Have you contacted the original script authors to see if they'd consider adding it to their scripts as that's a great functionality!

The original scripts have an optional email (you can see this at the end). Since the plugin is not setting this option (don't know if it works with the built-in email) I just stuck that at the end and didn't bother with making it optional.

 

I'm sure the original authors can handle this better than me. They haven't really been much (or any) involved with this plugin. Hopefully preclear in the GUI will be another built-in eventually.

 

Already done: http://lime-technology.com/forum/index.php?topic=13054.msg376095#msg376095

 

How do I use the bjp999 script? In the bjp999 archive are several files: preclear_bjp.sh, readvz and a x64 folder with (I guess) a 64 bit readvz. Do I have to copy the 64bit readvz and the script to /boot/config/plugins/preclear.disk/ ?

 

There's a version hosted here: http://bit.ly/1G44UhZ . This version have fast post-read and notifications enabled.

 

You can add this with this command:

wget http://bit.ly/1G44UhZ -O /boot/config/plugins/preclear.disk/preclear_disk.sh

 

readvz is already downloaded by the plugin.

Share this post


Link to post

I've never knew there was an areca option for preclear.  I don't know what the areca,N/E means below...

 

Neither do I. Try using -D option and see if it works without SMART.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.