Unassigned Devices Preclear - a utility to preclear disks before adding them to the array

[dlandon]

Just now, tech3475 said:

 

Just updated, the GUI is displaying ok now. Going to preclear now, thank you for the work.

UD is now using the same method as Unraid for the disk size display.

[cb]

Hi, JFYI with the last update i have "Fatal error: Cannot declare class TMUX, because the name is already in use in /usr/local/emhttp/plugins/preclear.disk/assets/lib.php on line 15" on the MAIN tab in 6.11.0-rc2, what causes empty lists. 

Am I the only one or is there a tip ? (reinstall 2 times with unraid reboot but still the same, web navigator refresh).

 

Edited July 31, 2022 by cb

[dlandon]

20 minutes ago, cb said:

Hi, JFYI with the last update i have "Fatal error: Cannot declare class TMUX, because the name is already in use in /usr/local/emhttp/plugins/preclear.disk/assets/lib.php on line 15" on the MAIN tab in 6.11.0-rc2, what causes empty lists. 

Am I the only one or is there a tip ? (reinstall 2 times with unraid reboot but still the same, web navigator refresh).

 

Post your diagnostics so I can have a look.

[cb]

Thanks for your help but in the meantime i have uninstall my 3 plugins :

community.applications.plg - 2022.07.26 
unassigned.devices.plg - 2022.07.12  
unassigned.devices.preclear.plg - 2022.07.29 

I checked inside flash/config/plugins and discover a directory which is i think "my old" preclear plugin installed outside CA long time ago. I've delete this directory, reinstall the 3 plugins and now it's ok.

 

[dlandon]

8 minutes ago, cb said:

Thanks for your help but in the meantime i have uninstall my 3 plugins :

community.applications.plg - 2022.07.26 
unassigned.devices.plg - 2022.07.12  
unassigned.devices.preclear.plg - 2022.07.29 

I checked inside flash/config/plugins and discover a directory which is i think "my old" preclear plugin installed outside CA long time ago. I've delete this directory, reinstall the 3 plugins and now it's ok.

 

I suspected that because the preclear.disk/ folder is not UD Preclear.  I wanted to confirm in your diagnostics.

[jbrukardt]

Is there a limit to this plugin that it can only pre-clear 10 disks at once?   Two of my disks for a new array arent showing up in the list.   

[dlandon]

17 minutes ago, jbrukardt said:

Is there a limit to this plugin that it can only pre-clear 10 disks at once?   Two of my disks for a new array arent showing up in the list.   

Do they show up in the UD page with the preclear icon?

[j.w.l]

I just recently installed the UD Preclear plugin to run on a few new drives that I wanted to add to my array.  No matter what options I select, I always get the following error on any of the new drives.

 

RESULT: FAIL! /dev/sdd DOESN'T have a valid Unraid MBR signature!!!

 

I'm running Unraid version 6.10.3 and have installed version 2022.08.19 of the plugin.  Any ideas on what might be going wrong?

[dlandon]

On 8/23/2022 at 2:51 PM, j.w.l said:

I just recently installed the UD Preclear plugin to run on a few new drives that I wanted to add to my array.  No matter what options I select, I always get the following error on any of the new drives.

 

RESULT: FAIL! /dev/sdd DOESN'T have a valid Unraid MBR signature!!!

 

I'm running Unraid version 6.10.3 and have installed version 2022.08.19 of the plugin.  Any ideas on what might be going wrong?

This can sometimes happen when you have bad memory.  Stop your server and run a memory test.

[j.w.l]

On 8/25/2022 at 3:47 AM, dlandon said:

This can sometimes happen when you have bad memory.  Stop your server and run a memory test.

 

Currently running Memtest86+ and so far after 5 passes there have been no errors.  Is there anything else that I can look into that may cause this error?

[dlandon]

Just now, j.w.l said:

 

Currently running Memtest86+ and so far after 5 passes there have been no errors.  Is there anything else that I can look into that may cause this error?

Download an post the preclear logs.

[j.w.l]

18 minutes ago, dlandon said:

Download an post the preclear logs.

 

Sure, here you go.

TOWER-unassigned.devices.preclear-20220826-1926.zip

[dlandon]

On 8/26/2022 at 9:40 PM, j.w.l said:

 

Sure, here you go.

TOWER-unassigned.devices.preclear-20220826-1926.zip 1.4 kB · 1 download

All I see in the logs is a preclear signature check.  Did you preclear the disks with UD preclear?  Start a preclear and choose 'Clear Disk'.

[daithi]

Searched this thread but didn't really find what I was after. I'm completely new to Unraid (in fact I haven't even booted it up yet, still doing research) so bear that in mind if my question isn't phrased as well as it could be.

Is this plugin in any way a replacement for badblocks? From watching a Spaceinvader video (this one) I see you can do a pre-read, post-read and define a number of cycles.

If not, is there a recommended way of testing new drives for "infant mortality" with Unraid? I've used badblocks in the past but it's a pain in the ass, I had to use all sorts of workarounds for bigger drives due to 32bit limitations and I don't remember what I did (I'm ready to leave no gui Ubuntu server behind forever).

Seems like it's an outdated piece of software that people are clinging to.

[WillCroPoint]

5 hours ago, daithi said:

Searched this thread but didn't really find what I was after. I'm completely new to Unraid (in fact I haven't even booted it up yet, still doing research) so bear that in mind if my question isn't phrased as well as it could be.

Is this plugin in any way a replacement for badblocks? From watching a Spaceinvader video (this one) I see you can do a pre-read, post-read and define a number of cycles.

If not, is there a recommended way of testing new drives for "infant mortality" with Unraid? I've used badblocks in the past but it's a pain in the ass, I had to use all sorts of workarounds for bigger drives due to 32bit limitations and I don't remember what I did (I'm ready to leave no gui Ubuntu server behind forever).

Seems like it's an outdated piece of software that people are clinging to.

I guess the answer is yes. 
 

Even though the plug-in clears the disks (and marks them as such) in order to avoid an unnecessary parity reconstruction when added to the array, it can also « stress test » the disks in order to highlight such defects. It must be emphasized the disk content are cleared. 

[DeathStar Darth]

I have added another drive and looked to perform the usual preclear process.

I had previously observed that the trusty precelar plugin I have used on 8+ disk previously was depreciated 

I have removed it and added the UD preclear plugin in its place.

 

However, I am running into issues resulting in the preclear seemingly doing nothing (no reported read/write activity in the gui).

Looking at the logs for my latest attempt I initially see

/usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh
--cycles 1 --no-prompt /dev/sdj

and shortly after nothing but repetitive lines of

-bash: /bin/dircolors: Argument list too long
-bash: /usr/bin/grep: Argument list too long
-bash: /bin/hostname: Argument list too long

As mentioned, the read/write rate in the unraid gui remains at zero throughout.

I have tried several times with various options, but all result in the same output

Sep 2 15:23:37 Preclear script invoked as: /usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh  --notify 4 --frequency 4 --cycles 2 --no-prompt /dev/sdj
Sep 2 15:33:32 Preclear stopped on device: Z180A014FJDH
Sep 2 15:34:46 Preclear script invoked as: /usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh  --notify 4 --frequency 4 --cycles 2 --skip-preread --skip-postread --no-prompt /dev/sdj
Sep 2 15:52:31 Preclear stopped on device: Z180A014FJDH
Sep 2 15:54:13 Preclear script invoked as: /usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh  --cycles 1 --no-prompt /dev/sdj

diskinfo looks fine

Fri Sep 2 14:31:33 BST 2022: SMART parameters for drive [/dev/sdj] not found, probing...
Fri Sep 2 14:31:33 BST 2022: Trying SMART parameter (-d auto) for disk [/dev/sdj]...
Fri Sep 2 14:31:33 BST 2022: SMART parameter for disk [/dev/sdj] (-d auto) found.
Fri Sep 2 15:05:58 BST 2022: SMART parameters for drive [/dev/sdj] not found, probing...
Fri Sep 2 15:05:58 BST 2022: Trying SMART parameter (-d auto) for disk [/dev/sdj]...
Fri Sep 2 15:05:58 BST 2022: SMART parameter for disk [/dev/sdj] (-d auto) found.
Fri Sep 2 15:06:53 BST 2022: Stopping Disk Info Daemon

 

Any advice of what to try/check from here in order to get the disk precleared?

[dlandon]

22 minutes ago, DeathStar Darth said:

I have added another drive and looked to perform the usual preclear process.

I had previously observed that the trusty precelar plugin I have used on 8+ disk previously was depreciated 

I have removed it and added the UD preclear plugin in its place.

 

However, I am running into issues resulting in the preclear seemingly doing nothing (no reported read/write activity in the gui).

Looking at the logs for my latest attempt I initially see

/usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh
--cycles 1 --no-prompt /dev/sdj

and shortly after nothing but repetitive lines of

-bash: /bin/dircolors: Argument list too long
-bash: /usr/bin/grep: Argument list too long
-bash: /bin/hostname: Argument list too long

As mentioned, the read/write rate in the unraid gui remains at zero throughout.

I have tried several times with various options, but all result in the same output

Sep 2 15:23:37 Preclear script invoked as: /usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh  --notify 4 --frequency 4 --cycles 2 --no-prompt /dev/sdj
Sep 2 15:33:32 Preclear stopped on device: Z180A014FJDH
Sep 2 15:34:46 Preclear script invoked as: /usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh  --notify 4 --frequency 4 --cycles 2 --skip-preread --skip-postread --no-prompt /dev/sdj
Sep 2 15:52:31 Preclear stopped on device: Z180A014FJDH
Sep 2 15:54:13 Preclear script invoked as: /usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh  --cycles 1 --no-prompt /dev/sdj

diskinfo looks fine

Fri Sep 2 14:31:33 BST 2022: SMART parameters for drive [/dev/sdj] not found, probing...
Fri Sep 2 14:31:33 BST 2022: Trying SMART parameter (-d auto) for disk [/dev/sdj]...
Fri Sep 2 14:31:33 BST 2022: SMART parameter for disk [/dev/sdj] (-d auto) found.
Fri Sep 2 15:05:58 BST 2022: SMART parameters for drive [/dev/sdj] not found, probing...
Fri Sep 2 15:05:58 BST 2022: Trying SMART parameter (-d auto) for disk [/dev/sdj]...
Fri Sep 2 15:05:58 BST 2022: SMART parameter for disk [/dev/sdj] (-d auto) found.
Fri Sep 2 15:06:53 BST 2022: Stopping Disk Info Daemon

 

Any advice of what to try/check from here in order to get the disk precleared?

What does the preclear status show?

[DeathStar Darth]

5 hours ago, dlandon said:

What does the preclear status show?

 

 

 

[dlandon]

7 minutes ago, DeathStar Darth said:

 

 

 

Be sure the old preclear disk plugin is removed and reboot.  Looks like some leftovers from the preclear disk remain.

[DeathStar Darth]

2 hours ago, dlandon said:

Be sure the old preclear disk plugin is removed and reboot.  Looks like some leftovers from the preclear disk remain.

 I don't see old the plugin in /boot/config/plugins-removed, but neither do I see it in /boot/config/plugins

 

I have rebooted and started the preclear again.

I still see it seemingly stuck in 'starting' state

and again the log captures the same repeating error entries.

 

Very occasionally hitting refresh I register some read activity suggesting it is at least trying to do something (the pre read I guess) in spite of what the logs show/don't show

 

The other change I have made is to upgrade to Pro license. I can't see that would cause this issue but added the info 'just in case'.

[dlandon]

What does the status show  Click on the eye icon.

 

Download the preclear logs.

[DeathStar Darth]

On 9/3/2022 at 2:21 AM, dlandon said:

What does the status show  Click on the eye icon.

 

Download the preclear logs.

 

Latest log files attached.

I notice the diskinfo_log file is empty on this run. Previous runs had it contain 

Fri Sep 2 14:31:33 BST 2022: SMART parameters for drive [/dev/sdj] not found, probing...
Fri Sep 2 14:31:33 BST 2022: Trying SMART parameter (-d auto) for disk [/dev/sdj]...
Fri Sep 2 14:31:33 BST 2022: SMART parameter for disk [/dev/sdj] (-d auto) found.
Fri Sep 2 15:05:58 BST 2022: SMART parameters for drive [/dev/sdj] not found, probing...
Fri Sep 2 15:05:58 BST 2022: Trying SMART parameter (-d auto) for disk [/dev/sdj]...
Fri Sep 2 15:05:58 BST 2022: SMART parameter for disk [/dev/sdj] (-d auto) found.
Fri Sep 2 15:06:53 BST 2022: Stopping Disk Info Daemon

 

THEDEATHSTAR-unassigned.devices.preclear-20220904-1503.zip

[dlandon]

3 hours ago, DeathStar Darth said:

 

Latest log files attached.

I notice the diskinfo_log file is empty on this run. Previous runs had it contain 

Fri Sep 2 14:31:33 BST 2022: SMART parameters for drive [/dev/sdj] not found, probing...
Fri Sep 2 14:31:33 BST 2022: Trying SMART parameter (-d auto) for disk [/dev/sdj]...
Fri Sep 2 14:31:33 BST 2022: SMART parameter for disk [/dev/sdj] (-d auto) found.
Fri Sep 2 15:05:58 BST 2022: SMART parameters for drive [/dev/sdj] not found, probing...
Fri Sep 2 15:05:58 BST 2022: Trying SMART parameter (-d auto) for disk [/dev/sdj]...
Fri Sep 2 15:05:58 BST 2022: SMART parameter for disk [/dev/sdj] (-d auto) found.
Fri Sep 2 15:06:53 BST 2022: Stopping Disk Info Daemon

 

THEDEATHSTAR-unassigned.devices.preclear-20220904-1503.zip 1.12 kB · 0 downloads

The diskinfo_log you posted is from the old preclear disk plugin and doesn't apply to the UD Preclear.

 

Post your diagnostics file.

[daithi]

On 9/2/2022 at 4:13 AM, WillCroPoint said:

I guess the answer is yes. 
 

Even though the plug-in clears the disks (and marks them as such) in order to avoid an unnecessary parity reconstruction when added to the array, it can also « stress test » the disks in order to highlight such defects. It must be emphasized the disk content are cleared. 

Stress test is what I'm after I think. I need to wipe the disks anyway as they're ext4 not xfs

[wgstarks]

6 minutes ago, daithi said:

Stress test is what I'm after I think. I need to wipe the disks anyway as they're ext4 not xfs

You can just use UD to remove any existing partitions (destructive mode must be enabled in settings) and then run as many preclear cycles as you like. I usually run 3 for a stress test. When they are complete don’t create a partition if you’re planning to add the drive to your array. Let unRAID create the partition when it’s added.