Unassigned Devices Preclear - a utility to preclear disks before adding them to the array

[itimpi]

On 6/14/2022 at 12:57 AM, mikela said:

So just stop the array and put it in position 10 and unraid will recognize the preclear signature and format it ?

The signature will be recognised but there is no automatic format.    The disk will show as unmountable and you will have the option to format unmountable disks.   Check it is only the one you expect before proceeding with the format.

[theothermatt_b]

Hey I just wanted to pop in here and say thanks for this series of plugins. I used them for the very first time today and (after doing a little bit of reading because the disk already had a filesystem that i had to delete first), it's currently in the middle of an Erase and Clear. Thanks!!

[dlandon]

18 minutes ago, theothermatt_b said:

the disk already had a filesystem that i had to delete first

This is done for safety.  You don't want to accidently erase a disk with data on it.

[theothermatt_b]

1 minute ago, dlandon said:

This is done for safety.  You don't want to accidently erase a disk with data on it.

No absolutely! It made a ton of sense, I just had to dig up how to get around it (which was not hard to do).

And it worked perfectly!

[dlandon]

Just now, DiscoverIt said:

The new update isn't processing periods in a host?

 

 

and all existing configurations are invalid?

 

This is UD not UD Preclear.  Please post in the UD forum.

[DiscoverIt]

Sorry missed the title suffix

[Indi]

I am new to using this plugin, I thought I understood reading the initial post on this thread on how to use it, but I am confused. 

I chose the default when clearing some new drives, which was one cycle. I didn't see in the details that one "cycle" is actually 5 steps? 

Step 1: Pre-read verification - Took 26 hours
Step 2: Zeroing the disk - Took 28 hours
Step 3: Write preclear sig
Step 4: Verify preclear sig
Step 5: (Where I am now) Post-Read in progress? 1% done. 

Overall, I expect this to all be done in approximately 80 hours. 

This seems way more extensive than if I had unraid perform it, I am not complaining though as I just stress tested the hell out of these new drives I got. 

Question: Is there a cycle that won't take 4 days to complete on 20TB drives? Or is this the fastest option for this plugin 

[wgstarks]

25 minutes ago, Indi said:

I am not complaining though as I just stress tested the hell out of these new drives I got. 

The stress test is probably the primary reason most people would use this plugin. In the past unRAID couldn’t clear a drive with the array started so users would PREclear drives to avoid long periods off-line. That’s no longer the case so it really only makes sense to preclear if you want to stress a new drive to be sure of its health status. I typically run 3 cycles but as you say they do take a while. I think 4 days for a 20TB drive is actually pretty fast though.

[itimpi]

10 hours ago, Indi said:

This seems way more extensive than if I had unraid perform it, I am not complaining though as I just stress tested the hell out of these new drives I got. 

 

There is a discussion of Clear v Preclear here in the online documentation accessible via the Manual link at the bottom of the Unraid GUI.  In addition every forum page has a DOCS link at the top and a Documentation link at the bottom.  

[nohuhu]

Hi dlandon,

 

I can't install the plugin on my brand new Unraid build, it shows the error message:

 

```

Requirements not met

The requirements for this upgrade have not been met on your system

This application has additional requirements

Unassigned Devices installed

```

 

This is despite having two unassigned devices listed on the Dashboard:

 

 

What can be wrong here? Thanks in advance.

 

[dlandon]

3 minutes ago, nohuhu said:

What can be wrong here? Thanks in advance.

You need the Unassigned Devices plugin installed.

[nohuhu]

2 hours ago, dlandon said:

You need the Unassigned Devices plugin installed.

 

Oh wow, thanks for such a quick reply! If I could suggest an improvement in error messaging, it would be great to have the Unassigned Devices plugin mentioned in the error. As it is now, it's very confusing and can be easily interpreted as the need to have unassigned devices.

 

Works like a charm now.

[Provocateur]

My new disk is failing during the pre-read. Have precleared disks in the past without issue (though, not within the last couple of years...). Unraid dashboard shows 50+ GB of free memory. Any ideas? Cheers.

 

Dec 07 19:17:32 preclear_disk_ZR5A2ZA6_13445: Blocks (512 bytes): 35156656128 
Dec 07 19:17:32 preclear_disk_ZR5A2ZA6_13445: Block size: 4096 
Dec 07 19:17:32 preclear_disk_ZR5A2ZA6_13445: Start sector: 0 
Dec 07 19:17:36 preclear_disk_ZR5A2ZA6_13445: Pre-read: pre-read verification started 1 of 5 retries... 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: size:         0, available:     0, free: 0% 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: Filesystem     1K-blocks    Used Available Use% Mounted on 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: tmpfs              32768     364     32404   2% /run 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: /dev/sda1       15248800 1072896  14175904   8% /boot 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: devtmpfs            8192       0      8192   0% /dev 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: tmpfs           32958456       0  32958456   0% /dev/shm 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: tmpfs             131072     320    130752   1% /var/log 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: tmpfs               1024       0      1024   0% /mnt/disks 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: tmpfs               1024       0      1024   0% /mnt/remotes 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: tmpfs               1024       0      1024   0% /mnt/addons 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: tmpfs               1024       0      1024   0% /mnt/rootshare 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: Low memory detected, aborting ... 
Dec 07 19:17:39 preclear_disk_ZR5A2ZA6_13445: Pre-read: pre-read verification failed! 
Dec 07 19:17:40 preclear_disk_ZR5A2ZA6_13445: ssmtp: Authorization failed (535 5.7.8  https://support.google.com/mail/?p=BadCredentials mm4-20020a0562145e8400b00679d7e76b64sm359241qvb.126 - gsmtp) 
Dec 07 19:17:41 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: Error: 
Dec 07 19:17:41 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: 
Dec 07 19:17:41 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: ATTRIBUTE               INITIAL NOW STATUS 
Dec 07 19:17:41 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: Reallocated_Sector_Ct   0       0   - 
Dec 07 19:17:42 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: Power_On_Hours          6       6   - 
Dec 07 19:17:42 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: Reported_Uncorrect      0       0   - 
Dec 07 19:17:42 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: Airflow_Temperature_Cel 29      29  - 
Dec 07 19:17:42 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: Current_Pending_Sector  0       0   - 
Dec 07 19:17:42 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: Offline_Uncorrectable   0       0   - 
Dec 07 19:17:42 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.: UDMA_CRC_Error_Count    0       0   - 
Dec 07 19:17:42 preclear_disk_ZR5A2ZA6_13445: S.M.A.R.T.:  
Dec 07 19:17:42 preclear_disk_ZR5A2ZA6_13445: error encountered, exiting ... 

 

 

tower-diagnostics-20231207-1948.zip

[dlandon]

7 minutes ago, Provocateur said:

My new disk is failing during the pre-read. Have precleared disks in the past without issue (though, not within the last couple of years...). Unraid dashboard shows 50+ GB of free memory. Any ideas? Cheers.

Go the the Boot->Flash->Syslinux Configuration and remove any "root=sda" you see on any of the boot options, then reboot.  Try preclear again.  It should work.

[Provocateur]

26 minutes ago, dlandon said:

Go the the Boot->Flash->Syslinux Configuration and remove any "root=sda" you see on any of the boot options, then reboot.  Try preclear again.  It should work.

 

Pre-read is up and running! Thanks much.

[ConnerVT]

I hope I am posting this in the right thread.

 

I am currently running preclear on two 16TB drives.  Usually I don't run preclear, but bought these from an eBay vendor I had not used before.  My logs are being spammed by these spin down / read SMART messages every 15 minutes (x2 as two drives pre-clearing).  Looks as it kicked in at Step 5 - Post-read.  See screenshot.  15 minutes is my default spin down delay.  Should that apply to unmounted, unformatted UD drives as well?

 

 

That would align with when my system began do display the highlighted below:

 

 

This is probably not the behavior that would be expected.  The preclear is almost complete, no errors so far.

malta-tower-diagnostics-20231208-0604.zip

[dlandon]

2 hours ago, ConnerVT said:

Should that apply to unmounted, unformatted UD drives as well?

Unraid does not try to spin down disks that have no partition.  Once the partition is created on the disk, Unraid will try to spin down the disk regardless of whether or not there is a file system on the disk.  It shouldn't cause any issues, just logs spin down messages.

[ConnerVT]

Understood.  Other than the log entries, everything ran fine.  Just thought I'd mention it, in case it was some unknown behavior.

[Hudson2023]

Should I actually uninstall the NerdPack Tmux package before installing this UD Preclear plugin?

[itimpi]

7 hours ago, Hudson2023 said:

Should I actually uninstall the NerdPack Tmux package before installing this UD Preclear plugin?

What version of Unraid are you running?   Nerdpack is incompatible with recent Unraid versions.

[linksta7]

Sorry if this is a noob question, but with a new disk in a new build, which preclear option should I be choosing? I assume "Erase and Clear Disk" since I want it zeroed by erasing and signed by clearing, but the definition "disk to be discard" in the original post is throwing me off as I'm not exactly sure what that means. Thank you!

[dlandon]

7 hours ago, linksta7 said:

Sorry if this is a noob question, but with a new disk in a new build, which preclear option should I be choosing?

"Clear DIsk" for a disk to be added to the array.

 

7 hours ago, linksta7 said:

"Erase and Clear Disk"

This is used to prepare a disk for disposal because it destroys all data on the disk.

[linksta7]

3 hours ago, dlandon said:

"Clear DIsk" for a disk to be added to the array.

 

This is used to prepare a disk for disposal because it destroys all data on the disk.

 

Makes sense, thank you Last night, I already began the erase and clear disk process before getting this reply. The disks are now on the erase step, is it safe to just stop the process and re-run a new clear-only run? Or should I just let it do its thing and complete the entire erase and clear process? As well, do I need to explicitly format my drives after the preclear process is complete or will unraid format them for me when I build the array for the first time?

[itimpi]

8 minutes ago, linksta7 said:

As well, do I need to explicitly format my drives after the preclear process is complete

You must NOT do this as it destroys the pre-clear signature.

[dlandon]

11 minutes ago, linksta7 said:

 As well, do I need to explicitly format my drives after the preclear process is complete

No.

 

11 minutes ago, linksta7 said:

or will unraid format them for me when I build the array for the first time?

Yes, but you need to use the "Clear Disk" option so the pre-clear signature is written to the disk.  When Unraid sees the signature, it will not clear the disk which has already been done by the pre-clear plugiun.

 

The normal steps are:

• Pre-clear the disk using "Clear Disk".
• Assign the disk to the Array.
• Start Array and you will be asked to confirm formatting the disk to the file format you've specified as either the default or changed for this particular disk.