Unassigned Devices Preclear - a utility to preclear disks before adding them to the array

[T0a]

I just tested this plugin with a new disk attached via USB to my Unraid server (6.9.2). I started the preclear process and paused it when the status reached 65%. Unfortunately, no "*.resume" was created on "/boot":

 

May 02 17:49:33 preclear_disk_WD-WX92DA1DAR18_12570: Preclear Disk Version: 1.0.25
May 02 17:49:33 preclear_disk_WD-WX92DA1DAR18_12570: Disk size: 4000787030016
May 02 17:49:33 preclear_disk_WD-WX92DA1DAR18_12570: Disk blocks: 976754646
May 02 17:49:33 preclear_disk_WD-WX92DA1DAR18_12570: Blocks (512 bytes): 7814037168
May 02 17:49:33 preclear_disk_WD-WX92DA1DAR18_12570: Block size: 4096
May 02 17:49:33 preclear_disk_WD-WX92DA1DAR18_12570: Start sector: 0
May 02 17:49:36 preclear_disk_WD-WX92DA1DAR18_12570: Zeroing: zeroing the disk started (1/5) ...
May 02 17:49:36 preclear_disk_WD-WX92DA1DAR18_12570: Zeroing: emptying the MBR.
May 02 19:13:16 preclear_disk_WD-WX92DA1DAR18_12570: Zeroing: progress - 25% zeroed @ 192 MB/s
May 02 20:45:26 preclear_disk_WD-WX92DA1DAR18_12570: Zeroing: progress - 50% zeroed @ 167 MB/s
May 02 21:49:18 preclear_disk_WD-WX92DA1DAR18_12570: Pause requested
May 02 21:49:18 preclear_disk_WD-WX92DA1DAR18_12570: cp: cannot create regular file '/boot/preclear_reports/WD-WX92DA1DAR18.resume': No such file or directory
May 02 21:49:18 preclear_disk_WD-WX92DA1DAR18_12570: Paused

 

After manually creating the folder "/boot/preclear_reports", starting the preclear process and pausing it again the file "WD-WX92DA1DAR18.resume" gets written to disk as expected:

 

root@server:/boot# ls /boot/preclear_reports/
WD-WX92DA1DAR18.resume
root@server:/boot# cat /boot/preclear_reports/WD-WX92DA1DAR18.resume 
# parsed arguments
verify_disk_mbr=''
erase_preclear='n'
short_test=''
read_stress='y'
erase_disk='n'
notify_freq='1'
format_html=''
verify_zeroed=''
write_disk_mbr=''
write_size=''
skip_preread='y'
read_size=''
notify_channel='4'
no_prompt='y'
cycles='1'
skip_postread=''
read_blocks=''

# current operation
current_op='zero'
current_pos='2621668589568'
current_timer='14431'
current_cycle='1'

# previous operations
preread_average=''
preread_speed=''
write_average=''
write_speed=''
postread_average=''
postread_speed=''

# current elapsed time
main_elapsed_time='14442'
cycle_elapsed_time='14441'

 

Edited May 2, 2022 by T0a

[dlandon]

15 hours ago, T0a said:

After manually creating the folder "/boot/preclear_reports", starting the preclear process and pausing it again the file "WD-WX92DA1DAR18.resume" gets written to disk as expected:

Fixed in the next release.

[T0a]

5 hours ago, dlandon said:

Fixed in the next release.

 

Thanks @dlandon. I cannot proceed my preclear session from yesterday after a server shutdown. Any ideas? Is this expected behavior? Are there any workarounds? I don't want to start the preclear from the beginning. 

 

Update: My mistake. You need to click "start preclear" and then the plugins asks to resume. I initially though "start preclear" will trigger a new preclear run. I expected the UI to show my paused session in the devices table.

 

Log:

May 3 16:22:27 Preclear resumed on all devices.

 

Edited May 3, 2022 by T0a

[dlandon]

2 hours ago, T0a said:

 

Update: My mistake. You need to click "start preclear" and then the plugins asks to resume.

Correct.  After a reboot the disk is not paused, but you can resume where you left off.

[mikela]

I have 8TB drives which take quite awhile to preclear.  I need to be able to shutdown my server nightly.  Is my understanding correct that I can shut down the server while Preclear is running and that upon start up it will resume?

[dlandon]

1 minute ago, mikela said:

I have 8TB drives which take quite awhile to preclear.  I need to be able to shutdown my server nightly.  Is my understanding correct that I can shut down the server while Preclear is running and that upon start up it will resume?

Yes, but you need to manually resume the preclear.  When you start the preclear, you will see a choice to resume preclear where it left off.

[srirams]

I successfully did a preclear on a new disk, but when I try to add it the array I'm getting this error "Unmountable: Unsupported partition layout"?

[dlandon]

18 hours ago, srirams said:

I successfully did a preclear on a new disk, but when I try to add it the array I'm getting this error "Unmountable: Unsupported partition layout"?

Go to the UD preclear page and download the logs and post the zip file.

[srirams]

2 hours ago, dlandon said:

Go to the UD preclear page and download the logs and post the zip file.

 

I've attached it below!

TRANTOR-unassigned.devices.preclear-20220603-1056.zip

[dlandon]

6 minutes ago, srirams said:

I've attached it below!

Looks good.  It won't be mountable until Unraid formats it.

[srirams]

That worked, thanks! Didn't realize that I had to format it in unraid after preclear.

[dlandon]

Preclear prepares it for inclusion in the array, but Unraid has to format it to your desired file format.

[mikela]

I was able to get my new 8TB drive precleared over the course of several days while stopping in the evenings and then resuming.  However, the "Mount" button was greyed out so I stopped the array and assigned the drive to the array.  I selected for the drive to be formatted but it seems that unraid wants to rebuild so I stopped everything for the moment and ran a diagnostic.  Thoughts?

 

tower-diagnostics-20220606-1054.zip

 

Edited June 6, 2022 by mikela

[itimpi]

Looking at the diagnostics I think you started the array with the new disk9, and later stopped the array and unassigned it and restarted the array?    This would cause Unraid to start ‘emulating’ disk9, so that when you later added it back in it wanted to rebuild it because it was now assuming that you were in a failed disk scenario.   It then looked as though you formatted the emulated disk, and Unraid is now attempting to rebuild an empty file system on the physical disk9.

[mikela]

56 minutes ago, itimpi said:

Looking at the diagnostics I think you started the array with the new disk9, and later stopped the array and unassigned it and restarted the array?    This would cause Unraid to start ‘emulating’ disk9, so that when you later added it back in it wanted to rebuild it because it was now assuming that you were in a failed disk scenario.   It then looked as though you formatted the emulated disk, and Unraid is now attempting to rebuild an empty file system on the physical disk9.

 

I believe you are right.  Definitely operator error on my part.

 

Is there any way to recover?

 

Should I just let it rebuild?

Edited June 6, 2022 by mikela

[itimpi]

1 hour ago, mikela said:

Should I just let it rebuild?

At this point that would be the easiest and fastest way to get back to a consistent state.

[Creativity404]

Goodday everyone,

 

I'm adding 2, 12 TB  drives to my system. 1 precleared succesfully the other one did not.
In the log it says:

Y5JB79JC_19045: Post-Read: cmp command failed - disk not zeroed
Y5JB79JC_19045: Post-Read: dd command failed, exit code [141]

Would anybody have some advice for me? It would be greatly appreciated.
Is the drive faulty? should i run another preclear?

 

APOLLO-unassigned.devices.preclear-20220607-0200.zip

Edited June 7, 2022 by Creativity404
Added logs

[mgadbois]

Running Unraid 6.10.2 and Preclear plugin version.22.6.10

Adding a 10th drive to my system. A shucked Easystore 14tb. All my other dives are the same. Put shucked drive in bay and I see it in unassigned Devices, but there is no Preclear option. In preclear under tools the drive does not show up. Any thought as how to get this thing precleared so I can add it to my array. All the other easystore drives precleared(earlier version of Unraid) just by putting them in a drive bay (24 bay super micro server) and going to preclear plugin. Screenshot of unassigned drives.

 

Thanks in advance.

astro-server-diagnostics-20220610-2011.zip

Edited June 11, 2022 by mgadbois
added pic

[dlandon]

8 minutes ago, mgadbois said:

Running Unraid 6.10.2 and Preclear plugin version.22.6.10

Adding a 10th drive to my system. A shucked Easystore 14tb. All my other dives are the same. Put shucked drive in bay and I see it in unassigned Devices, but there is no Preclear option. In preclear under tools the drive does not show up. Any thought as how to get this thing precleared so I can add it to my array. All the other easystore drives precleared(earlier version of Unraid) just by putting them in a drive bay (24 bay super micro server) and going to preclear plugin. Screenshot of unassigned drives.

 

Thanks in advance.

astro-server-diagnostics-20220610-2011.zip 167.31 kB · 0 downloads

You have to remove the partition before UD Preclear will allow you to preclear the disk.  Click on the red X next to the drive serial number and clear the disk.  It will then show up as being able to preclear.

[mgadbois]

Well that was easy. I can't believe I missed that. It is now running the preclear process.

Thanks for the help and the fast response.

[mikela]

I did a preclear over the course of several days shutting down at night.  It successfully completed.  However, the "Mount" button was greyed out so I deleted the preclear log per the instructions and still am not able to mount.

 

 

Do I need to stop the array and press the red x to clear the disk?

Edited June 13, 2022 by mikela

[dlandon]

7 minutes ago, mikela said:

I did a preclear over the course of several days shutting down at night.  It successfully completed.  However, the "Mount" button was greyed out so I deleted the preclear log per the instructions and still am not able to mount.

 

Click on the red X next to the disk serial to clear the disk and then format the disk.

[mikela]

15 minutes ago, dlandon said:

Click on the red X next to the disk serial to clear the disk and then format the disk.

 

I started to do that but read the following warning:

 

"If this disk was pre-cleared, you will remove the pre-clear signature. If you add the disk to the array and the pre-clear signature is removed, Unraid will go through the lengthy process of clearing the disk again."

[dlandon]

2 minutes ago, mikela said:

 

I started to do that but read the following warning:

 

"If this disk was pre-cleared, you will remove the pre-clear signature. If you add the disk to the array and the pre-clear signature is removed, Unraid will go through the lengthy process of clearing the disk again."

If you're going to put it into the array, don't clear the disk.  Unraid will format it.  Preclearing creates a partition, but does not create a file system.  That's why it is not mountable.

[mikela]

4 minutes ago, dlandon said:

If you're going to put it into the array, don't clear the disk.  Unraid will format it.  Preclearing creates a partition, but does not create a file system.  That's why it is not mountable.

 

So just stop the array and put it in position 10 and unraid will recognize the preclear signature and format it ?