Unassigned Devices Preclear - a utility to preclear disks before adding them to the array

[CyberMew]

Thanks, I verified disk and it says disk not zeroed out. Not sure how it even cleared Stage 4 if so..

 

I am now clearing the disk again starting with pre-read

[CyberMew]

Failing again. Feels like something is not right. Doesn’t seem like an issue with the disk? Any idea what’s the issue and how to resolve it?
 

#################################################################################################### # Unraid Server Preclear of disk xxx # # Cycle 1 of 1, partition start on sector 64. # # # # Step 1 of 5 - Pre-read verification: [23:22:47 @ 213 MB/s] SUCCESS # # Step 2 of 5 - Zeroing the disk: [23:23:33 @ 213 MB/s] SUCCESS # # Step 3 of 5 - Writing Unraid's Preclear signature: SUCCESS # # Step 4 of 5 - Verifying Unraid's Preclear signature: SUCCESS # # Step 5 of 5 - Post-Read verification: FAIL # # # # # # # #################################################################################################### # Cycle elapsed time: 47:11:04 | Total elapsed time: 47:11:05 # #################################################################################################### #################################################################################################### # S.M.A.R.T. Status (device type: default) # # # # ATTRIBUTE INITIAL STATUS # # Reallocated_Sector_Ct 0 - # # Power_On_Hours 94 - # # Temperature_Celsius 36 - # # Reallocated_Event_Count 0 - # # Current_Pending_Sector 0 - # # Offline_Uncorrectable 0 - # # UDMA_CRC_Error_Count 0 - # # # # # #################################################################################################### # # ####################################################################################################

 

…

May 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 408604901376 bytes (409 GB, 381 GiB) copied, 1449.55 s, 282 MB/sMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 196567+0 records inMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 196566+0 records outMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 412228780032 bytes (412 GB, 384 GiB) copied, 1462.4 s, 282 MB/sMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 198295+0 records inMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 198294+0 records outMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 415852658688 bytes (416 GB, 387 GiB) copied, 1475.27 s, 282 MB/sMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: post-read verification failed!May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Error:May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.:May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: ATTRIBUTE               INITIAL NOW STATUSMay 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Reallocated_Sector_Ct   0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Power_On_Hours          94      141 Up 47May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Temperature_Celsius     36      51  Up 15May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Reallocated_Event_Count 0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Current_Pending_Sector  0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Offline_Uncorrectable   0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: UDMA_CRC_Error_Count    0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: May 26 22:33:25 preclear_disk_xxx_2299: error encountered, exiting ...

[dlandon]

1 hour ago, CyberMew said:

Failing again. Feels like something is not right. Doesn’t seem like an issue with the disk? Any idea what’s the issue and how to resolve it?
 

#################################################################################################### # Unraid Server Preclear of disk xxx # # Cycle 1 of 1, partition start on sector 64. # # # # Step 1 of 5 - Pre-read verification: [23:22:47 @ 213 MB/s] SUCCESS # # Step 2 of 5 - Zeroing the disk: [23:23:33 @ 213 MB/s] SUCCESS # # Step 3 of 5 - Writing Unraid's Preclear signature: SUCCESS # # Step 4 of 5 - Verifying Unraid's Preclear signature: SUCCESS # # Step 5 of 5 - Post-Read verification: FAIL # # # # # # # #################################################################################################### # Cycle elapsed time: 47:11:04 | Total elapsed time: 47:11:05 # #################################################################################################### #################################################################################################### # S.M.A.R.T. Status (device type: default) # # # # ATTRIBUTE INITIAL STATUS # # Reallocated_Sector_Ct 0 - # # Power_On_Hours 94 - # # Temperature_Celsius 36 - # # Reallocated_Event_Count 0 - # # Current_Pending_Sector 0 - # # Offline_Uncorrectable 0 - # # UDMA_CRC_Error_Count 0 - # # # # # #################################################################################################### # # ####################################################################################################

 

…

May 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 408604901376 bytes (409 GB, 381 GiB) copied, 1449.55 s, 282 MB/sMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 196567+0 records inMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 196566+0 records outMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 412228780032 bytes (412 GB, 384 GiB) copied, 1462.4 s, 282 MB/sMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 198295+0 records inMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 198294+0 records outMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: dd output: 415852658688 bytes (416 GB, 387 GiB) copied, 1475.27 s, 282 MB/sMay 26 22:33:20 preclear_disk_xxx_2299: Post-Read: post-read verification failed!May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Error:May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.:May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: ATTRIBUTE               INITIAL NOW STATUSMay 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Reallocated_Sector_Ct   0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Power_On_Hours          94      141 Up 47May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Temperature_Celsius     36      51  Up 15May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Reallocated_Event_Count 0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Current_Pending_Sector  0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: Offline_Uncorrectable   0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: UDMA_CRC_Error_Count    0       0   -May 26 22:33:25 preclear_disk_xxx_2299: S.M.A.R.T.: May 26 22:33:25 preclear_disk_xxx_2299: error encountered, exiting ...

Try the following:

• Post your diagnostics and the UD Preclear log so I can see if there is a reason I can find for the failure.
• Install the Binhex Preclear Docker container.
• Run UD preclear and select the docker from the dropdown to run the preclear.  See if the results are the same.

[CyberMew]

My array is currently stopped due to disabled drive and this new drive is supposed to replace that using the drive swap procedure, so unfortunately I can’t start and install the docker and run it.. is there another way to do it?

 

I’ll post the unraid diagnostics once I am able to. How do I get the full UD Preclear log? 

[dlandon]

6 minutes ago, CyberMew said:

My array is currently stopped due to disabled drive and this new drive is supposed to replace that using the drive swap procedure

That drive should be emulated, assuming you have parity, and you can start the array.

 

8 minutes ago, CyberMew said:

I’ll post the unraid diagnostics once I am able to. How do I get the full UD Preclear log? 

Use 'diagnostics' on a command line to get diagnottics.  Preclear log can be downloaded from the UD Preclear page.  Click on the download icon.

[CyberMew]

7 hours ago, dlandon said:

That drive should be emulated, assuming you have parity, and you can start the array.

 

Use 'diagnostics' on a command line to get diagnottics.  Preclear log can be downloaded from the UD Preclear page.  Click on the download icon.

My disk 5 is unassigned right now with a red X, will it know to emulate disk 5 or remove it and update my parity (losing data)?

 

Also have uploaded the requested logs here:

TOWER-unassigned.devices.preclear-20230527-0806.zip tower-diagnostics-20230527-0048.zip

[dlandon]

11 hours ago, CyberMew said:

My disk 5 is unassigned right now with a red X, will it know to emulate disk 5 or remove it and update my parity (losing data)?

I'm not sure what you've done, but try to start the array and the disk should be emulated.

[trurl]

On 5/23/2023 at 10:38 AM, CyberMew said:

#   Step 5 of 5 - Post-Read verification:                                          FAIL            #

Could be bad RAM

[CyberMew]

I tried to preclear using the binhex docker as suggested, and it precleared successfully.

 

So maybe there is some bug with the plugin..? Should I stop preclear and Verify Disk using this plugin, or is the drive good to go now?

 

Quote

================================================================== 1.22
=                unRAID server Pre-Clear disk /dev/sdk
=               cycle 1 of 1, partition start on sector 64
= Disk Pre-Clear-Read completed                                 DONE
= Step 1 of 10 - Copying zeros to first 2048k bytes             DONE
= Step 2 of 10 - Copying zeros to remainder of disk to clear it DONE
= Step 3 of 10 - Disk is now cleared from MBR onward.           DONE
= Step 4 of 10 - Clearing MBR bytes for partition 2,3 & 4       DONE
= Step 5 of 10 - Clearing MBR code area                         DONE
= Step 6 of 10 - Setting MBR signature bytes                    DONE
= Step 7 of 10 - Setting partition 1 to precleared state        DONE
= Step 8 of 10 - Notifying kernel we changed the partitioning   DONE
= Step 9 of 10 - Creating the /dev/disk/by* entries             DONE
= Step 10 of 10 - Verifying if the MBR is cleared.              DONE
= Disk Post-Clear-Read completed                                DONE
Disk Temperature: 56C, Elapsed Time:  101:02:38
========================================================================1.22
== WDCWD181KFGX-68AFPN0  
== Disk /dev/sdk has been successfully precleared
== with a starting sector of 64
============================================================================
** Changed attributes in files: /tmp/smart_start_sdk  /tmp/smart_finish_sdk
                ATTRIBUTE   NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS      RAW_VALUE
      Temperature_Celsius =    35      53            0        ok          56
 No SMART attributes are FAILING_NOW

 0 sectors were pending re-allocation before the start of the preclear.
 0 sectors were pending re-allocation after pre-read in cycle 1 of 1.
 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1.
 0 sectors are pending re-allocation at the end of the preclear,
    the number of sectors pending re-allocation did not change.
 0 sectors had been re-allocated before the start of the preclear.
 0 sectors are re-allocated at the end of the preclear,
    the number of sectors re-allocated did not change.

 

[dlandon]

8 hours ago, CyberMew said:

So maybe there is some bug with the plugin..?

There have been several reports of post read issues on larger disks.  I haven't been able to get enough information to figure out what this might be.  That's why I suggested using the docker for your preclear.

 

8 hours ago, CyberMew said:

Should I stop preclear and Verify Disk using this plugin, or is the drive good to go now?

The disk should be good to go.

[CyberMew]

5 minutes ago, dlandon said:

There have been several reports of post read issues on larger disks.  I haven't been able to get enough information to figure out what this might be.  That's why I suggested using the docker for your preclear.

 

The disk should be good to go.

Thanks for confirming! Hopefully my logs were helpful in helping you figure it out..

[comet424]

not sure what a sweetalert2 is  but i got it.. i have gotten a couple different preclear errors before when i was working with the UD plugin..  i not sure if its due to i doing 3 14TB  drives at once..  if its hogging the cpu  a i did get 500 error trying to access website  so  not sure  figured id report it before i reboot and try a file to fix my UD warning error i getting

tardis-diagnostics-20230602-0856.zip

[dlandon]

9 minutes ago, comet424 said:

not sure what a sweetalert2

Sweetalert2 is used for the dialogs used in UD and UD Preclear.

[comet424]

ah ok.. do you happen to know what caused it?  from my diag   is it from  running 3 Preclears at once  that it could be cpu over load..  i was thinking of cpu pinning  so  the first cpu be just for unraid   not sure

[dlandon]

I think the GUI timed out because the processor was so busy.

[comet424]

ah ok  i know it doesnt look like the cpu is bogged down  least not 100% utlizied  but cant afford the more expensive cpus  but probably better to use my backupserver later.. it doesnt nothing just waits for copies least be less cpu load yet its only a 2 core cpu for now

 

but i appreciate your plugin  it helps to make sure these hard drives are good 🙂 

[dlandon]

1 minute ago, comet424 said:

ah ok  i know it doesnt look like the cpu is bogged down  least not 100% utlizied  but cant afford the more expensive cpus  but probably better to use my backupserver later.. it doesnt nothing just waits for copies least be less cpu load yet its only a 2 core cpu for now

 

but i appreciate your plugin  it helps to make sure these hard drives are good 🙂 

Doing three preclears at once could be a bit much for a less powerful processor.

[comet424]

ya thats what i thinking  i know i did 2  on the 2 core cpu but i didnt touch it for a week..  so it didnt have an issue

 

my main is 

AMD Ryzen 7 3700X 8-Core @ 3600 MHz

 

and sure runs other crap lol...

 

i do enjoy the resume feature... the old preclear i used to use a couple years ago i couldnt do a reboot to test other things  and then you gotta start from scratch... so love this version  🙂

 

[comet424]

my one 14tb drive picked up some read errors in the system log a few hours ago..  after my last post .,... does it say what caused it...  like there is no errors on the drive showing and this is the 3rd run  of the preclear  on step 1... just wanna see what caused it possibly

 

 

tardis-diagnostics-20230602-1502.zip

[dlandon]

Check your cables:

Jun  2 11:43:56 Tardis kernel: sd 11:0:7:0: [sdk] tag#3115 UNKNOWN(0x2003) Result: hostbyte=0x0b driverbyte=DRIVER_OK cmd_age=0s
Jun  2 11:43:56 Tardis kernel: sd 11:0:7:0: [sdk] tag#3115 CDB: opcode=0x88 88 00 00 00 00 00 8b c8 1e 08 00 00 05 70 00 00
Jun  2 11:43:56 Tardis kernel: I/O error, dev sdk, sector 2345147912 op 0x0:(READ) flags 0x84700 phys_seg 128 prio class 0
Jun  2 11:43:56 Tardis kernel: sd 11:0:7:0: [sdk] tag#3121 UNKNOWN(0x2003) Result: hostbyte=0x0b driverbyte=DRIVER_OK cmd_age=0s
Jun  2 11:43:56 Tardis kernel: sd 11:0:7:0: [sdk] tag#3121 CDB: opcode=0x88 88 00 00 00 00 00 8b c8 28 00 00 00 02 00 00 00
Jun  2 11:43:56 Tardis kernel: I/O error, dev sdk, sector 2345150464 op 0x0:(READ) flags 0x80700 phys_seg 64 prio class 0
Jun  2 11:43:56 Tardis kernel: sd 11:0:7:0: [sdk] tag#3117 UNKNOWN(0x2003) Result: hostbyte=0x0b driverbyte=DRIVER_OK cmd_age=0s
Jun  2 11:43:56 Tardis kernel: sd 11:0:7:0: [sdk] tag#3117 CDB: opcode=0x88 88 00 00 00 00 00 8b c8 24 00 00 00 04 00 00 00
Jun  2 11:43:56 Tardis kernel: I/O error, dev sdk, sector 2345149440 op 0x0:(READ) flags 0x84700 phys_seg 128 prio class 0
Jun  2 11:43:56 Tardis kernel: sd 11:0:7:0: [sdk] tag#3116 UNKNOWN(0x2003) Result: hostbyte=0x0b driverbyte=DRIVER_OK cmd_age=0s
Jun  2 11:43:56 Tardis kernel: sd 11:0:7:0: [sdk] tag#3116 CDB: opcode=0x88 88 00 00 00 00 00 8b c8 23 78 00 00 00 88 00 00
Jun  2 11:43:56 Tardis kernel: I/O error, dev sdk, sector 2345149304 op 0x0:(READ) flags 0x80700 phys_seg 17 prio class 0
Jun  2 11:43:57 Tardis kernel: sd 11:0:7:0: Power-on or device reset occurred

 

[comet424]

ok i looked nothings moved...  they go from sas expander card to cable that splits to 4 satas into  backplane...  everything looks tight and nothing moves in there

 

and a power on or device reset  what does that mean that the hard drive turned off and turned back on...    or it reset something?

[comet424]

ive seen this a few times  i get that sweetalart  but i get this when i downloading from the apps page  a new app

why is the preclear linked to downloading a new app?  is it something i need to fix?

 

tardis-diagnostics-20230606-1847.zip

[comet424]

why did i get another pre clear error?  i not even running it now  just parity building its its going extremly slow i dont know why

 

but i get this error 

Jun 7 11:40:05 Tardis nginx: 2023/06/07 11:40:05 [error] 11505#11505: *753955 auth request unexpected status: 504 while sending to client, client: 192.168.0.227, server: , request: "POST /plugins/unassigned.devices.preclear/Preclear.php HTTP/1.1", host: "192.168.0.3", referrer: "http://192.168.0.3/Docker"

 

 

tardis-diagnostics-20230607-1459.zip

[dlandon]

That's an nginx error, not a preclear error.

[comet424]

ah well it says preclear  

Jun 7 11:40:05 Tardis nginx: 2023/06/07 11:40:05 [error] 11505#11505: *753955 auth request unexpected status: 504 while sending to client, client: 192.168.0.227, server: , request: "POST /plugins/unassigned.devices.preclear/Preclear.php HTTP/1.1", host: "192.168.0.3", referrer: "http://192.168.0.3/Docker"

 

if its nginx  why is iit posting preclear?  is my question