Preclear plugin


Recommended Posts

1 hour ago, itimpi said:

Encryption is irrelevant to this as parity runs at the physical sector level.   Unraid will go straight into clearing a disk when you add it any time it does not recognise a ‘clear’ signature as being present on the drive.

I wonder why it didn't recognize the correct signature then.

Link to comment
  • 2 weeks later...

Any help would be appreciated. 

 

I am running in to PreClear errors... without any actual errors showing, except a warning? In the MAIN tab I see an error stating: 

 

"Error encountered, please verify the log"

 

The log: 

 

Dec 10 08:15:08 Tower kernel: sd 1:0:0:0: [sdb] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB)
Dec 10 08:15:08 Tower kernel: sd 1:0:0:0: [sdb] 4096-byte physical blocks
Dec 10 08:15:08 Tower kernel: sd 1:0:0:0: [sdb] Write Protect is off
Dec 10 08:15:08 Tower kernel: sd 1:0:0:0: [sdb] Mode Sense: 7f 00 10 08
Dec 10 08:15:08 Tower kernel: sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
Dec 10 08:15:08 Tower kernel: sd 1:0:0:0: [sdb] Attached SCSI disk
Dec 10 08:15:32 Tower emhttpd: WD-XXXXXXX
Dec 10 08:25:18 Tower preclear_disk_WD-XXXXXXX: Command: /usr/local/emhttp/plugins/preclear.disk/script/preclear_disk.sh --notify 2 --frequency 1 --cycles 1 --no-prompt /dev/sdb
Dec 10 08:25:23 Tower preclear_disk_WD-XXXXXXX: Pre-Read: dd if=/dev/sdb of=/dev/null bs=2097152 skip=0 count=6001175126016 conv=notrunc,noerror iflag=nocache,count_bytes,skip_bytes
Dec 10 08:30:31 Tower preclear.disk: Pausing preclear of disk 'sdb'
Dec 10 08:30:51 Tower preclear.disk: Resuming preclear of disk 'sdb'
Dec 10 12:40:07 Tower preclear_disk_WD-XXXXXXX: Zeroing: dd if=/dev/zero of=/dev/sdb bs=2097152 seek=2097152 count=6001173028864 conv=notrunc iflag=count_bytes,nocache,fullblock oflag=seek_bytes

 

 

And the Preview shows the following:

 

unRAID Server Preclear of disk WD-XXXXXXX

Cycle 1 of 1, partition start on sector 64. 

Step 1 of 5 - Pre-read verification: [4:14:25 @ 393 MB/s] SUCCESS

Step 2 of 5 - Zeroing in progress: (99% Done)

Time elapsed: 11:22:38 | Write speed: 86 MB/s | Average speed: 146 MB/s

Cycle elapsed time: 15:37:07 | Total elapsed time: 15:37:08

##### # # #

S.M.A.R.T. Status (device type: default)

ATTRIBUTE INITIAL STATUS 

5-Reallocated_Sector_Ct 0 - 

9-Power_On_Hours 0 - 

194-Temperature_Celsius 25 -

196-Reallocated_Event_Count 0 - 

197-Current_Pending_Sector 0 -

198-Offline_Uncorrectable 0 -

199-UDMA_CRC_Error_Count 0 - 

SMART overall-health self-assessment test result: PASSED 

 

It looks to be running, but no time indicator anymore. I still have the red X option to "Stop Preclear" 

 

Anyone have any thoughts on this before I hit the X in error when I should be letting it do it's thing? 

 

At the bottom of my window it does state: "Read Check 46%" but I don't know if that is to do with the Preclear. I think this is to do with the Array Operation, not this disk itself since the disk size is 6TB and it is currently at: 

10 TB

Elapsed time:11 hours, 1 minute

Current position:4.82 TB (48.2 %)

Estimated speed:105.7 MB/sec

Estimated finish:13 hours, 38 minutes

 

I assume this is the Parity drive, which is a 10TB drive. 

Edited by NLDer
Link to comment
17 minutes ago, NLDer said:

At the bottom of my window it does state: "Read Check 46%" but I don't know if that is to do with the Preclear. I think this is to do with the Array Operation,

This the array operation. The fact it is a "Read Check" implies that you have at least as many drives marked as disabled as you have parity drives.  Is this actually the case?

Link to comment
1 minute ago, itimpi said:

This the array operation. The fact it is a "Read Check" implies that you have at least as many drives marked as disabled as you have parity drives.  Is this actually the case?

I have one drive I am replacing at this time. ANd I have only 1 Parity disk. 

Link to comment
2 hours ago, NLDer said:

What I am unsure of at this time, is the Preclear still running, or not? 

There is no "duration" information shown on the Main page any more. However the red X to sop Preclear is still available. 

 

#AmConfused


image.png.a042362e4a20541c3791ddc4b4384371.png

Looks like you are using an old version of UD.  You should update it.

Link to comment
On 12/11/2020 at 11:39 AM, NLDer said:

What I am unsure of at this time, is the Preclear still running, or not? 

There is no "duration" information shown on the Main page any more. However the red X to sop Preclear is still available. 

 

#AmConfused


image.png.a042362e4a20541c3791ddc4b4384371.png

 

 

Sometimes, when I get a bad array drive, I take the chance and test some safety mesures I put on the code, like webgui lock prevention, prevent filling up the rootfs etc. This time I tried to intercept some errors from the code and export them out to the logs, but there are some commands which normally exites in errors, like the <cmp> compare program we use to verify the zeroing of the disk that exits with an error simply because it reaches the end of file.

 

So this was a regression introduced recently in the script, and it's fixed now. Thanks for reporting!

Link to comment
On 12/11/2020 at 2:39 PM, NLDer said:

What I am unsure of at this time, is the Preclear still running, or not? 

There is no "duration" information shown on the Main page any more. However the red X to sop Preclear is still available. 

 

#AmConfused


image.png.a042362e4a20541c3791ddc4b4384371.png

 

Today I had the exact same thing happen on my new shucked 12TB WD drive. It reached 99% writing zero then halted with an error. I noticed both Preclear Disk & Unassigned Devices plugins had updates available - I've now updated both and started a new pre-clear using the default script. Fingers crossed!

Link to comment
1 hour ago, eric90000 said:

 

Today I had the exact same thing happen on my new shucked 12TB WD drive. It reached 99% writing zero then halted with an error. I noticed both Preclear Disk & Unassigned Devices plugins had updates available - I've now updated both and started a new pre-clear using the default script. Fingers crossed!

Yep, that's the bug I fixed yesterday.

  • Like 1
Link to comment
48 minutes ago, gfjardim said:

Yep, that's the bug I fixed yesterday.

THanks for fixing... am running it again on a different drive. The one from above I simply let unRaid rebuild it and take a chance. This second drive is now running pre-clear. Will update to let you know how it goes.... 

 

Thank you gfjardim

Link to comment

Hi,

 

am i blind?

I cant see the benefits of this Plugin anymore.

 

It sounds so great at the beginning so i downloaded it. (I read that the array is offline while unraid-clearing is in progress)

So i started a pre-clearing of a 4TB sshd - failed at 73% (Default script (gfjardim i think) enabled with pre- and post read)

Needed Time == 11h

 

Next is startet the same like above - failed at 99%

Needed Time == 10h

 

I thought maybe a script error because of sshd.

So i startet again with the second included Script (cant look for the name at the moment)

(pre- and postread disabled)

Needed Time ==11h -> no errors but post read startet with estimated time of 9h

 

Now i added the disk to my array and startet it (clearing per UnRaid)

Estimated Time == 7h

Array is online and working.

 

Ah before i forgot: no errors in the logs in all of the jobruns...

 

Please can somone explain me the purpose/benefits of this Plugin?

 

Greetings

Protheus

 

P.S: The Reason for this excessive asking is:

My default HDDs are 12TB HDDs. When i have so long times with a 4TB disk i am afraid to add a new disk.

How long should this run? 40 hours (without pre- or postread)

Link to comment
1 hour ago, Protheus said:

explain me the purpose/benefits of this Plugin?

Preclear can be useful for testing disks and to help eliminate "infant mortality".

 

1 hour ago, Protheus said:

I read that the array is offline while unraid-clearing is in progress

Unraid does NOT take the array offline if it needs to clear a disk. It used to work that way on old versions but hasn't been like that for a long time.

 

There is only one scenario where Unraid actually requires a clear disk. This is when ADDING a disk to a NEW data slot in the parity array when parity is already valid. This is so parity will remain valid, since a clear disk is all zeros and so has no effect on parity. For all other scenarios, a clear disk IS NOT required.

 

Don't know why you had a problem with the plugin. Maybe diagnostics and the logs from preclear would tell.

1 hour ago, Protheus said:

pre-clearing of a 4TB sshd - failed at 73% (Default script (gfjardim i think) enabled with pre- and post read)

Needed Time == 11h

It would normally take a lot longer than 11h for 4TB with pre and post read.

Link to comment

Hi guys,

 

Having some issues in the last few hours even since updating the Preclear Disk plugin. before the update it failed at 99% zeroing, now it is giving an error during the pre-read stage. I've attached the full log file but this jumps out at me:

 

Dec 13 23:58:00 preclear_disk_9RKHVUBD_15516: Low memory detected, aborting...
Dec 13 23:58:01 preclear_disk_9RKHVUBD_15516: S.M.A.R.T.: Error:

 

Not sure why that would be the case as I have 16gb RAM and very little running on the server (a Plex RAM transcode could be an issue but the ram usage rarely goes above 15/20%).

 

Any help greatly appreciated! It's a new 12TB drive. I could potentially try the alternative script, up to now I've just been using the default plugin settings.

preclear_disk_9RKHVUBD_15516(1).txt

Link to comment
19 minutes ago, eric90000 said:

Hi guys,

 

Having some issues in the last few hours even since updating the Preclear Disk plugin. before the update it failed at 99% zeroing, now it is giving an error during the pre-read stage. I've attached the full log file but this jumps out at me:

 

Dec 13 23:58:00 preclear_disk_9RKHVUBD_15516: Low memory detected, aborting...
Dec 13 23:58:01 preclear_disk_9RKHVUBD_15516: S.M.A.R.T.: Error:

 

Not sure why that would be the case as I have 16gb RAM and very little running on the server (a Plex RAM transcode could be an issue but the ram usage rarely goes above 15/20%).

 

Any help greatly appreciated! It's a new 12TB drive. I could potentially try the alternative script, up to now I've just been using the default plugin settings.

preclear_disk_9RKHVUBD_15516(1).txt 2.16 kB · 3 downloads

Please PM me your Diagnostics file.

Link to comment
2 minutes ago, gfjardim said:

Can't find the bug. I've added some verbosity to version 2020.12.13b, please update and try again, and if it fails please send me your logs.

Great thank you gfjardim, I have updated the plugin and I am restarting the pre-clear on the same disk now. I will let you know how it goes.

Link to comment

Need some help with 2 new drives. 

 

A WD red plus drive was making a clicking sound during pre read phase. The click itself didn't sound abnormal for a dive(this is my 6th WD red, so i am familiar with how they sound, but first plus drive) but the click was happening exactly  once a second like a clock. Pre read completed successfully. After pre read, clicking stooped. 

 

An other drive I just bought (a seagate ironwolf if matters) also passed pre clear. Put in the logs section I see the line: 

Dec 13 14:50:07 preclear_disk_VDKAS47P_25498: Pre-Read: dd - read 8001565319168 of 8001563222016.

since the numbers don't match should I be alarmed? (full log posted below) 

 

This is my first post on the forums, since I always try to solve things by my self first, but the pre-clearing (especially pre reading) issues seems to be not very discussed online. Any help appreciated!

Full Logs:
 

Screen Shot 2020-12-13 at 7.10.18 PM.png

Link to comment

 Hi trurl,

 

6 hours ago, trurl said:

Preclear can be useful for testing disks and to help eliminate "infant mortality".

OK, testing benefits, i understand (had some DOA disks in my life :) )

 

6 hours ago, trurl said:

Don't know why you had a problem with the plugin. Maybe diagnostics and the logs from preclear would tell.

Like i said, in the Logs was nothing error-related.

 

6 hours ago, trurl said:
7 hours ago, Protheus said:

pre-clearing of a 4TB sshd - failed at 73% (Default script (gfjardim i think) enabled with pre- and post read)

Needed Time == 11h

It would normally take a lot longer than 11h for 4TB with pre and post read.

Yes, i thought so...but it failed at 73% in zeroing phase.

 

I'll try the plugin with the next hdd i get (or i have laying around).

Thanks for the reply.

 

Greetings

Protheus

Link to comment
On 12/14/2020 at 1:44 AM, gfjardim said:

Can't find the bug. I've added some verbosity to version 2020.12.13b, please update and try again, and if it fails please send me your logs.

Finished successfully this time! Thanks for all the help.

 

12TB EMFZ WD Elements Drive - 'Preclear: total elapsed time: 62:16:06'

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.