Preclear plugin


Recommended Posts

So I had 2 disks preclear successfully, but that's where they're stuck, it says "preclear finished successfully!" and it doesn't really clear from there

 

If I click cancel, unassigned devices says the FS is precleared but I can't mount them, and pre-clear gives me a resume option

 

Here's the log from one of them:

 

Jul 12 14:14:18 preclear_disk_MK0351YHHRV25A_18063: Command: /usr/local/emhttp/plugins/preclear.disk/script/preclear_disk.sh --load-file /tmp/.preclear/MK0351YHHRV25A.resume /dev/sdp
Jul 12 14:14:18 preclear_disk_MK0351YHHRV25A_18063: Preclear Disk Version: 1.0.4
Jul 12 14:14:18 preclear_disk_MK0351YHHRV25A_18063: Restoring previous instance of preclear
Jul 12 14:14:20 preclear_disk_MK0351YHHRV25A_18063: S.M.A.R.T. info type: default
Jul 12 14:14:20 preclear_disk_MK0351YHHRV25A_18063: S.M.A.R.T. attrs type: default
Jul 12 14:14:24 preclear_disk_MK0351YHHRV25A_18063: Continuing disk read from byte 3000590884864
Jul 12 14:14:24 preclear_disk_MK0351YHHRV25A_18063: Post-Read: verifying the rest of the disk.
Jul 12 14:14:24 preclear_disk_MK0351YHHRV25A_18063: Post-Read: cmp /tmp/.preclear/sdp/fifo /dev/zero
Jul 12 14:14:24 preclear_disk_MK0351YHHRV25A_18063: Post-Read: dd if=/dev/sdp of=/tmp/.preclear/sdp/fifo bs=2097152 skip=3000590884864 count=2097152 conv=notrunc iflag=nocache,count_bytes,skip_bytes
Jul 12 14:14:30 preclear_disk_MK0351YHHRV25A_18063: Post-Read: progress - 100% verified
Jul 12 14:14:31 preclear_disk_MK0351YHHRV25A_18063: Post-Read: dd - read 3000592982016 of 3000592982016.
Jul 12 14:14:31 preclear_disk_MK0351YHHRV25A_18063: Post-Read: dd exit code - 0
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: S.M.A.R.T.: 5 Reallocated_Sector_Ct 0
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: S.M.A.R.T.: 9 Power_On_Hours 47043
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: S.M.A.R.T.: 194 Temperature_Celsius 52
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: S.M.A.R.T.: 196 Reallocated_Event_Count 0
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: S.M.A.R.T.: 197 Current_Pending_Sector 0
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: S.M.A.R.T.: 198 Offline_Uncorrectable 0
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: S.M.A.R.T.: 199 UDMA_CRC_Error_Count 0
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] 5860533168 512-byte logical blocks: (3.00 TB/2.73 TiB)
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Write Protect is off
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Mode Sense: 43 00 00 00
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sdp: sdp1 sdp2
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Attached SCSI disk
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Synchronizing SCSI cache
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Synchronize Cache(10) failed: Result: hostbyte=0x01 driverbyte=0x00
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] 5860533168 512-byte logical blocks: (3.00 TB/2.73 TiB)
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Write Protect is off
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Mode Sense: 43 00 00 00
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sdp: sdp1
Jul 12 14:14:35 preclear_disk_MK0351YHHRV25A_18063: syslog: sd 20:0:0:1: [sdp] Attached SCSI disk

 

Link to comment
1 hour ago, richardsim7 said:

So I had 2 disks preclear successfully, but that's where they're stuck, it says "preclear finished successfully!" and it doesn't really clear from there

 

If I click cancel, unassigned devices says the FS is precleared but I can't mount them, and pre-clear gives me a resume option

Are you trying to mount them after adding to the array or under Unassigned Devices? Either way they need to be formatted. But for disks that are pre-cleared and added to the unRAID array, the Format option is on the Main page just below the Start/Stop button. Don't format them under Unassigned Devices if you want to add them to the array, or you'll have to run at least a zero stage to pre-clear them again, or let unRAID do it once added to the array.

 

Link to comment
2 minutes ago, Squid said:

Have you enabled destructive mode in UD settings

Sent from my NSA monitored device
 

Ah ha, bingo. Thanks Squid. I hadn't used Unassigned Devices in so long that I totally forgot that was a thing. After enabling that and pressing Stop Preclear, Format shows up again :)

Link to comment

I have three 3 TB WD reds that I am adding to my array. For two of them preclear works as expected. However, for one of them I get this kind of log in the beginning:

 

Jul 13 17:12:11 preclear_disk_WD-WMC1T0861973_29132: Command: /usr/local/emhttp/plugins/preclear.disk/script/preclear_disk.sh --notify 3 --frequency 3 --cycles 1 --no-prompt /dev/sde
Jul 13 17:12:11 preclear_disk_WD-WMC1T0861973_29132: Preclear Disk Version: 1.0.4
Jul 13 17:12:11 preclear_disk_WD-WMC1T0861973_29132: /usr/local/emhttp/plugins/preclear.disk/script/preclear_disk.sh: line 1716: let: parts+=( / 4096): syntax error: operand expected (error token is "/ 4096)")
Jul 13 17:12:12 preclear_disk_WD-WMC1T0861973_29132: S.M.A.R.T. info type: default
Jul 13 17:12:12 preclear_disk_WD-WMC1T0861973_29132: S.M.A.R.T. attrs type: default
Jul 13 17:12:15 preclear_disk_WD-WMC1T0861973_29132: Pre-Read: dd if=/dev/sde of=/dev/null bs=2097152 skip=0 count=3000592982016 conv=notrunc iflag=nocache,count_bytes,skip_bytes

 

After this, the preclear seems to be fine.

 

I tried to search this topic but didn't find anything for this. So, is something wrong with the disk or with something else? Thank you for the info!

 

Link to comment

@Ruato

could be the incompatibility of the current preclear plugin with the latest stable unraid..

uninstall preclear 

reboot 

reinstall preclear....  

it should not give you issues till next reboot or several runs..  since release of 6.7.2   i had issues.. i either downgraded back to 6.6.6 to get preclear to work  no issues  or i had to play with uninstall reinstall etc..

 

i dont know if there is a new release of preclear as i set unraid to update check/update every night...  and my last hard drive i precleared was a 12TB that takes 7 days to do 3 cycles a month ago..

but give that a try the uninstall reinstall  and then run preclear....

 

now dont take my word if there is a latest version fix  for preclear.. i just going off what i been doing since 6.7.2 release and did a few hds and SSDs and thats what i had to do to get 0 issues

 

Edited by comet424
Link to comment
15 hours ago, comet424 said:

@Ruato

could be the incompatibility of the current preclear plugin with the latest stable unraid..

uninstall preclear 

reboot 

reinstall preclear....  

...

now dont take my word if there is a latest version fix  for preclear.. i just going off what i been doing since 6.7.2 release and did a few hds and SSDs and thats what i had to do to get 0 issues

 

I will try this and let you know how it goes. Thank you!

Link to comment
On 7/12/2019 at 6:59 AM, Frank1940 said:

I believe you will have to format them.  (A precleared disk has no file system on it so it can not be mounted until it has one.) 

If I preclear in another PC, how will Unraid know it was Precleared if there is no file system?

Link to comment
14 minutes ago, Paul_Ber said:

A new 6TB EFAX WD Red Nas.  I have a 5TB WD Red Nas starting to get errors, last week 336 errors and yesterday that went up to 10000 errors.  Doing the preclear in an old unused PC.

Depending on the exact errors, you may be better off doing the replacement sooner rather than later. A replacement drive doesn't need to be clear, as unraid will be rebuilding the entire drive content anyway and ignores the preclear flag. Testing is prudent, but if you are relatively confident the new drive is flawless, I'd proceed with getting the failed drive replaced rather than waiting on the preclear process to finish.

 

This all depends on the errors you are seeing on the 5TB, if they are cable induced communication errors then replacing the drive isn't going to help. The smart report should tell the tale.

Link to comment
23 minutes ago, jonathanm said:

Depending on the exact errors, you may be better off doing the replacement sooner rather than later. A replacement drive doesn't need to be clear, as unraid will be rebuilding the entire drive content anyway and ignores the preclear flag. Testing is prudent, but if you are relatively confident the new drive is flawless, I'd proceed with getting the failed drive replaced rather than waiting on the preclear process to finish.

 

This all depends on the errors you are seeing on the 5TB, if they are cable induced communication errors then replacing the drive isn't going to help. The smart report should tell the tale.

It is at 88% for step 2 of 5.  How long does steps 3,4,5 take?

 

Same cable for more than 4 years and haven't opened the case in a long time to mess with anything.

 

Pastebin Disk error log.  https://pastebin.com/sfMTZuhb

 

image.thumb.png.998d159cc4d1903b8a5c86958c9c567d.png

 

Edited by Paul_Ber
Add Smart
Link to comment
42 minutes ago, Paul_Ber said:

It is at 88% for step 2 of 5.  How long does steps 3,4,5 take?

It's approximately the same amount of time for each of the 3 phases of a single-pass pre-clear. Since your pre-read took about 6 hours, the entire process should be complete in about 18 hours. There are 5 steps listed in the status, but only 3 actual phases.

 

EDIT: Just noticed your write speed is quite a bit lower than your read speed. I haven't seen this with the drives I've pre-cleared... writing is usually slower, but not usually as much as what your screenshot shows. Do you have other tasks doing writing to other drives on that system? Or perhaps some DMA bound processes if your SATA controller is older and resides on the older PCI bus? Many older motherboards have this issue if it's a motherboard SATA port.

 

Edited by AgentXXL
Link to comment
5 hours ago, AgentXXL said:

It's approximately the same amount of time for each of the 3 phases of a single-pass pre-clear. Since your pre-read took about 6 hours, the entire process should be complete in about 18 hours. There are 5 steps listed in the status, but only 3 actual phases.

 

EDIT: Just noticed your write speed is quite a bit lower than your read speed. I haven't seen this with the drives I've pre-cleared... writing is usually slower, but not usually as much as what your screenshot shows. Do you have other tasks doing writing to other drives on that system? Or perhaps some DMA bound processes if your SATA controller is older and resides on the older PCI bus? Many older motherboards have this issue if it's a motherboard SATA port.

 

It is being done in a really old PC with the Motherboard being 9 or more years old.  Bios 2010.

 

No other tasks on that old PC just Unraid and Preclear.

Edited by Paul_Ber
Link to comment
13 hours ago, Ruato said:

I will try this and let you know how it goes. Thank you!

@Ruato hopefully it fixes your issues...  ya only other thing that worked for me was downgrade unraid to 6.6.6 do the preclear.. then reupgrade back to 6.7.2

that was my other way of fixing it to get the preclear jobs done

but who wants to keep downgrading upgrading lol

 

Link to comment
1 hour ago, comet424 said:

@Ruato hopefully it fixes your issues...  ya only other thing that worked for me was downgrade unraid to 6.6.6 do the preclear.. then reupgrade back to 6.7.2

that was my other way of fixing it to get the preclear jobs done

but who wants to keep downgrading upgrading lol

 

When I made the usb stick to use as a trial on the old PC, I picked the 6.6.6 version.

 

6hrs on step 1

11.5hrs on step 2

In step 5 now, 21.5hrs so far all together for 6TB 

 

This PC is so old, the BIOS is from 2010 running an AMD 1090T

 

Edited by Paul_Ber
Link to comment

@Paul_Ber

ya 6.6.6  more stable with the preclear..  also your old pc is probably a Sata 2  possible Sata1 speeds   so you wont get the faster speed of the newer sata3.2

like a sata 3.2 12tb 7200rpm  will take me 7 days to run 3 cycles..  dont wanna know how long it take at Sata1 or 2 lol

but i use a Acer H340 (Sata1 or 2) with 1GB ram from 2006  i used that as a unraid backupserver  and she slow to do preclear but then i only did 2 -4 tb drives on that puppy lol

but unraid works nice also on a 1gb machine

Link to comment
1 hour ago, comet424 said:

@Paul_Ber

ya 6.6.6  more stable with the preclear..  also your old pc is probably a Sata 2  possible Sata1 speeds   so you wont get the faster speed of the newer sata3.2

like a sata 3.2 12tb 7200rpm  will take me 7 days to run 3 cycles..  dont wanna know how long it take at Sata1 or 2 lol

but i use a Acer H340 (Sata1 or 2) with 1GB ram from 2006  i used that as a unraid backupserver  and she slow to do preclear but then i only did 2 -4 tb drives on that puppy lol

but unraid works nice also on a 1gb machine

Are you saying you do the Preclear 3 times?

Link to comment
Just now, Paul_Ber said:

Are you saying you do the Preclear 3 times?

yes i do  it gives it a stability test  if it can pass 3 times i know it should last a while plus my WD come with 5 yr warrentys  and i have a 12TB parity drive 

but ya all my drives i do 3 times i just set it to run 3 times in the options and come back in 7 days or whatever its done

 

Link to comment
3 hours ago, comet424 said:

yes i do  it gives it a stability test  if it can pass 3 times i know it should last a while plus my WD come with 5 yr warrentys  and i have a 12TB parity drive 

but ya all my drives i do 3 times i just set it to run 3 times in the options and come back in 7 days or whatever its done

 

Everyone has their own idea of what qualifies as a successful stress test of a new drive. In the past before using unRAID, I often would use the drive manufacturers test tools for at least one pass. Sometimes though it was as simple as doing a full format (not a quick format) of the drive (when using Windows).

 

For me, I've not had many occurrences of drive 'infant mortality' in recent years so I'm now comfortable with a single 3-phase pass of the pre-clear plugin. With 8TB or 10TB drives, that usually takes up to 48hrs to do all 3 phases. Around 36 hours for the 8TB drives and around 45 hours for the 10TB ones.

 

Right now I'm less concerned about the new drives but more with cabling, controller and potential oxidation on the hot-swap SATA backplanes in my Norcotek enclosure. Half of the 20 drive bays have never had anything installed in them until recently, but having moved from FreeNAS to unRAID, I now have a number of those previously unused bays in operation. They sat for over 6 years with nothing connected so oxidation of the connectors is a real potential issue that I now have to address.

 

First I just want to get my array back to stable without the one drive - for some reason always Disk 8, that keeps disabling after reboots or adding another new drives to the array. I've got a new topic created about 10 minutes ago asking for suggestions on how to proceed.

 

Regardless, I'm not experiencing the same bugs that some are with the pre-clear plugin on 6.7.2. I did have an unRAID VM setup on a Windows system that had an eSATA connection pass-through so that I could use it for pre-clearing, but the Windows box suffered a massive hardware failure and so that option isn't available until I get it back up and running. For now I'm back to doing pre-clears on my 'production' unRAID.

 

Edited by AgentXXL
Link to comment
22 minutes ago, AgentXXL said:

Everyone has their own idea of what qualifies as a successful stress test of a new drive. In the past before using unRAID, I often would use the drive manufacturers test tools for at least one pass. Sometimes though it was as simple as doing a full format (not a quick format) of the drive (when using Windows).

 

For me, I've not had many occurrences of drive 'infant mortality' in recent years so I'm now comfortable with a single 3-phase pass of the pre-clear plugin. With 8TB or 10TB drives, that usually takes up to 48hrs to do all 3 phases. Around 36 hours for the 8TB drives and around 45 hours for the 10TB ones.

 

Right now I'm less concerned about the new drives but more with cabling, controller and potential oxidation on the hot-swap SATA backplanes in my Norcotek enclosure. Half of the 20 drive bays have never had anything installed in them until recently, but having moved from FreeNAS to unRAID, I now have a number of those previously unused bays in operation. They sat for over 6 years with nothing connected so oxidation of the connectors is a real potential issue that I now have to address.

 

First I just want to get my array back to stable without the one drive - for some reason always Disk 8, that keeps disabling after reboots or adding another new drives to the array. I've got a new topic created about 10 minutes ago asking for suggestions on how to proceed.

 

Regardless, I'm not experiencing the same bugs that some are with the pre-clear plugin on 6.7.2. I did have an unRAID VM setup on a Windows system that had an eSATA connection pass-through so that I could use it for pre-clearing, but the Windows box suffered a massive hardware failure and so that option isn't available until I get it back up and running. For now I'm back to doing pre-clears on my 'production' unRAID.

 

"potential oxidation on the hot-swap SATA backplanes in my Norcotek enclosure. Half of the 20 drive bays have never had anything installed in them until recently" Too bad when the drive bays were new, there wasn't a dummy blank you could left plugged in so that the connections couldn't oxidize?  That is like the old game cartridges from the 1980s that the card edge would need an eraser every now and then.

Link to comment
2 minutes ago, Paul_Ber said:

Too bad when the drive bays were new, there wasn't a dummy blank you could left plugged in so that the connections couldn't oxidize?  That is like the old game cartridges from the 1980s that the card edge would need an eraser every now and then.

The drive trays were left in all slots to prevent dust, etc from entering the unused bays, but yes, open air exposure for the SAS/SATA connectors on 2.5 shelves. I'm more than willing to do the cleanup and plan to apply some contact stabilizer like the Stabilant 22 I use for a lot of my audio gear. I just want to wait until the new MiniSAS cables arrive as I'm pretty convinced the current extra-long ones are part of the issue too.

Link to comment
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.