Jump to content
gfjardim

Preclear plugin

2811 posts in this topic Last Reply

Recommended Posts

1 hour ago, dev_guy said:

You would think, however, if I had bad RAM it would eventually crash Unraid and the entire machine. Still, it's worth a try.

You might think that... The thing about bad RAM is that you never know exactly how it may fail.  It may be a single bit error at a single location.  It may be an addressing fault (accessing the same physical bits at more than one location).  It may be an error across a block of addresses.  It may crash unRAID or it may remain invisible.  The same goes for other operating sytems such as Windows.  RAM can also go bad after months or years of rock-solid service.  I would always recommend at least three complete passes of Memtest, and even then that may not catch everything.  I would also recommend anyone to run it maybe once a year if they have some server down time. 

Share this post


Link to post
4 hours ago, Nanobug said:

It's not empty. It was running with a Windows 10 Pro before. So I just wanna clear it completely, and then use it, or whatever makes more sense.

You have to go to Tools > New Config, set Preserve current assignments to None, check Yes I want to do this and then click Apply.

Share this post


Link to post
5 hours ago, S80_UK said:

You might think that... The thing about bad RAM is that you never know exactly how it may fail. 

I agree memory problems can be elusive. I've been running Memtest for the last 6 hours with no errors so far. I'll let it run overnight. I think a more likely explanation is there's some weird incompatibility with the current version of Preclear (or the script) that's not compatible with these drives at least on that hardware. They're 2.5" drives which I suspect not many Unraid users use. The target hardware is a 1U shallow rack Xeon Supermicro server with ECC RAM but I'm doing the preclear offline on a different system where all the supposed "failures" occurred.

Share this post


Link to post
48 minutes ago, dev_guy said:

I agree memory problems can be elusive. I've been running Memtest for the last 6 hours with no errors so far. I'll let it run overnight. I think a more likely explanation is there's some weird incompatibility with the current version of Preclear (or the script) that's not compatible with these drives at least on that hardware. They're 2.5" drives which I suspect not many Unraid users use. The target hardware is a 1U shallow rack Xeon Supermicro server with ECC RAM but I'm doing the preclear offline on a different system where all the supposed "failures" occurred.

There are no incompatibilities between the script and your hard drives; or the data stream became corrupted before it was written, or it became corrupted after it was read, or your system produces zero filled data streams that aren't composed only by zeros. Many things can corrupt data, like bad memories, bad PSU, bad cables, bad HBA. The script relies on applications widely used around the world, like dd for read and cmp for comparison. And no incompatibility can lead to hard drives being dropped by the HBA, only a driver or a hardware problem can drop drives like that. I'm not defending the script by itself, I'm alerting you that your system may have more serious issues than it appear to have.

Share this post


Link to post
1 hour ago, gfjardim said:

There are no incompatibilities between the script and your hard drives; or the data stream became corrupted before it was written, or it became corrupted after it was read, or your system produces zero filled data streams that aren't composed only by zeros. Many things can corrupt data, like bad memories, bad PSU, bad cables, bad HBA. The script relies on applications widely used around the world, like dd for read and cmp for comparison. And no incompatibility can lead to hard drives being dropped by the HBA, only a driver or a hardware problem can drop drives like that. I'm not defending the script by itself, I'm alerting you that your system may have more serious issues than it appear to have.

Thank you for the reply and also the suggestions from S80_UK and Trurl. I ran HTOP with columns for I/O enabled and noticed DD and CMP were being used. And I agree those by themselves should not be an issue. I am currently trying to preclear the drives again on different hardware.

 

I ran HTOP because Unraid was reporting 100% CPU doing the preclear on 7 drives at the same time. But HTOP was only reporting around 50% total CPU usage so I'm assuming Unraid somehow has that wrong in the dashboard (at least for my configuration). But I would assume, even if the preclear process is limited by 100% CPU, it still should not cause false failures.

 

I'll report what happens on the new hardware when the new batch of preclears complete. Thanks again!

Share this post


Link to post
19 hours ago, trurl said:

Have you done memtest? Bad RAM has caused this in the past, including for me.

Thank you Trurl! The system I was doing the preclear on DID have bad RAM! It has 32GB or RAM so it took a while for it to show up in the memory test. So thanks for that tip! And, likewise, the same drives precleared successfully on different hardware.

 

The drive that had logged a SMART error for a pending sector needed to be precleared with pre-read disabled so it would not abort before it ever got to the zero phase. During the zero phase the pending sector was recovered.

 

Thanks also to S80_UK and Gfjardim for their input. Indirectly preclear helped me find bad RAM rather than bad drives.

 

A question to gfjardim or anyone else: Is the pre-read designed to cause a preclear to abort early if a drive has problems? I suspect this is documented elsewhere in this 109 page thread, but it seems to fail drives with "soft" errors unless you disable the pre-read and then the pending sector is recovered during the zero phase? Someone using the default settings might assume an expensive hard drive is defective when, in reality, it's not.

 

Share this post


Link to post
26 minutes ago, dev_guy said:

Thanks also to S80_UK and Gfjardim for their input. Indirectly preclear helped me find bad RAM rather than bad drives.

 

A question to gfjardim or anyone else: Is the pre-read designed to cause a preclear to abort early if a drive has problems? I suspect this is documented elsewhere in this 109 page thread, but it seems to fail drives with "soft" errors unless you disable the pre-read and then the pending sector is recovered during the zero phase? Someone using the default settings might assume an expensive hard drive is defective when, in reality, it's not.

 

 

Glad it helped both ways!

 

Pre-read will abort if there is any read errors, including those generated by pending sectors. The problem here is documentation IMO, because the pending sector will show in the log but the user don't have any indication of what to do. Maybe more Q/A entries on the OP should help Thanks for pointing out.

Share this post


Link to post

Hello, first time using Preclear (well, first time using unraid). I am preclearing a disk (WD RED 4TB) and I was wondering if this is normal (the decrease in speed, now it is at 98 % with a speed of 87 MB/s):

 

Mar 07 13:35:27 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 10% read @ 181 MB/s
Mar 07 14:12:22 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 20% read @ 177 MB/s
Mar 07 14:50:27 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 30% read @ 171 MB/s
Mar 07 15:30:10 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 40% read @ 163 MB/s
Mar 07 16:12:06 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 50% read @ 153 MB/s
Mar 07 16:56:43 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 60% read @ 145 MB/s
Mar 07 17:44:45 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 70% read @ 133 MB/s
Mar 07 18:37:33 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 80% read @ 119 MB/s
Mar 07 19:36:42 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 90% read @ 104 MB/s

if this is normal can somebody explain why? Thank you!

 

Share this post


Link to post
Posted (edited)
7 minutes ago, jorge16 said:

Hello, first time using Preclear (well, first time using unraid). I am preclearing a disk (WD RED 4TB) and I was wondering if this is normal (the decrease in speed, now it is at 98 % with a speed of 87 MB/s):

 


Mar 07 13:35:27 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 10% read @ 181 MB/s
Mar 07 14:12:22 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 20% read @ 177 MB/s
Mar 07 14:50:27 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 30% read @ 171 MB/s
Mar 07 15:30:10 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 40% read @ 163 MB/s
Mar 07 16:12:06 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 50% read @ 153 MB/s
Mar 07 16:56:43 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 60% read @ 145 MB/s
Mar 07 17:44:45 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 70% read @ 133 MB/s
Mar 07 18:37:33 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 80% read @ 119 MB/s
Mar 07 19:36:42 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 90% read @ 104 MB/s

if this is normal can somebody explain why? Thank you!

 

If it's a spinning drive (mechanical) it's normal.
Imagine it's a CD/DVD disk, on the edge it will access more data, because the circle is larger, because it has more area to read/write from, the closer it gets to the center, it will have less, because the circle is smaller.

Edited by Nanobug

Share this post


Link to post
2 minutes ago, jorge16 said:

Hello, first time using Preclear (well, first time using unraid). I am preclearing a disk (WD RED 4TB) and I was wondering if this is normal (the decrease in speed, now it is at 98 % with a speed of 87 MB/s):

 


Mar 07 13:35:27 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 10% read @ 181 MB/s
Mar 07 14:12:22 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 20% read @ 177 MB/s
Mar 07 14:50:27 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 30% read @ 171 MB/s
Mar 07 15:30:10 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 40% read @ 163 MB/s
Mar 07 16:12:06 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 50% read @ 153 MB/s
Mar 07 16:56:43 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 60% read @ 145 MB/s
Mar 07 17:44:45 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 70% read @ 133 MB/s
Mar 07 18:37:33 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 80% read @ 119 MB/s
Mar 07 19:36:42 preclear_disk_WD-WCC7K5VD2F2P_28187: Pre-Read: progress - 90% read @ 104 MB/s

if this is normal can somebody explain why? Thank you!

 

The inner "cylinders" have less data in them (shorter in circumference) but the rotation rate is the same. It starts with the outer (longer, more data) and works towards the inner (shorter, less data). So the data rate will be less as it works through the whole disk.

Share this post


Link to post
Posted (edited)

Hi,

 

My hard disk, WD green serial No. WMAZA 9837135 indicated it has finished preclear successfully.

 

But if I click on the link for seeing the report, it shows another WD green HDD.

It shows WMAZA 9742022? Why this happening?

 

It is bug within preclear?

 

Actually  WD green serial No. WMAZA 9742022 is still hasn't finished preclear.

See the bottom last picture attached.

 

 

Preclear-Error-1.jpg

Preclear-Error-2.jpg

Preclear-Error-3.jpg

Edited by wepee

Share this post


Link to post

@wepee, please close all your browser windows and tabs and try it again. Let's try to replicate it.

Share this post


Link to post
Posted (edited)
2 hours ago, gfjardim said:

@wepee, please close all your browser windows and tabs and try it again. Let's try to replicate it.

I have closed my firefox browser and restarted again, log into my unraid server, 

click on the preclear icon, and then click the same hard disk, 

WD green serial No. WMAZA 9837135, no luck, no change, same issue.

 

Next step, I am going to restart my unraid server and see, before

that I need to wait for the preclear operation to finish the last HDD,

up till 90% at Post-read phase.

Edited by wepee

Share this post


Link to post

After a few hours, WD green serial No. WMAZA 9742022 has finished preclear.

See the bottom last picture attached.

Its preclear report shown the correct hdd with correct serial no. that is WMAZA 9742022

Correct-Preclear-Report-Shown.jpg

Share this post


Link to post

I've looked the code and I see no obvious way this is happening. It filters the report from /boot/preclear_reports using disk short serial number, and that value you see in the link it's the file base name without the extension. I've tried and couldn't replicate it too....

Share this post


Link to post
Posted (edited)
1 hour ago, gfjardim said:

I've looked the code and I see no obvious way this is happening. It filters the report from /boot/preclear_reports using disk short serial number, and that value you see in the link it's the file base name without the extension. I've tried and couldn't replicate it too....

Err.....bad news.....sobbing.......

I think the culprit came within my Kingmax usb 2.0 thumb drive.

 

I rebooted my unRAID server.......but couldn't log in. What was the problem? I am asking myself.......

 

Troubleshooting started....

I plugged in to the monitor of my server, the motherboard UEFI bios couldn't detected the Kingmax usb 2.0 thumb drive.

I plugged in an unused thumb drive, the UEFI bios was able to detect a thumb drive inserted.

Unplugged the Kingmax usb2.0 thumb drive, and inserted into my Windows 10 PC, it wasn't able to read the thumb drive.

 

So, I guess, pretty sure my Kingmax usb 2.0 thumb drive has kicked the bucket for good.

 

Question, does unRAID thumb drive has build-in software/plugin to minimize the wear on the flash memory on the drive itself?

 

 

Kingmax-4GB-USB2.0.jpg

Edited by wepee

Share this post


Link to post
14 minutes ago, wepee said:

 

Question, does unRAID thumb drive has build-in software/plugin to minimize the wear on the flash memory on the drive itself?

 

 

Unraid itself writes very little into the flash drive. You will see writes mainly during settings change, SO update and plugins update.

 

Plugins are a whole different matter. Many plugins use flash drive to store their own installation files and their configuration only, but some will store some status too like Preclear Plugin that stores reports and resume status. Resume status is written every minute, they are less than 1KB size, so every instance of preclear will write at most 1.44MB each 24 hours, which isn't much. It writes two 4KB reports too at the end, again, not much.

 

Not only NAND wearing kills flash drives. Some controllers too have low MTBF and will render your flash drive useless even if your memory isn't wore out.

 

I do recommend Sandisk flash drives since them all have wear leveling, but it's good to make a flash backup now and then.

Share this post


Link to post
Posted (edited)
Quote

I do recommend Sandisk flash drives since them all have wear leveling, but it's good to make a flash backup now and then.

Yes, I am installiing a fresh copy of unRAID on a Sandisk flash drive 8GB usb 2.0 thumb drive.

 

I think I should not be doing another round of preclear, since the last preclear operation has completely successfully,

 

unRAID will be noticed it, once I get the unRAID server up and running, right?

 

I mean it will recognize the hard disks since it has a preclear signature written on them from the last preclear operation, right?

Edited by wepee

Share this post


Link to post

Hi, today I tried the Preclear Plugin but when I clicked "Start Preclear" nothing happened. I try 2 HDD but none of both worked.

Share this post


Link to post
3 minutes ago, Damix21 said:

Hi, today I tried the Preclear Plugin but when I clicked "Start Preclear" nothing happened. I try 2 HDD but none of both worked.

Please send me your diagnostics file.

Share this post


Link to post
1 minute ago, gfjardim said:

Please send me your diagnostics file.

Where I can find it?

Share this post


Link to post
5 minutes ago, Damix21 said:

Where I can find it?

In Tools > Diagnostics

Share this post


Link to post
Posted (edited)
3 minutes ago, Damix21 said:

The 2 HDD I tried to preclear are WDC_WD20PURX

 

Your login session is no longer valid, just close your web browser and login again into the webgui. 

Edited by gfjardim

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.