Preclear plugin


3040 posts in this topic Last Reply

Recommended Posts

 

Quote

<snip>

Q) The plugin asked me to send some statistics information. How does the statistics report system work? Is it safe? Is it anonymous?

A) To better track the usage of the plugin, a statistics report system was put in place. The main goals I intend to archive are:

  • know number of disks that gets precleared;
  • fix any silent bugs that gets reported on the logs;
  • know average size of disks, their model, their average speed and elapsed time we should expect from that model;
  • success rate;
  • rate of disks with SMART problems;

 

This system is totally optional and users will get prompted if they want to send each report. It is also safe and totally anonymous, since all data is sent to Google Forms and no identifying data is exported, like disks serial numbers. Detailed info can be found here. The statistics are public and can be found here.

 

<snip>

I'm not too smart but I think the link is dead.

@gfjardim

Edited by Hikakiller
Link to post
  • Replies 3k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Hi guys, this is a simple plugin that allows users to clear their disks before add them to the array.   The main characteristics of this plugin are: Modularity: can be used standalon

I think Thanks should be directed at all those members that have been dealing with the mess the last few months. @dlandon and @Squid have been working behind the scenes to try and clean up, and o

Dontlookoverthere over on the unRAID subreddit figured out how to get the plugin running again with a simple edit to the plg:   https://www.reddit.com/r/unRAID/comments/7wjpus/preclear_worki

Posted Images

This problem has returned today for me too.  I was having this problem with a bunch of brand new 4TB drives Friday last week, but updated the plugin to 209.11.28 just after it was released, and it was all fixed.  A few days later it, another update showed for the plugin, so I updated again, but noticed the version was still the same afterwards.  Today I've thrown in an old but unused 3TB shucked WD blue, and it's failing: Invalid unRAID`s MBR signature.

 

I'm not sure if it really is failing, or if that problem has reared it's head again.  Could that last update have just downgraded libevent?  This is all far beyond my skillset.

Link to post
4 hours ago, Hikakiller said:

Why not? Many of us have hotswap bays and restarting things like 12th gen poweredge servers can take a LONG time. 

Because it isn't fully supported by Unraid. You need more than a hot-pluggable drive bay to support hot-swapping so you might have to forego some convenience for reliability.

Link to post
3 minutes ago, John_M said:

Because it isn't fully supported by Unraid. You need more than a hot-pluggable drive bay to support hot-swapping so you might have to forego some convenience for reliability.

How can removing drives that aren't on the array cause instability? 

Link to post
On 12/8/2019 at 3:26 PM, Hikakiller said:

How can removing drives that aren't on the array cause instability? 

I didn't say instability.

 

You should read the posts that preceded and followed the one you quoted from 3 years ago and 60 pages back in order to understand what we were discussing, not quote me out of context and then start arguing.

 

Edited by John_M
It was "only" 3 years ago!
Link to post

As this question relates to both UD and Preclear plugins, I’ll post it in both support threads. I’m one of the users who hasn’t had any UD or preclear issues until recently. I am currently preclearing 4 x 10TB drives (USB attached) via the UD integration with the preclear plugin. I’m 60% through the zero stage on all 4 drives but have seen some unusual issues today.

 

The only change to unRAID prior to these issues was updating the UD plugin to 2019.12.08 earlier this morning. I have been deploying the various updates for UD and preclear as they’ve been released over the last 2 weeks and up until today things seemed normal. Earlier this afternoon I started seeing a LOT of sporadic buffering/disk I/O issues and the CPU performance on my unRAID system is showing 65 - 75% load  pretty consistently.

 

This is based on the unRAID Dashboard tab of the web GUI. TOP in a terminal session reports about 15% utilization.

The buffering/disk I/O issues are real as transfers to/from unRAID speed up and slow down at random. I’m also seeing major interruptions to streams from Plex. As I can’t determine why the web GUI shows high CPU load but TOP doesn’t I’m wondering if the last update(s) to UD/Preclear might have broken something?

 

I’m still running unRAID 6.7.2 and note that most of the updates I’m seeing are for 6.8 related issues. I could update to 6.8 RC9 (with NVidia extensions) but I’ve held off. As the preclear is 60% through stage 2 (zeroing), I’m concerned about the upgrade and reboot possibly wiping the preclear state and making me start it over. If I have to I’ll tough it out until the preclear finishes tomorrow on all 4 drives.

Any thoughts on whether the preclear status will be saved if I update? Is there a way I can manually stop the preclears and save the state for restarting after the upgrade?

 

I’ve tried shutting down all unnecessary plugins/Dockers/VMs and the unRAID web GUI is still showing 60%+ CPU load and I’m still seeing disk I/O issues. What could cause the discrepancy between TOP in a terminal session and the web GUI?

 

Thoughts? Suggestions? Diagnostics would be attached but 3 attempts to download them have taken over an hour during the collection stage…. another bug/related issue? Tried on 2 separate computers on 3 different browsers (Safari/Firefox on MacOS Mojave and Firefox/Chrome on Ubuntu 18.04.

 

EDIT: I'm attaching the diagnostics that I was able to get by running the 'diagnostics' command from a terminal shell. Still couldn't get the Tools -> Diagnostics option in the web GUI to work.

 

Dale

animnas-diagnostics-20191209-2301.zip

Edited by AgentXXL
Attaching diagnostics file
Link to post
9 hours ago, AgentXXL said:

I’ll post it in both support threads.

Please don't do this. Pick one thread to receive replies in and post in that one. If you must, post a link to that in the related thread.

 

It is impossible to coordinate replies when you crosspost.

Link to post

Hello everybody,

 

i am pretty new to unraid, first installation, 6.8.

 

I installed a PCIx (1x) USB 3.0 card and tried to preclear a new Seagate 8TB USB3 external Harddrive.

This is what happens:

- Card is recognized by OS

- Ext. harddrive over USB card also

- Start of preclear

here comes the fun part

- HD vanishes

- Card not working anymore

- Only reinstalling the card makes it working again

Any ideas? I will try the same over onboard USB und will give an update

Thanks in advance
mindfest

Link to post
7 minutes ago, mindfest said:

Hello everybody,

 

i am pretty new to unraid, first installation, 6.8.

 

I installed a PCIx (1x) USB 3.0 card and tried to preclear a new Seagate 8TB USB3 external Harddrive.

This is what happens:

- Card is recognized by OS

- Ext. harddrive over USB card also

- Start of preclear

here comes the fun part

- HD vanishes

- Card not working anymore

- Only reinstalling the card makes it working again

Any ideas? I will try the same over onboard USB und will give an update

Thanks in advance
mindfest

incompatible card i would guess, it doesnt use a marvell chipset by any chance does it?, these are known to have issues. if i were you i would stick to onboard usb.

Link to post
4 minutes ago, binhex said:

incompatible card i would guess, it doesnt use a marvell chipset by any chance does it?, these are known to have issues. if i were you i would stick to onboard usb.

I tried to find information like that by searchin the webs but had no luck. Thank You. The card itself is still recognized but not working.

Link to post
  • 2 weeks later...

I'm getting preclear errors that zeroing the disk failed during the post read (log attached). I read somewhere it could be indicative of bad RAM, but I've run a memtest for ~36 hours with no errors and have seen this same error probably 5 times now. Is the disk bad? Any other explanation? I attached smart info for the disk as well. thanks!

preclear_disk_S2H7J1CZA01988_7991.txt SAMSUNG_HD204UI_S2H7J1CZA01988-20191229-0818.txt

Link to post
20 hours ago, BRiT said:

Have you already checked this out and done firmware update if available?

 

==> WARNING: Using smartmontools or hdparm with this drive may result in data loss due to a firmware bug.

****** THIS DRIVE MAY OR MAY NOT BE AFFECTED! ******

Buggy and fixed firmware report same version number! See the following web pages for details: http://knowledge.seagate.com/articles/en_US/FAQ/223571en https://www.smartmontools.org/wiki/SamsungF4EGBadBlocks

I have not, think that may be the issue? Can give it a try if you think it's worth a shot.

Link to post
6 minutes ago, nerv said:

think that may be the issue?

It can be if it's not updated, since the drive corrupts data when reading SMART at the same time, and the plugin is checking SMART every few minutes to get the temp, unfortunately SAMSUNG in its infinite wisdom chose to not change the firmware version for the fixed one, which IMHO is very stupid.

Link to post

I'm having a problem for the first time with the preclear plugin. When I click on the preclear icon from unassigned devices, I get the first option screen and when I click start button another screen flashes by (too fast to see) and now my preclear status says "starting..." and that's where it's stuck. 

 

I'm on 6.8.0. I tried uninstalling the preclear plugin and reinstalling it, but that's the extent of my troubleshooting.

unraid1-diagnostics-20200102-0910.zip

Link to post
11 minutes ago, x88dually said:

Just installed Preclear plugin, and at the end of install :

 

preclear.disk has been installed.
This plugin requires Dynamix webGui to operate
Copyright 2015-2017, gfjardim
Version: 2019.11.28

The Dynamix webGui is what is now shipped as the standard Unraid GUI (assuming that is what you are asking about)

Link to post

Question about 'pre-clear' vs 'clearing'.

 

Situation -a 'pre-cleared' drive is now 'clearing' after adding it to the array.

 

Background - bought two ‘renewed’ drives so ran them through pre-clear; one failed “Post Read” while the other completed without error. Sending one back, kept the other Passed drive in the system. Pre-clear is up-to-date and running unRAID 6.8 stable.

 

Question - curious why drive that completed full ‘pre-clear’ is now also doing a 12 hour “clearing’? BTW, I am asking as I have used the pre-clear method for a couple of years now and this is the first time I have come across just 'clearing' or read about it.

 

So, why the ‘clear’ after ‘pre-clear’ and  thoughts about whether or not I should be suspicious about the accuracy/longevity of the drive?

Reports attached.

 

Thanks all!

 

 

 

 

 

13117689437580.sreport.sent 16882006332408.sreport.sent 01-07-19_HGST_8TB_logging.pdf

Link to post

A clear drive is all zeroes, so it can be added to the parity array without changing the parity calculation. The preclear process was originally developed because in the distant past unraid kept the whole array unavailable while the new drive was being cleared. Now, the array can be used during that process, so preclear isn't really needed for that function.

 

It can still be used as a confidence check for a drive of unknown quality, be it new or used. Whether unraid detects that the drive is successfully precleared and skips the clear phase of adding a new drive slot depends on detection of the special watermark that the preclear utility is supposed to put on the drive after completion. Any attempt to mount or write anything to the drive after preclear is done will alter the watermark and cause unraid to perform a clear before the drive is added.

 

This watermark and detection scheme is not actively maintained by limetech afaik, since preclear is not an official tool.

Link to post
45 minutes ago, jonathanm said:

A clear drive is all zeroes, so it can be added to the parity array without changing the parity calculation.

Note that the only time a disk needs to be clear is when adding it to a new data slot. A clear disk isn't needed when replacing a disk, or when adding a parity disk, since that disk is going to be completely overwritten from the parity calculation anyway. And as mentioned, if you don't preclear a disk, Unraid will clear it if it is added to a new data slot.

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.