Preclear plugin


3041 posts in this topic Last Reply

Recommended Posts

Each function the script run in parallel fork the script into another pid, simple as that.

 

Let's see. In clearing operation, we have the parent PID, the write function PID, display refresh function PID, and so goes on...

 

 

OK, so that's normal. Any advise on the "missing csrf_token"?

 

 

I updated Unassigned Devices and I think its fixed.

Link to post
  • Replies 3k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Hi guys, this is a simple plugin that allows users to clear their disks before add them to the array.   The main characteristics of this plugin are: Modularity: can be used standalon

I think Thanks should be directed at all those members that have been dealing with the mess the last few months. @dlandon and @Squid have been working behind the scenes to try and clean up, and o

Dontlookoverthere over on the unRAID subreddit figured out how to get the plugin running again with a simple edit to the plg:   https://www.reddit.com/r/unRAID/comments/7wjpus/preclear_worki

Posted Images

On a thought experiment, a missing csrf could also be caused by having a browser open to say the Main Tab when running on 6.2.4 and then upgrading to 6.3.0 and rebooting via another browser / tab / device.  The original still open tab would not have the csrf information and would also generate the error.

Link to post

On a thought experiment, a missing csrf could also be caused by having a browser open to say the Main Tab when running on 6.2.4 and then upgrading to 6.3.0 and rebooting via another browser / tab / device.  The original still open tab would not have the csrf information and would also generate the error.

 

Wow.  That's esoteric.

Link to post

On a thought experiment, a missing csrf could also be caused by having a browser open to say the Main Tab when running on 6.2.4 and then upgrading to 6.3.0 and rebooting via another browser / tab / device.  The original still open tab would not have the csrf information and would also generate the error.

 

Wow.  That's esoteric.

I'm rather proud of myself on that one  ;D
Link to post

It looks like preclear does not want to clear my drive that has been part of the array already.  I want to rebuild disk2 from parity back to itself, but I want to run a few preclear cycles it before doing that; the drive is suspect.  The array is stopped and disk2=no device.  If I try to clear this disk, it gets stuck.  I have a brand new unformatted unassigned disk that has never been part of the array.  I am able to successfully kick off a preclear of this disk.  Did I miss a step or is this not normal?  I'm running unRAID v6.2.4

Try starting the array with the disk unassigned. The disk assignments don't actually get updated until the array is started.

 

That was it.  Thanks!

Link to post

Hi guys, I have some news.

 

Some of the reports you sent are being invaluable to track bugs. One of them is related to "Erase" function not executing properly. Hopefully this is fixed in the current version, so please update your plugin.

 

Another functionality I added is "Erase and Clear the Disk". This is a "partial" fulfillment of dgaschk's request, because it won't intercalate between random and zero data writing. Instead it will erase the disk with random data and then write zeroes on it.

 

In The Statistics Sender plugin, I added a Dismissed Statistics Reports under /Tools. If you dismissed a report and want to send it later, you can go there and submit it later.

 

Well, just a reminder: please keep your plugins updated. Some of the reports I'm receiving are from old versions, and contain some errors that have already been squashed.

 

Link to post

So, what should I do about this one? 

 

All plugins are up to date.  I have closed the browser window, and opened a new one.

 

I do have these entries in the log:

Feb  4 23:18:38 Tower root: error: webGui/include/DeviceList.php: missing csrf_token
Feb  4 23:18:47 Tower root: error: webGui/include/DeviceList.php: missing csrf_token

Screenshot_from_2017-02-06_09-31-01.png.524a897e4596e6197ec20fd0db80deea.png

Link to post

So, what should I do about this one? 

 

All plugins are up to date.  I have closed the browser window, and opened a new one.

 

I do have these entries in the log:

Feb  4 23:18:38 Tower root: error: webGui/include/DeviceList.php: missing csrf_token
Feb  4 23:18:47 Tower root: error: webGui/include/DeviceList.php: missing csrf_token

 

I'm pretty sure you are stuck with an old version. Please remove and reinstall the plugin.

 

Link to post

Trying to use this plugin on 6.3.0 to clear my disks, it gets so far then the web GUI stops updating so can't see any more progress and not sure if it ever actually finishes.

 

First time doing all 5 disks at once it didn't get past the pre-read stage, doing 2 drives it got much further but has frozen during the verify on one drive and suck at pre-read on the other.

HP-NAS-preclear.disk-20170209-0440.zip

Link to post

Trying to use this plugin on 6.3.0 to clear my disks, it gets so far then the web GUI stops updating so can't see any more progress and not sure if it ever actually finishes.

 

First time doing all 5 disks at once it didn't get past the pre-read stage, doing 2 drives it got much further but has frozen during the verify on one drive and suck at pre-read on the other.

 

How much RAM do you have?

 

Link to post

Got a new WD 4TB Red WD40EFRX connected through an LSI Controller.  Pre SMART test (short) looked good. Tried to use the preclear plugin for the first time.  Preclear failed on post-read.

 

Here is the log the plugin uploaded.

 

"Command: /usr/local/emhttp/plugins/preclear.disk/script/preclear_disk.sh --notify 3 --frequency 2 --cycles 3 --no-prompt /dev/sdf
Preclear Disk Version: 0.8.4-beta
S.M.A.R.T. info type: default
S.M.A.R.T. attrs type: default
Pre-Read: dd if=/dev/sdf of=/dev/null bs=2097152  iflag=direct
Pre-Read: dd pid [20863]
Pre-Read: dd - read 4000787030016 of 4000787030016.
: 0
Zeroing: dd if=/dev/zero of=/dev/sdf bs=2097152 seek=1 conv=fdatasync,noerror oflag=direct
Zeroing: dd pid [5328]
Zeroing: dd - wrote 4000784932864 of 4000787030016.
Zeroing: dd command failed, exit code: 1
Post-Read: dd if=/dev/sdf bs=512 count=4096 skip=1 iflag=direct
Post-Read: dd pid [16018]
Post-Read: dd if=/dev/sdf bs=2097152  skip=1 iflag=direct 2>/tmp/.preclear/sdf/dd_output | cmp - /dev/zero &>/tmp/.preclear/sdf/cmp_out 
Post-Read: dd pid [16053]
Post-Read: dd - read 634730315776 of 4000787030016.
Zeroing: 1
Post-Read: fail - disk not zeroed"

 

I noticed the pre-read, zeroing and post read numbers do not match.

 

Pre-Read: dd - read 4000787030016 of 4000787030016.

Zeroing: dd - wrote 4000784932864 of 4000787030016.

Post-Read: dd - read 634730315776 of 4000787030016.

 

I ran another SMART test (extended) and the results look good.

After the extended test, I uninstalled the preclear plugin, and reinstalled it.

Started another preclear using the plugin this morning.

 

With the smart tests still showing good, I am thinking this drive is fine.  If the preclear plugin fails again, I will probably try through the console as I've done in the past.

 

I am not really familiar with the logs, and am not sure how to interpret the results. Any insight would be greatly appreciated.

 

Thanks

Link to post

The system only has 1gb of RAM but it never shows more than 60% utilisation under the dashboard.

 

You WILL need more memory to run 6.3.0!!!  1GB is absolute minimum and even then you will probably have problems.  (As you have found.)  I consider 2GB the absolute minimum for most NAS only servers and would personally recommend 4GB of RAM for any serious installation.  Each preclear operation on a disk requires a sizeable block of memory and 5 of them requires 5 times that amount.  (Discussing memory usage, the various stats reports and their significance with Linux users is akin to the medieval discussion about the number of angels that can sit on the head of a needle.) 

Link to post

I have a failing drive that I'm 3X pre-clearing to see if the attributes will stabilize. Pre-clearing this drive makes the entire unRAID webGUI become unresponsive for hours at a time. If I wait long enough the GUI starts responding again. The drive is on its second pre-read and has over 6000 newly reallocated sectors. I know that this is an unusual case but I'm also certain that it will happen again to someone else. Is there anything that can be done to keep the GUI responsive when working with a bad drive?

 

 

PS. I can send you the drive if it'll be helpful.

Link to post

Thank you, Baby Jesus, for this!

 

I used to dread having to add a drive to my unraid.  I've wait and wait until it was absolutely necessary.  I'd have to re-read how to do it (because I'd forget between pre-clears).  It was always a pain in my butt.

 

This plugin is a dream come true!  Thank you, thank you, thank you...

Link to post

I have a failing drive that I'm 3X pre-clearing to see if the attributes will stabilize. Pre-clearing this drive makes the entire unRAID webGUI become unresponsive for hours at a time. If I wait long enough the GUI starts responding again. The drive is on its second pre-read and has over 6000 newly reallocated sectors. I know that this is an unusual case but I'm also certain that it will happen again to someone else. Is there anything that can be done to keep the GUI responsive when working with a bad drive?

 

 

PS. I can send you the drive if it'll be helpful.

 

This is in /Main or in /Tools/Preclear page?

 

Thank you, Baby Jesus, for this!

 

I used to dread having to add a drive to my unraid.  I've wait and wait until it was absolutely necessary.  I'd have to re-read how to do it (because I'd forget between pre-clears).  It was always a pain in my butt.

 

This plugin is a dream come true!  Thank you, thank you, thank you...

 

You're welcome!

Link to post

I have a failing drive that I'm 3X pre-clearing to see if the attributes will stabilize. Pre-clearing this drive makes the entire unRAID webGUI become unresponsive for hours at a time. If I wait long enough the GUI starts responding again. The drive is on its second pre-read and has over 6000 newly reallocated sectors. I know that this is an unusual case but I'm also certain that it will happen again to someone else. Is there anything that can be done to keep the GUI responsive when working with a bad drive?

 

 

PS. I can send you the drive if it'll be helpful.

 

This is in /Main or in /Tools/Preclear page?

 

 

Main. The GUI is unresponsive now. I entered "smartctl -a -A /dev/sde &" via ssh and the command has not returned. I think it will eventually. It seems to hang until the zeroing is done.

Link to post

I have a failing drive that I'm 3X pre-clearing to see if the attributes will stabilize. Pre-clearing this drive makes the entire unRAID webGUI become unresponsive for hours at a time. If I wait long enough the GUI starts responding again. The drive is on its second pre-read and has over 6000 newly reallocated sectors. I know that this is an unusual case but I'm also certain that it will happen again to someone else. Is there anything that can be done to keep the GUI responsive when working with a bad drive?

 

 

PS. I can send you the drive if it'll be helpful.

 

This is in /Main or in /Tools/Preclear page?

 

 

Main. The GUI is unresponsive now. I entered "smartctl -a -A /dev/sde &" via ssh and the command has not returned. I think it will eventually. It seems to hang until the zeroing is done.

 

When the disk's firmware is performing any task (like remap a bad sector, retry a read etc...)  it will become unresponsive to any query, specially SMART ones. The Preclear plugin page will cache most of the data needed to render, and only probe the temperature every 5 minutes, but not all pages will do the same; Unassigned Devices uses some caching, but  I think it probes the temperature every time. Because of all of that, you have an unresponsive webUI. There isn't a fix for that; either you preclear those drives in other machine or you wait until the preclear is done.

Link to post

Hi there

 

Im trying to run preclear on unRAID 6.3.1 but when I open preclear from tools it seems to be stuck on

 

Please wait... retrieving information!

 

I've also tried running the script in a terminal from /boot/config/plugins/preclear.disk/preclear.sh and it fails saying

 

sfdisk: invalid option -- 'R'
For more details see sfdisk(.
Sorry: Device /dev/sdp is busy.: 1

 

I've also tried clicking the icon on the main page under the unassigned device section, and the button doesn't do anything.

 

I've also tried using a different browser (Chrome instead of Safari)

 

Any ideas, as I'm very keen to start a preclear on a new disk, especially as its 6TB and I'm expecting it will take much longer than I'm used to. Thanks in advance for any help!!

Link to post

Hi there

 

Im trying to run preclear on unRAID 6.3.1 but when I open preclear from tools it seems to be stuck on

 

Please wait... retrieving information!

 

I've also tried running the script in a terminal from /boot/config/plugins/preclear.disk/preclear.sh and it fails saying

 

sfdisk: invalid option -- 'R'
For more details see sfdisk(.
Sorry: Device /dev/sdp is busy.: 1

 

I've also tried clicking the icon on the main page under the unassigned device section, and the button doesn't do anything.

 

I've also tried using a different browser (Chrome instead of Safari)

 

Any ideas, as I'm very keen to start a preclear on a new disk, especially as its 6TB and I'm expecting it will take much longer than I'm used to. Thanks in advance for any help!!

 

Hi.

 

First step is to remove and install this plugin again, to make sure it's in the latest version. The next step is choose which script you want to use to preclear; if you use mine, all you have to do is start the preclear. If you want to Joe L. script, you have to follow the steps on the OP, and then patch the script like described here: http://lime-technology.com/forum/index.php?topic=13054.msg510567#msg510567

Link to post
  • trurl featured, unfeatured and pinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.