Question. I want to add my previous parity drive into the array as a data drive.

Do I need to preclear it, and if so, is there a way to speed this up as I already know the drive is solid as it's worked flawlessly for 3 years.

 

Also, is it possible to change multiple data drives at once and then do one re-build from parity, or can I only change 1 drive, re-build from parity, change another drive, re-build from parity...?

 

Thanks.

Edited August 26, 20178 yr by johnieutah

13 minutes ago, johnieutah said:

Question. I want to add my previous parity drive into the array as a data drive.

Do I need to preclear it, and if so, is there a way to speed this up as I already know the drive is solid as it's worked flawlessly for 3 years.

 

If your going to use it to replace a smaller drive no, if adding to the array you just clear it using the GUI.

 

14 minutes ago, johnieutah said:

Also, is it possible to change multiple data drives at once and then do one re-build from parity, or can I only change 1 drive, re-build from parity, change another drive, re-build from parity...?

 

You can change 2 at the same time if you have dual parity, with single parity just one at a time.

13 minutes ago, johnieutah said:

Question. I want to add my previous parity drive into the array as a data drive.

Do I need to preclear it, and if so, is there a way to speed this up as I already know the drive is solid as it's worked flawlessly for 3 years.

 

Also, is it possible to change multiple data drives at once and then do one re-build from parity, or can I only change 1 drive, re-build from parity, change another drive, re-build from parity...?

 

Thanks.

It is not ever necessary to pre-clear a drive that is being added as a new data drive or replacing an existing one.   The preclear is just a confidence test of the drive.   If adding it as a new drive, UnRAID will carry out a clear operation (equivalent to the write phase of pre-clear) when adding a drive that is not pre-cleared.   If replacing a drive then the rebuild process overwrites every sector anyway.

 

if replacing a drive then the maximum number of drives that can be replaced at once is the same as the number of parity drives you have.

 

 

1 minute ago, johnnie.black said:

 

If your going to use it to replace a smaller drive no, if adding to the array you just clear it using the GUI.

 

 

You can change 2 at the same time if you have dual parity, with single parity just one at a time.

When you say clear it using the GUI, you mean the standard Unraid process for a swapping out a data drive.

7 minutes ago, itimpi said:

It is not ever necessary to pre-clear a drive that is being added as a new data drive or replacing an existing one.   The preclear is just a confidence test of the drive.   If adding it as a new drive, UnRAID will carry out a clear operation (equivalent to the write phase of pre-clear) when adding a drive that is not pre-cleared.   If replacing a drive then the rebuild process overwrites every sector anyway.

 

if replacing a drive then the maximum number of drives that can be replaced at once is the same as the number of parity drives you have.

 

 

So basically there are no short cuts I can take here. I simply upgrade the drive as one normally would and let Unraid rebuild from Parity?

 

Thanks guys.

Just now, johnieutah said:

When you say clear it using the GUI, you mean the standard Unraid process for a swapping out a data drive.

So basically there are no short cuts I can take here. I simply upgrade the drive as one normally would and let Unraid rebuild from Parity?

 

Thanks guys.

It is worth pointing out that you never ‘rebuild from parity’ in isolation.     What you are doing is rebuilding from parity + the good data drives.   That is why the number of drives that can be rebuilt is limited to the number of parity drives because you also need the other data drives to be good for the rebuild to succeed. 

1 hour ago, johnieutah said:

When you say clear it using the GUI, you mean the standard Unraid process for a swapping out a data drive.

 

No, there's no clearing during when replacing an existing data disk, only when adding a new disk to the array.

6 minutes ago, johnnie.black said:

 

No, there's no clearing during when replacing an existing data disk, only when adding a new disk to the array.

Ok thanks. I must have misunderstood as I thought that the preclear process also added some sort of "signature" to the drive, Unraid then recognises this... Parity drives don't have this signature I thought... Anyways, like I said I must have misunderstood the way Unraid works.

 

Basically I just need to swap out the 750GB drive with the 3TB red and let Unraid do its thing.

 

8 minutes ago, johnieutah said:

Ok thanks. I must have misunderstood as I thought that the preclear process also added some sort of "signature" to the drive, Unraid then recognises this... Parity drives don't have this signature I thought... Anyways, like I said I must have misunderstood the way Unraid works

The preclear process DOES add a signature.   However this is only relevant when adding a new drive.   In the absence of this signature unRAID carries out its own 'clear' process to set the disk to all zeroes.   As was mentioned this is equivalent to the 'write' phase of pre-clear so only takes about 30% of the time of the full preclear process.

 

When replacing a disk the existing contents are irrelevant as the rebuild process overwrites every sector anyway.   The only requirement is that the disk is 'good' so that the writes of the rebuild process complete without error.

Thanks for the replies.

The array  upgrade process seems to be going (touch wood) smoothly after everything from the preclear went fine.

 

One final question: during a rebuild of a data disk, it looks like the contents of the drive are already there, even though it's less than 10% through reconstructing from parity. Shouldn't Unraid deny access to any shares where the data is on a drive which is currently on-going a data rebuild?

 

Thanks!

1 hour ago, johnieutah said:

during a rebuild of a data disk, it looks like the contents of the drive are already there, even though it's less than 10% through reconstructing from parity. Shouldn't Unraid deny access to any shares where the data is on a drive which is currently on-going a data rebuild?

No, why should it? Any changes made to the files will be done real time, same as if it was emulated, during rebuild, or normally. It will slow down the rebuild though.

So I guess the question is more, how can the drive contents be shown if the rebuild is no where near finished?

1 minute ago, johnieutah said:

So I guess the question is more, how can the drive contents be shown if the rebuild is no where near finished?

 

You're seeing (and using) the emulated disk, only when the rebuild finishes it switches to the actual disk.

Got it. So the contents can be see but not necessarily read (e.g. music can't be played)?

1 minute ago, johnieutah said:

Got it. So the contents can be see but not necessarily read (e.g. music can't be played)?

No.  An emulated disk can operate exactly like an unemulated version.   Therefore both reading and writing are supported.   The only limitation is that while rebuilding performance can be impacted, and until the rebuild completes ant new data is not protected.

When unRAID rebuilds a disk, it reads parity and all the other disks and uses this data to calculate the data of the missing disk.

 

When you read an emulated disk, it does the exact same thing, calculating the missing data from parity and all the other disks.

 

When you write an emulated disk, it reads parity and all the other disks, calculates the parity change that would happen due to the write, and writes parity. So even writing when a disk is missing or being rebuilt, still results in data that can be calculated by reading parity and all the other disks.

 

Have you seen the wiki about how parity works? It isn't very complicated and unRAID makes a lot more sense if you understand parity.

 

https://wiki.lime-technology.com/UnRAID_6/Overview#Parity-Protected_Array

 

 

16 hours ago, trurl said:

Have you seen the wiki about how parity works? It isn't very complicated and unRAID makes a lot more sense if you understand parity.

 

https://wiki.lime-technology.com/UnRAID_6/Overview#Parity-Protected_Array

 

 

Good explanation, thanks.

 

So rebuild is plodding along at only ~30MB/s!  Just read around and apparently the HP Gen. 8 that I have has a write cache disabled by default in the bios. Will fix that after everything is completed.

On 8/13/2017 at 7:49 PM, Vibe said:

Thank you BobPhoenix - I greatly appreciate it.  I see what I did wrong - I mistakenly thought that bjp999's version (preclear_bjp.sh from the mediafire.com link) had the modification for v6.2 already.  However, after doing a file comparison with preclear_disk_15c.sh that you pointed out, I see where the required change needs to be made (line #968).  I am also seeing the changes added for notifications.  In the meantime I am trying one more attempt using the default gfjardim-0.8.9-beta script to test a new hard drive I received yesterday (currently pre-read at 50% @ 194MB/sec).  If this fails I will I will try the versions you mentioned (with modification on bjp999's script) to see if I am successful.  If I can get this to work as intended, the convenience will be a tremendous addition to a fantastic script.

 

Thanks again for steering me in the right direction!

Did you have any success with subsequent attempts on your new drive or HGST 4TB drives using the plugin with default gfjardim-0.8.9-beta script?

 

What about with a modified script run from the command line?

 

One thing that would be hugely helpful is posting with any modified working preclear scripts that can be run from the command line in unRAID v6.

 

Thanks!

My latest preclear of a Western Digital 8 TB WDC_WD80EFAX got stuck at post read 94%

 

I'm using Preclear Disks 2017.07.10, Unassigned Devices 2017.07.09c and unRAID 6.3.5

 

Attached are the logs. 

 

I had a similar failure with a WD 8TB Parity WDC_WD80EFZX using an earlier version of the plugin.

 

I noted back in July that based on several similar posts the WDC_WD80 seemed problematic with the preclear plugin.  I don't know if there have been any posts noting success with the plugin and these drives since.

 

My last successfull preclear was on a Seagate Archive 8 TB ST8000AS0002 using Preclear Disks 2017.07.05c, Unassigned Devices 2017.07.04a and unRAID 6.3.3

 

Also please note the below previous recent posts reporting similar behavior with HGST 4TB and 8TB as well as a Hitachi HDS5C4040ALE630 4TB.

 

13-AUG-2017 Vibe HGST 4TB

13-AUG-2017 JarDo Hitachi HDS5C4040ALE630 4TB

16-AUG-2017 bbcorn88 8TB HGST

 

raid_01-diagnostics-20170830-0912.zip

RAID_01-preclear.disk-20170830-0859.zip

Edited August 30, 20178 yr by dabl
formatting

7 minutes ago, dabl said:

One thing that would be hugely helpful is posting with any modified working preclear scripts that can be run from the command line in unRAID v6.

 

Se here for a the changes to @Joe L.   original preclear script:

EDIT:  be sure to read the next several posts as Joe had a problem with the variables.  See here for his update:

 

Edited August 30, 20178 yr by Frank1940

1 hour ago, Frank1940 said:

Se here for a the changes to @Joe L.   original preclear script:

EDIT:  be sure to read the next several posts as Joe had a problem with the variables.  See here for his update:

 

Thanks.  

 

 

Edited August 30, 20178 yr by dabl

Just now, dabl said:

Again, it would be hugely helpful if some kind soul who slogged through all the posts and got a working version could simply post it.

 

This had been done in the past but the folks who did the original development work on these scripts took considerable offense to the practice of modifying their scripts and then reposting the modified script.  (And there are actually several excellent reason for their feeling this way...)

I see that the gfjardim script has a lot more options than it did the last time I used it. Which option is best for preclearing a new drive? I'm guessing that erasing is probably not needed, right?

I have two "dismissed" reports that just won't send... does anyone else have the same issue? I've been trying to send them over the last few days/week... and they just won't send... all the other reports I've generated from different precleared disks have been sent.

 

I've attached screenshots of the syslog (my IP removed)...and the "sending statistics" screen...

 

Nothing else happens...it just sits there... and never sends...

 

 

I'm having an issue where drives will only appear in UD if I delete this plugin. It has happened with at least one drive, possibly two. Would be glad to furnish any logs if necessary.

8 minutes ago, wgstarks said:

I'm having an issue where drives will only appear in UD if I delete this plugin. It has happened with at least one drive, possibly two. Would be glad to furnish any logs if necessary.

I am going to look into a configurable option in UD that will default to not using the preclear background task that is causing this problem.  It could be turned on to help with preclear issues, but would then potentially cause this problem.  I will see if this is feasible and makes sense.