[SOLVED] New build - Parity check after preclearing all drives?

[Omid]

Greetings all,

 

I'll describe what I did in simple steps (as to not confuse anyone; more importantly myself ).

tl;dr: see last 2 lines of post

[*]Finished putting together my build 2 weeks ago

[*]Put 4x WD RED 2TBs in and had them all precleared over the last week (3 cycles, 2 drives at a time)

[*]Installed unMENU and SimpleFeatures

[*]Added Pro2.key to ./config/

[*]Opened up web GUI (SimpleFeatures)

[*]Selected parity and disks 1, 2 & 3 on the dropdowns

[*]Clicked start array

...it's now calculating parity.

Parity-Sync in progress.

Estimated finish: 4 hours, 2 minutes

 

Is this right? I was under the impression preclearing will eliminate this long wait and essentially skip the parity-sync. All drives are 0'ed out, no?

Just a tad confused, sorry. I'm sure I've missed/not read something.

 

Thanks,

Omid.

[itimpi]

The pre-clear eliminates the need to re-calculate the parity when adding the drive as a new data disk to an existing array.  It does not eliminate the need to do the parity calculation when you first add the parity disk.

[Omid]

Just the type of answer I was looking for. Many thanks, itimpi!

I'll leave it running then; let it do its thing!...

[garycase]

Just the type of answer I was looking for. Many thanks, itimpi!

I'll leave it running then; let it do its thing!...

 

In addition to waiting for it to calculate parity; you should run a parity check after it's completed -- this will also take a long time.    This confirms everything went well with the parity calculations.    After that, a parity check ~ once/month is all you need to do.

 

[Omid]

Thanks, Gary.

So, that initial parity sync is different to a parity check (monthly/manually triggered)?

 

I waited until parity sync finished then spotted all my data disks are "unformatted".

I formatted them (hope this was right) and have the array running 

 

There's only one thing that's worrying me a little...

This is showing in unMENU

Parity is Valid:.  Last parity check  15851  days ago with no sync errors.

and in SimpleFeatures

Parity is valid. Parity has not been checked yet.

^ the above statement being slightly contradictory, no? 

Must relate to what you've said - I need to run a parity check.

 

And yes, I've set up monthly parity check through SimpleFeatures. Seen it add to the crontab, so hope that's it.

 

I think I'm spending the rest of this weekend playing with this. Pretty amazing so far!

[Frank1940]

You have surmised correctly--  A parity sync is NOT a parity check.  I would suggest that you run a non-correcting parity check some time in the next couple of days.  Do it overnight while you are not using the server since it takes roughly the same amount of time as the parity sync.  If all is well, it will finish with zero errors and 'fix' the problem with 'Parity has not been checked' and the date since the last parity check should then be correct.  (The time shown is elapsed time from the base date of Linux since there has never been a parity check and is stored time value for the parity check is 'zero'.)

[Omid]

Ah yes, how lazy of me. Definitely the correct number of days since unix epoch

 

Thanks, I'll run a non-correcting parity check tonight.

Anyone out there care to explain the difference between sync and check?

[Joe L.]

Ah yes, how lazy of me. Definitely the correct number of days since unix epoch

 

Thanks, I'll run a non-correcting parity check tonight.

Anyone out there care to explain the difference between sync and check?

sync = calculate parity based on READING ALL DATA DISKS and WRITING calculated values to parity disk.

check = calculate parity based on reading all data disks and then compare calculations to values READ from parity disk.  (it should match, if not, then parity is incorrect)

 

sync = writing parity

check = reading parity

 

Joe L.

[Omid]

Think I've got it. To clarify...

 

sync will always overwrite parity.

check will verify against parity (read only), but will write if you tick that "Write corrections to parity disk" box?

 

Omid.

[mrow]

 

Think I've got it. To clarify...

 

sync will always overwrite parity.

check will verify against parity (read only), but will write if you tick that "Write corrections to parity disk" box?

 

Omid.

 

 

You've got it.

[garycase]

As noted, you've got it down pat.

 

One note:  If you choose to run a "non-correcting" parity check, and happen to have a non-zero result, in most cases that simply means you need to run a correcting parity check to fix it.  UnRAID always assumes that a parity error during a check is an error on the parity disk -- NOT an error on one of the data disks.

 

In my experience, this has ALWAYS been correct.    On the few occasions when I've had a non-zero result, I've run a complete comparison of all the data on my array against my backups -- and have NEVER had any differences in the data.    Personally, I always just run a correcting check.

 

[Omid]

Awesome, thanks.

[henris]

While the initial parity sync is running, should I do anything regarding the unformatted data drives? In my case I'm testing with only two pre_cleared drives, one for parity and one data. When I started the array for the first time it started to perform parity sync as described above in this thread. However it also marked the data disk as Unformatted (parity disk status is "-") and the UI presents the option to format any unformatted drives ("Format will create a file system in all Unformatted disks, discarding all data currently on those disks. -> Yes I want to do this"). I think this was not visible before I started the array but I cannot swear to it.

 

My logic says don't touch anything while the initial parity sync but the Format option is present and I could click it. If formatting is not allowed during parity sync then the option should not be visible/enabled imho.

 

Can anyone clarify this?

[garycase]

While the initial parity sync is running, should I do anything regarding the unformatted data drives? In my case I'm testing with only two pre_cleared drives, one for parity and one data. When I started the array for the first time it started to perform parity sync as described above in this thread. However it also marked the data disk as Unformatted (parity disk status is "-") and the UI presents the option to format any unformatted drives ("Format will create a file system in all Unformatted disks, discarding all data currently on those disks. -> Yes I want to do this"). I think this was not visible before I started the array but I cannot swear to it.

 

My logic says don't touch anything while the initial parity sync but the Format option is present and I could click it. If formatting is not allowed during parity sync then the option should not be visible/enabled imho.

 

Can anyone clarify this?

 

Formatting the disk during the parity sync will work -- but as your instinct tells you, it's not something I'd do.    Doing so causes the disk involved to thrash while it's involved in both the parity sync and formatting ... which slows down both the format operation and the parity sync.    I'd just wait for the parity sync to complete;  then do the format operation.    It's NOT necessary to do this -- but it just seems "kinder" to the disk 

 

... but you can actually use the array all you want during the parity sync -- format the disk; copy files to it; stream files from it; etc.    I just don't do those things, as they simply make the parity sync take longer AND stress the drives more.    Some folks like to do this just as a "stress test".  [i.e. do a parity check; stream 3 or 4 movies;  write data to the array; etc. all at once just "because you can"]