(SOLVED) New parity disc = Preclearing included?

[Uno]

Hi,

Does the build of a new parity disc also include the preclearing?

THX

 

 

Edited November 17, 2020 by Uno

[trurl]

Unraid never "preclears". Preclearing is a function provided by some addons that will clear a disk before (pre) adding it to the array.

 

There is only one scenario where Unraid requires a clear disk. This is when ADDING a disk to a NEW data slot in an array that already has valid parity. This is so parity will remain valid. A clear disk is all zeros so has no effect on parity.

 

In that one and only scenario, if a disk isn't already clear when it is added to the new data slot, Unraid will clear it.

 

So there is never any reason to clear parity. Building parity completely overwrites every single bit of the disk with parity, so it doesn't matter at all what was already on the disk.

 

In fact, rebuilding any disk, parity or data, completely overwrites every single bit of the disk with the result of the parity calculation, so it doesn't matter at all what was already on the disk.

 

If you want to test a disk by preclearing it (that is, before assigning it to the array), you will have to use one of the addons.

[Uno]

13 hours ago, trurl said:

So there is never any reason to clear parity. Building parity completely overwrites every single bit of the disk with parity, so it doesn't matter at all what was already on the disk.

In fact, rebuilding any disk, parity or data, completely overwrites every single bit of the disk with the result of the parity calculation, so it doesn't matter at all what was already on the disk.

Do i understand correctly? The process that builds up a new parity disc or rebuilds a failed one, does a big part of the preclearing job? If yes, which part of the PC job is missing and what is the lack of information that comes from this?

 

 

[itimpi]

14 minutes ago, Uno said:

Do i understand correctly? The process that builds up a new parity disc or rebuilds a failed one, does a big part of the preclearing job? If yes, which part of the PC job is missing and what is the lack of information that comes from this?

 

 

No is no clearing involved.    In many ways it is closest to the write zeroes phase of a pre-clear, except that is writing the actual contents rather than zero.   It does not carry out any read phases.   The net effect is to not do the additional stress testing that the pre-clear addin does, but take much less time.

[Frank1940]

1 hour ago, Uno said:

If yes, which part of the PC job is missing and what is the lack of information that comes from this?

Preclear was always a separate job.   The first script to do that operation was developed back when the built-in clearing operation made took the array completely off line until it was completed.  As disk capacity increased in size, that time became a large chunk of a day!  When the script did the preclear, it wrote a special signature that Unraid recognized which allowed it to proceed directly to the formatting operation.  This meant that adding a new data disk to the array now took only the time required to format the disk.  Two to three minutes... 

 

However, it was known that HD disks have a failure mode known as the bathtub curve.  (Just google for explanation!)  Folks found that a hard disk often failed during the preclear operation or (worse!) shortly after being added to the array!  It was soon realized that a rigorous exercising of the HD could detect these early failures with out having the added problems associated with dealing with data recovery if the HD failed shortly after being added to the array.  (It appears that current HD's may have fewer early failures...) 

 

I, personally, always try to put about a hundred hours read-write activity on a disk before I add it to the array.  BUT I has always have a spare disk available to replace any disk that has a problem.  (If a disk goes off line, I replace it, rebuild either data or parity on the spare disk.   At that point, I figure out if the removed disk is defective or a victim of some other event.)  It could be several months to a couple of years before a new spare disk ever makes it into service.  Preclearing to see if the disk has an early problem allows me to use the vendors DOA exchange policy rather than dealing with the manufacturer warranty exchange.  Plus, the warranty could expire before this 'cold' spare was ever put into service!

Edited November 15, 2020 by Frank1940

[Uno]

Thanks, very interesting, so what do you think about this:

• My build's HD'd where in operation in my former NAS for years a failure can occour at any time (but mustn't). Since the theme is new to me, i don't know (yet) wht's the SMART status.
• I simply think to backup my data, attach the HD's to my new Unraid build, let Unraid to his Parity job (without preclearing) and then copy my data to the new Unraid server.
• Of course after all this, a new Backup has to been made und since the build in SMART surveillances will give me infos ragarding the condition of my HD's, i will keep an eye on this.
• And since Parity is a great ting (i really didn't know about) as soon as a HD fails, i will replace it with a new one. And yes, this one will be precleared before in order to check out some DOA problems.

THX

[Frank1940]

You have a reasonable plan.  Older disks (that are currently OK) have a very low failure rate until they reach the end-of-life portion of the bathtub curve.  Unraid will still do its clearing operation when they are added to the array.  (Having said that, I can not remember what happens when a new array is first started but I suspect that only parity is built in that particular case.) 

[trurl]

2 hours ago, Uno said:

the build in SMART surveillances will give me infos ragarding the condition of my HD's, i will keep an eye on this.

Set up Notifications and it will tell you if there are problems by email or other agent.

 

Bad connections are a much more common cause of problems than bad disks so pay close attention to that. A bad connection will cause a disk to be out-of-sync and have to be rebuilt.

[Uno]

Well, before i start: What happens, if the Parity disk fails and can the PD become to small (I knos, it has to be bigger as the biggest disk)...

[trurl]

41 minutes ago, Uno said:

if the Parity disk fails and can the PD become to small (I knos, it has to be bigger as the biggest disk)...

If you mean it fails in some way that makes it have less capacity of course it must be replaced and can't imagine why you would consider using it again for anything 

[Frank1940]

50 minutes ago, Uno said:

Well, before i start: What happens, if the Parity disk fails and can the PD become to small (I knos, it has to be bigger as the biggest disk)...

Is this the situation you are asking about?  You have a Unraid server with a 8TB parity drive and the largest data disk is a 4TB disk.  What you want to know is can you replace the failed 8TG parity disk with a 6TB drive?  The answer is yes.  Thw only requirement is that the parity disk(s) be larger than the largest data disk.  (As a point of information, with a system where the largest data disk is 4TB and the parity disk is 8TB, you can add a second parity disk having any size larger than 4TB.   (The second parity disk does not have to be 8TB...)

Edited November 15, 2020 by Frank1940

[trurl]

48 minutes ago, Frank1940 said:

Is this the situation you are asking about?

That does make more sense.

[Uno]

I was wondering about two things:

• when will a paritiy disk be full and must be replaced with a bigger one?
  • There's no indication about in the "main" menue...
• will a failed parity disk be rebuilt once it is replaced by a new one?

I think, now i asked in a better way, thanks.

[Frank1940]

6 minutes ago, Uno said:

when will a paritiy disk be full and must be replaced with a bigger one?

Here is a description of how parity works:

 

     https://wiki.unraid.net/index.php/UnRAID_Manual_6#Parity-Protected_Array

 

It is a simple XOR operation on performed on every byte of all the data disks and the result for each byte on stored on the corresponding byte on the parity disk.  So the maximum number of parity bytes needed is equivalent to the number of bytes on the largest data disk.

 

10 minutes ago, Uno said:

will a failed parity disk be rebuilt once it is replaced by a new one?

Yes

[trurl]

14 hours ago, Uno said:

when will a paritiy disk be full

Parity disk is always full because all of parity is always used. If you want to have a data disk larger than parity then parity must be replaced with a larger one since no data disk can be larger than parity.

[Uno]

All clear, THX