Understanding when to Parity rebuild?

[je82]

Hi, i just want to make sure i understand this correctly.

 

Pairity rebuild is only needed when you replace a disk to an already configured array or create a new array.

 

If you add another drive to an array (expand the array) rebuilding the parity is not needed.

 

Is this correct?

[itimpi]

1 minute ago, je82 said:

when you replace a disk

A parity rebuild is not required for this.

 

The only time you need to do a parity rebuild is when setting up a new array, or when you use the New Config tool (which puts the array into the same state as on a new array).

[je82]

7 minutes ago, itimpi said:

A parity rebuild is not required for this.

 

The only time you need to do a parity rebuild is when setting up a new array, or when you use the New Config tool (which puts the array into the same state as on a new array).

Okey, problem is, i tried to add a drive to my already configured array and only way to successfully do it was to do "new config" then keep the settings for the old drives.

 

If i didnt do new config and adding the unassigned device to the array unraid would not let me start the array. How exactly do you expand your array without doing new config?

 

Note: during the time i tried to add a new device my parity drives were not completely built, was this the reason i could not add the device without doing new config?

[itimpi]

1 hour ago, je82 said:

Note: during the time i tried to add a new device my parity drives were not completely built, was this the reason i could not add the device without doing new config?

Quite likely!    Since you have to stop the array to add the new drive it would not be normal to even attempt this if currently building parity!

 

I must admit I have never tried the scenario of adding some parity drives and then before parity is built trying to add an array disk.   I might try this on mu test system to confirm the behaviour.

Edited October 20, 2019 by itimpi

[Frank1940]

57 minutes ago, je82 said:

Note: during the time i tried to add a new device my parity drives were not completely built, was this the reason i could not add the device without doing new config?

I would think so.  The adding process described in the manual assumes that parity is already correct.  The fact that is was not would certainly put things into an area where most of us have never been.  See here for that manual procedure:

 

        https://wiki.unraid.net/UnRAID_6/Storage_Management#Adding_disks

 

Think about it.  Doing a parity check/build requires reading ALL of the data disks in the array to do the required calculations. (With a check the data is compared to what is on the parity disk(s) and with a build, existing parity is overwritten.)  What is it going to so when you try to add a other disk to the array when it has alright partially built parity for all of the disks EXCEPT for the one you are now adding?

 

I always try to caution people to make only one change at a time to a system.  That way should you have a problem, your investigation can center on what you were doing at that time. 

[je82]

3 minutes ago, itimpi said:

Quite likely!    Since you have to stop the array to add the new drive it would not be normal to even attempt this if currently building parity!

 

I must admit I have never tried the scenario of adding some parity drives and then before parity is built trying to add an array disk.   I might try this on mu test system to confirm the behaviour.

Yeah i have setup unraid in a test lab with crap stuff i had laying around, just trying to understand my way around it completely before i migrate my actual real stuff over to it.

 

I still have no idea what kind of wizard you have to be to make this kind of software. I setup a smb share which exclusively put data on one of the drives and ignoring the other, after i had sent 400gb and the parity was built i simulated that the drive crashed by disconnecing it via sata cable, suprirsingly all the data was still available via smb, i have absolutely no idea how that is possible.

[Frank1940]

6 minutes ago, je82 said:

I still have no idea what kind of wizard you have to be to make this kind of software. I setup a smb share which exclusively put data on one of the drives and ignoring the other, after i had sent 400gb and the parity was built i simulated that the drive crashed by disconnecing it via sata cable, suprirsingly all the data was still available via smb, i have absolutely no idea how that is possible.

IF you had a valid parity, the disk was being emulated.   (By using the parity disk and remaining data disk(s) to caalculate the data on the missing disk.) 

 

See here:

    https://wiki.unraid.net/index.php/UnRAID_Manual_6#Parity-Protected_Array

Edited October 20, 2019 by Frank1940

[je82]

Just now, Frank1940 said:

IF you had a valid parity, the disk was being emulated.   (By using the parity disk and remaining data disk(s) to recover that the data on the missing disk.) 

 

See here:

    https://wiki.unraid.net/index.php/UnRAID_Manual_6#Parity-Protected_Array

Yeah but how can it emulate it on the fly? I though it had to be "rebuilt" using the parity, not that the data would be instantly available via emulation, it's actually very impressive. I also do not understand if i have 2x 12tb parity drives, and then i have an over 100tb array, how can the parity devices of 24TB total cover all that data? To a mortal like myself it simply does not makes sense, but i thank you all for developing this incredible piece of software, i will be purchasing 2 full licences very soon!

[Frank1940]

Read it and you will see that it can instantly emulate the data on a 'missing' disk.  Since the capability is there, Unraid just does it.  It a actually a real great feature.  Many folks use that capability to have access to data until a replacement disk arrives. 

[JonathanM]

6 minutes ago, je82 said:

I also do not understand if i have 2x 12tb parity drives, and then i have an over 100tb array, how can the parity devices of 24TB total cover all that data?

It doesn't, it only covers 2 drives, but any 2. All the rest of the drives are used as well. If you have more than 2 failures, 1 for each parity, then you will lose the data on all the failed disks.

 

Parity doesn't hold any data, only the answers to the equation formed by all the other drives. All remaining drives are used to emulate the missing one, or two if you have dual parity.

[itimpi]

17 minutes ago, jonathanm said:

Parity doesn't hold any data, only the answers to the equation formed by all the other drives. All remaining drives are used to emulate the missing one, or two if you have dual parity.

These are the important points!  In addition you need to realize that parity works at the physical sector level and has no understanding of what the contents of that sector might mean - the sector contents are just treated as a bit pattern. 

 

If a drive is missing, then Unraid can read a particular sector number off all the data drives and the parity drive.  It can then calculate what must have been in the corresponding sector on the missing drive to give the result for that sector that is currently stored on the parity drive. 

[Frank1940]

Quick example using regular arithmetic.

 

   P =  A + B + C + D  + E  Where 'P' is our parity drive and 'A', B', ... ''E' are our data drives

 

and we have the following condition (Where Drive 'C' has failed after the 'P' was initially calculated):

 

  102 = 12 + 23 + C  + 47 + 7   

What is the value of 'C', you should be able to determine that it is 13 using simple arithmetic (or even trial-and-error).  Now what happens if we have have this:

 

  98 = 21 + B + 32  + D + 17

 

We can't solve this problem as there is a range of values for B and D which would provide the correct sum.

 

[JorgeB]

18 hours ago, je82 said:

Note: during the time i tried to add a new device my parity drives were not completely built, was this the reason i could not add the device without doing new config?

You can't sync parity and add a new disk at the same time, though the error Unraid gives in that scenario is not correct.