Is there a way to simulate a failure with a known drive?

[Syndacate]

Hi,

I'm using the trial of 6.10.3 and I wanted to simulate a drive failure.
Right now I have 1 1TB drive + 1 1TB parity.  Initially I had only the former, and then I added the parity and it built the parity - both drives had the same data when individually removed & checked, it was a pseudo-RAID1, everything was good.  So that essentially tests a failure one direction.

I wanted to see it rebuild the array though, so I removed DRIVE1, formatted it, popped it back in hoping to get prompted for rebuild, which didn't happen.  It then hit me that the problem is likely that the unique identifier (mode+serial?) is known to the config and therefore maybe it's the case that DRIVE1 without any data is now the "source of truth" and the "valid data" on the parity drive is interpreted to just be a drive full of parity errors?  It indeed shows many errors.

I did some Googling and I saw what I thought was a solution, which was to:
1.  Remove the parity
2.  Up/down
3.  Remove the DRIVE1
4.  Tools -> New Config
5.  Add DRIVE1 + Parity back in their respective slots
5.  Check "Parity is valid"
6.  Up

Didn't quite do what I'd hoped, drive shows empty, but with parity errors (I did not have it do parity corrections).

So I think this is just an invalid test because the unique identifier is the same, do I have the correct interpretation that if a different drive with a previously unknown identifier was put into SLOT1 it would prompt me to rebuild based on the parity data?

And then it would behave similarly if I was to add more 1TB drives o the array, correct?  Always prompting for a rebuild when a previously unknown drive is inserted into a slot which is assigned to a different unique identifier?

Then as a side question, without deviating too much from the topic:  Is the parity file level or drive level?

Thanks!

Edited August 8, 2022 by Syndacate

[trurl]

All you have to do to get Unraid to rebuild a drive from the parity calculation, no matter how many drives you have, or which drive it is (even parity drive)

1. Stop array
2. Unassign drive to be rebuilt
3. Start array with drive unassigned
4. Stop array
5. Reassign drive to be rebuilt
6. Start array to begin rebuild

[trurl]

1 hour ago, Syndacate said:

Is the parity file level or drive level

drive level, in the sense that it can only rebuild a drive.

 

It doesn't know anything about files.

 

But, it really doesn't know anything about drives either. It is really about all the drives together.

 

It is all just bits. Parity is just an extra bit that allows a missing bit to be calculated from all the other bits.

 

https://wiki.unraid.net/Manual/Overview#Parity-Protected_Array

[trurl]

All New Config does is allow you to assign drives however you want, then optionally (by default) rebuild parity onto any drive assigned to parity slots, based on the contents of all the drives assigned to data slots. It doesn't do anything to any drives assigned to data slots.

 

Not entirely sure what you were trying to do with your experiment.

[Syndacate]

On 8/7/2022 at 7:10 PM, trurl said:

All New Config does is allow you to assign drives however you want, then optionally (by default) rebuild parity onto any drive assigned to parity slots, based on the contents of all the drives assigned to data slots. It doesn't do anything to any drives assigned to data slots.

 

Not entirely sure what you were trying to do with your experiment.

What is different when the "parity is valid" checkbox is checked?  Does it just skip the parity rebuild?  I assumed it meant it would use parity as a source of truth and trigger a rebuild on DATA1, but it didn't.  That assumption doesn't make sense in hindsight though, because if there were multiple drives in the array and all unique identifiers in all the slots were the same, it wouldn't know which DATA drive I had wanted to "rebuild", I'd imagine.

Do note, there was just random files on there, I did this method without digging much into it knowing it may blow everything away and was OK with it.

The experiment was to just trigger a rebuild.  After I formatted the DATA1 drive, there is an obvious mis-match, which I wanted to see it rectify.

Is the "New Config" usage common?  If I add drives I thought I'd just need to assign the new drives to the array.
What's the difference between using "New Config" to re-assign versus just unassigning/assigning as preferred then spinning up and hitting "check" with "write corrections to parity" set?
 

On 8/7/2022 at 7:04 PM, trurl said:

drive level, in the sense that it can only rebuild a drive.

 

It doesn't know anything about files.

 

But, it really doesn't know anything about drives either. It is really about all the drives together.

 

It is all just bits. Parity is just an extra bit that allows a missing bit to be calculated from all the other bits.

 

https://wiki.unraid.net/Manual/Overview#Parity-Protected_Array

I was just surprised to see the parity drive was mountable and all files readable, I expected it to be just garbage without the context of the data drive.  But I guess when there's only 1 DATA + 1 parity and there's even parity it becomes a mirror automatically, allowing the valid FS to be mirrored?  Am I right in assuming if there were more drives it'd just be garbage & no valid FS on the parity?
 

On 8/7/2022 at 7:02 PM, trurl said:

All you have to do to get Unraid to rebuild a drive from the parity calculation, no matter how many drives you have, or which drive it is (even parity drive)

1. Stop array
2. Unassign drive to be rebuilt
3. Start array with drive unassigned
4. Stop array
5. Reassign drive to be rebuilt
6. Start array to begin rebuild

I'll give this a whirl, thanks.  

[trurl]

4 hours ago, Syndacate said:

Does it just skip the parity rebuild?

yes, you are telling it to not rebuild parity.

 

4 hours ago, Syndacate said:

I assumed it meant it would use parity as a source of truth and trigger a rebuild on DATA1, but it didn't

New Config does nothing to any data drives. That is sort of the whole point of New Config, to take your data disks just as they are and create a new array from them. Rebuilding parity (and parity2) would be required if any disks were added or removed. Rebuilding parity2 would also be required if disks were reordered.

 

4 hours ago, Syndacate said:

Is the "New Config" usage common?  If I add drives I thought I'd just need to assign the new drives to the array.

If you add a drive to a new data slot in an array that already has valid parity, Unraid will clear the drive in the new data slot unless it has been precleared. This is so parity will remain valid, since a clear drive is all zeros which has no effect on parity. After the drive is clear Unraid will let you format it. This is the normal way to add drives.

 

If instead of adding, you replace a drive in the array, Unraid will rebuild the drive from the parity calculation. Replacing either parity drive works the same as replacing a data drive. Of course, a replacement drive must be at least as large as the drive it is replacing, and no data drive can be larger than any parity drive.

 

And removing a drive from the array will require New Config with parity rebuild, since parity won't be in sync without the bits from the removed drive.

 

4 hours ago, Syndacate said:

parity drive was mountable and all files readable, I expected it to be just garbage without the context of the data drive.  But I guess when there's only 1 DATA + 1 parity and there's even parity it becomes a mirror automatically, allowing the valid FS to be mirrored?  Am I right in assuming if there were more drives it'd just be garbage & no valid FS on the parity?

Yes, that is just how it works out when there is only 1 data drive, the parity calculation results in a mirror. If there is more than one data drive, parity isn't mountable because it has no filesystem to mount.

 

 

Sometimes New Config with valid parity can be a way to make Unraid leave everything as is so you can force it to rebuild a data disk that it doesn't think is supposed to need rebuilding. There are some extra steps involved with this so don't try it without advice.

 

 

[trurl]

https://wiki.unraid.net/Manual/Tools#New_Config

[Syndacate]

On 8/9/2022 at 8:32 AM, trurl said:

New Config does nothing to any data drives. That is sort of the whole point of New Config, to take your data disks just as they are and create a new array from them. Rebuilding parity (and parity2) would be required if any disks were added or removed. Rebuilding parity2 would also be required if disks were reordered.

On 8/9/2022 at 8:32 AM, trurl said:

New Config does nothing to any data drives. That is sort of the whole point of New Config, to take your data disks just as they are and create a new array from them. Rebuilding parity (and parity2) would be required if any disks were added or removed. Rebuilding parity2 would also be required if disks were reordered.

...

If you add a drive to a new data slot in an array that already has valid parity, Unraid will clear the drive in the new data slot unless it has been precleared. This is so parity will remain valid, since a clear drive is all zeros which has no effect on parity. After the drive is clear Unraid will let you format it. This is the normal way to add drives.

 

If instead of adding, you replace a drive in the array, Unraid will rebuild the drive from the parity calculation. Replacing either parity drive works the same as replacing a data drive. Of course, a replacement drive must be at least as large as the drive it is replacing, and no data drive can be larger than any parity drive.

 

And removing a drive from the array will require New Config with parity rebuild, since parity won't be in sync without the bits from the removed drive.

On 8/9/2022 at 8:32 AM, trurl said:

https://wiki.unraid.net/Manual/Tools#New_Config

Gotcha, OK, makes sense.  I think I understand!
 

On 8/9/2022 at 8:32 AM, trurl said:

Yes, that is just how it works out when there is only 1 data drive, the parity calculation results in a mirror. If there is more than one data drive, parity isn't mountable because it has no filesystem to mount.

 

 

Sometimes New Config with valid parity can be a way to make Unraid leave everything as is so you can force it to rebuild a data disk that it doesn't think is supposed to need rebuilding. There are some extra steps involved with this so don't try it without advice.

Gotcha, yeah I'll leave it alone, planning on doing everything as standard as possible, not getting fancy with it.
 

On 8/9/2022 at 3:43 AM, Syndacate said:

I'll give this a whirl, thanks.  

So I gave that a try and it worked as expected, rebuilt the drive and was usable in the meantime.

Thanks for your help!

Edited August 16, 2022 by Syndacate