Can I survive from 2 Disks Missing, 1 Disabled ?

[georgez]

Hi,

 

I'm having a serious problem and really need help! My array lost a disk couple of days ago (the disk was disabled), but since I have 2 parity disks, I didn't replace it immediately but kept using for some more days. Today, my new disk arrived, I was trying to replace the bad one, but a good disk dropped on the ground by accident, then when I put all disks back into array (the disabled was replaced with a new disk), I was surprised to find that 2 disks were missing (one is the old disabled disk which is expected, another is the dropped one), 1 was disabled. I was confused about the disabled one 'cos it was working fine before last shutdown. I tried to mount it manually and seemed everything's ok, SMART quick test also done without error, so I think it's just fine but it's still disabled.

 

Now I'm in a situation that 2 disks are missing, 1 is disabled. Can some one enlighten me the correct way to handle this? Diagnostic data attached.

 

Thank you.

 

George.

unraid-diagnostics-20210807-0025.zip

[trurl]

Just to clarify, it looks like you have 2 missing disks, and you have 2 disabled disks, but these are not both the same disks.

 

One of the missing disks is disabled, and another disk which isn't missing is also disabled. And another disk which is missing isn't disabled.

 

Since you have too many missing and/or disabled you can't start the array. Is that all correct?

 

Post a screenshot of Main

[georgez]

Just to clarify, it looks like you have 2 missing disks, and you have 2 disabled disks, but these are not both the same disks.
 
One of the missing disks is disabled, and another disk which isn't missing is also disabled. And another disk which is missing isn't disabled.
 
Since you have too many missing and/or disabled you can't start the array. Is that all correct?
 
Post a screenshot of Main

Nope, there’re 2 missing disks, and 1 disabled as shown in the attached screenshot. I didn’t try to start the array as I was scared to make it worse.




Sent from my iPhone using Tapatalk

[trurl]

Maybe my clarification could have been a little clearer.

 

Disk1 is disabled.

Disk2 is missing.

Disk11 is both missing and disabled.

 

If you had disk2, since you have dual parity you could rebuild both disk1 and disk11. Those are the disks Unraid wants to rebuild. But you can't without disk2.

 

What evidence do you have that the original disk11 was really bad and not just disabled due to bad connection? Bad connections are much more common that bad disks. Do you still have that original disk11?

[georgez]

You’re right. For the disk 11, it’s disabled for quite some days, I didn’t know if it’s really bad or just cable issue, even if it’s still useable its data must has been out of sync because there were data written to array after it’s disabled.

My only hope is that if I could re-enable disk 1, then the raid would be able to emulated disk 2 and 11. Is this even possible?


Sent from my iPhone using Tapatalk

[trurl]

 

[trurl]

In your case with dual parity, after New Config - Parity Valid, you would unassign both the disks to be rebuilt, check that they (emulated) mount, then reassign them to rebuild.

[georgez]

Thanks trurl, I’ll give it a try when my new disk (to replace the dropped one) arrives.


Sent from my iPhone using Tapatalk

[itimpi]

4 hours ago, trurl said:

In your case with dual parity, after New Config - Parity Valid, you would unassign both the disks to be rebuilt, check that they (emulated) mount, then reassign them to rebuild.

Will the New Config - Parity is valid approach work if one drive is missing?    If not then it may be a case of getting @JorgeB to give one of his ‘invalid slot’ sequence of steps?

[trurl]

I think invalidslot at the command line basically does the same thing as the newer steps outlined in that post I linked. I think the key is not mounting the disks during New Config by starting in Maintenance mode.

 

Both missing disks, 2, 11, will be rebuilt after disk1 is enabled by New Config.

[trurl]

56 minutes ago, trurl said:

I think

but I would welcome additional comments from @JorgeB Looks like he hasn't been on the forum for a few days.

[georgez]

On 8/7/2021 at 9:53 AM, trurl said:

In your case with dual parity, after New Config - Parity Valid, you would unassign both the disks to be rebuilt, check that they (emulated) mount, then reassign them to rebuild.

I’ve got my new disk, so I was trying to follow re-enable disk 1, however, after New Config I didn’t go on, instead I shutdown to replace the disk. When I restart, the array were empty, none of slots were assigned, can I assign all slots manually (I remember the previous assignments) and go ahead ? Or there’s any way to restore the old config and restart the re-enable procedure again?

 

Update: I noticed the diagnostics data contains a backup of the old config, if I put the backup files to USB boot stick, will it restore to the old array config?

Sent from my iPhone using Tapatalk

Edited August 8, 2021 by georgez

[trurl]

14 hours ago, georgez said:

after New Config I didn’t go on, instead I shutdown

New Config deleted your disk assignments, and since you didn't start the array with any new assignments, you don't have any assigned disks.

 

Will it let you do New Config now? I don't know since you don't currently have any disk assignments.

 

If you can still do New Config at this point, assign all disks exactly as they were except with the replacements, check parity valid box, then start in Maintenance mode as already explained, then it should still work. Disk1 would be enabled because of New Config, and then you could unassign then reassign the replacement disks to get them rebuilt.

 

Please ask if you have any questions. Better yet, explain in your own words exactly what steps you intend to take.

 

[georgez]

Please ask if you have any questions. Better yet, explain in your own words exactly what steps you intend to take.

I’m thinking of the possible way to restored the array to the state that just after the New Config. Sine it’s now an empty array, suppose I assign all disks (except the 2 missing) manually as what they were before New Config, how do I make the array know that there were 2 missing disks?


Sent from my iPhone using Tapatalk

[trurl]

You don't need to have any missing disks. Just go ahead and assign the replacements to those slots and the exact disks as before to all other slots.

 

Then, when you check parity valid box (so it won't rebuild parity) and check Maintenance Mode (so no disks are mounted), then start the array, disk assignments will be recorded, but nothing will be read or written to any disks (because no data disks are mounted and parity is valid). At that point, no disks will be disabled, which gets us disk1 back so it can be used with your dual parity plus all other disks to rebuild the 2 replacements.

 

Then, you stop the array, unassign the replacement disks, start the array in normal mode, and see if the emulated missing disks mount. It should show you on Main how much space they have if they mount. You can even take a look at the files on the emulated disks, even though the replacement disks are not assigned and even though those replacement disks don't actually contain any of the data. Their data is emulated by calculating it from parity plus all other disks. This is what parity is all about.

 

If that all looks good, stop the array, reassign the replacement disks, and start the array to begin rebuilding the emulated data to those disks.

 

Ask if you have any questions or if something doesn't seem to work as explained, and please don't deviate at all from what I have described.

[georgez]

You don't need to have any missing disks. Just go ahead and assign the replacements to those slots and the exact disks as before to all other slots.
 
Then, when you check parity valid box (so it won't rebuild parity) and check Maintenance Mode (so no disks are mounted), then start the array, disk assignments will be recorded, but nothing will be read or written to any disks (because no data disks are mounted and parity is valid). At that point, no disks will be disabled, which gets us disk1 back so it can be used with your dual parity plus all other disks to rebuild the 2 replacements.
 
Then, you stop the array, unassign the replacement disks, start the array in normal mode, and see if the emulated missing disks mount. It should show you on Main how much space they have if they mount. You can even take a look at the files on the emulated disks, even though the replacement disks are not assigned and even though those replacement disks don't actually contain any of the data. Their data is emulated by calculating it from parity plus all other disks. This is what parity is all about.
 
If that all looks good, stop the array, reassign the replacement disks, and start the array to begin rebuilding the emulated data to those disks.
 
Ask if you have any questions or if something doesn't seem to work as explained, and please don't deviate at all from what I have described.

I followed the steps, looks like somethings wrong here, the 2 missing disks were not emulated (screenshot). I still have old config files (disk.cfg and super.dat) so I think I can scroll back to old config but I don’t know why the missing disks were not emulated.


Sent from my iPhone using Tapatalk

[trurl]

Looks like that screenshot is after this:

32 minutes ago, trurl said:

stop the array, unassign the replacement disks, start the array in normal mode, and see if the emulated missing disks mount.

Are you sure you did all of this:

7 minutes ago, georgez said:

assign the replacements to those slots and the exact disks as before to all other slots.
 
Then, when you check parity valid box (so it won't rebuild parity) and check Maintenance Mode (so no disks are mounted), then start the array

before that?

[trurl]

Actually, I can't tell from that screenshot whether or not the emulated disks mount because you have clipped off all the part to the right of the screen that would show

37 minutes ago, trurl said:

on Main how much space they have if they mount

 

[georgez]

Yes I did assign all disks include the replacement ones, I tried it again, here’s the screenshot:


Sent from my iPhone using Tapatalk

[georgez]

Ah I misunderstood the thing, the emulation is working, I have to move mouse cursor to the cross to show it, can I go ahead and assign back disks to rebuild?


Sent from my iPhone using Tapatalk

[trurl]

2 minutes ago, georgez said:

the emulation is working

but the emulated disk doesn't mount, and the results of rebuild will be exactly what is emulated. It doesn't even seem to know what filesystem should be on the disks.

 

OK try this

23 minutes ago, georgez said:

I still have old config files (disk.cfg and super.dat) so I think I can scroll back to old config

but before rebooting, check disk.cfg to make sure it is set to not autostart.

 

Reboot, don't start the array, post new diagnostics and a screenshot of Main.

[georgez]

Here's the files.

unraid-diagnostics-20210809-0625.zip

[trurl]

OK let's try again from that starting point. 

 

New Config - Preserve All, Apply.

Assign the replacements for disks 2 and 11. 

Go to Array Operation, check both parity is valid and maintenance mode and start the array.

Stop the array, unassign disks 2 and 11, start the array in normal mode (not maintenance mode), and post a screenshot.

 

 

 

 

 

[georgez]

Same result, I attached 3 screen shots, 1 before Maintenance start, 1 before normal start, 1 after normal start. What if I format 2 replacement disks before maintenance ?

[trurl]

Maybe those 2 disks really were unmountable before all this started.

 

 

Stop the array, start it in Maintenance mode, click on disk2 to get to its settings, and post a screenshot of the Check Filesystem Status section.