Single Parity, Dual Disk failures, Rebuild Possible?

[JorgeB]

-Tools -> New Config -> Retain current configuration: All -> Apply
-Check all assignments and assign any missing disk(s) if needed, including cloned disk3 and the new disk4, replacement disk should be same size or larger than the old one
-IMPORTANT - Check both "parity is already valid" and "maintenance mode" and start the array (note that the GUI will still show that data on parity disk(s) will be overwritten, this is normal as it doesn't account for the checkbox, but it won't be as long as it's checked)
-Stop array
-Unassign disk4
-Start array (in normal mode now), ideally the emulated disk will now mount and contents look correct.

-Grab and post new diags.

[bdcmrt07]

Okay a couple of clarifications...

1) Do I need to keep the old disk3 connected still? Can I physically disconnect it and then proceed with the above? My plan was to disconnect the old disk3 now that we've rescued it and connect the replacement disk for disk4. Please confirm this is correct. 

 

2)  On the faq for ddrescue it had mentioned after completion you can mount the destination disk (in this case disk3) and copy contents to the array..I'm NOT doing that correct? Just want to make sure there is no assumption that i followed that direction in the post.

 

3) "ideally the emulated disk will now mount and contents look correct" --this is referring to the new disk3 that we ran ddrescue on? Or the contents from disk4 that died? Sorry just trying to understand for my own benefit because both disk3/disk4 were disabled with contents emulated. You had mentioned in a previous post it may be possible to retrieve data from disk4. Is my assumption that the "contents emulated" message from disk4 means that now that we were able to successfully retrieve the contents from disk3, disk3 together with Parity and the remaining disks holds the contents for disk4?

 

4) If my understanding of question3 is correct, how do the contents of old disk4 get onto replacement disk4? Or do i have it all wrong?

 

Thanks!

Edited October 6, 2022 by bdcmrt07

[JorgeB]

17 minutes ago, bdcmrt07 said:

Do I need to keep the old disk3 connected still?

Nope, just the clone.

 

17 minutes ago, bdcmrt07 said:

I'm NOT doing that correct?

Correct.

 

18 minutes ago, bdcmrt07 said:

this is referring to the new disk3 that we ran ddrescue on? Or the contents from disk4 that died?

In this case to both.

 

19 minutes ago, bdcmrt07 said:

Is my assumption that the "contents emulated" message from disk4 means that now that we were able to successfully retrieve the contents from disk3, disk3 together with Parity and the remaining disks holds the contents for disk4?

Yes.

 

20 minutes ago, bdcmrt07 said:

how do the contents of old disk4 get onto replacement disk4?

If the emulated disk 4 looks good you rebuild it onto the new disk.

 

 

 

 

[bdcmrt07]

Thanks for the confirmation! I followed your steps very carefully and I can see disk3 mounted now and the contents are there! So current state is the array is started and we just have the new disk4 unassigned and that's showing as "Not Installed....Unmountable: not mounted". 

 

Diagnostics attached. Awaiting further instructions.

 

 

Cheers!

tower-diagnostics-20221006-1803.zip

[JorgeB]

Check filesystem on the emulated disk4.

[bdcmrt07]

"The first step is to identify the file system of the drive you wish to test or repair. If you don't know for sure, then go to the Main page of the webGui, and click on the name of the drive (Disk 3, Cache, etc). Look for File system type, and you will see the file system format for your drive (should be xfs, reiserfs, or btrfs)."

 

disk4 is not mounted and listed as missing so it just says Auto. Please advise....Thx

[JorgeB]

Click on the disk and set the fs to xfs, then check filesystem.

[bdcmrt07]

It does not let me change it to xfs. When i select xfs i don't get the option to "Apply". All i can click is "Done" which doesn't save the change. When i go back to main screen it still shows as auto.

[JorgeB]

Strange but not wort losing time over that, do it on the command line, start the array in maintenance mode then type:

xfs_repair -v /dev/md4

 

[bdcmrt07]

Put into maintenance mode...executed that command and received the following..see screenshot

[JorgeB]

Type:

xfs_repair -vL /dev/md4

 

[bdcmrt07]

Complete...

[JorgeB]

Start the array in normal mode, disk should mount now, check contents and look for a lost+found folder.

[bdcmrt07]

Started in Array in normal mode...See attached..I've been browsing through all of the folders on disk4 and i haven't been able to locate a lost+found folder. 

[trurl]

lost+found would be a user share since it would be a top level folder on the repaired disk. Do you have a lost+found user share?

[bdcmrt07]

Nope, no lost+found there either. Doesn't it only get created if something is very corrupted? Is there a command i can use to easily serach and locate it if it does exist?

Edited October 7, 2022 by bdcmrt07

[JorgeB]

12 hours ago, bdcmrt07 said:

no lost+found there either.

That's a good thing, so most data should be good, of course there can still be some corruption due to disk3 errors, but it will be small since 99.99% was recovered.

[bdcmrt07]

All my content (or 99.999% of what i remember) appears to be there. So is the last step to stop the array, assign back disk4, and start back the array now so disk4 doesn't show as disabled/not installed? Current state, disk4 still unassigned. Thanks!

[JorgeB]

25 minutes ago, bdcmrt07 said:

So is the last step to stop the array, assign back disk4, and start back the array now

Yep, rebuild it using the new disk.

[itimpi]

20 hours ago, bdcmrt07 said:

Doesn't it only get created if something is very corrupted?

It gets created if the repair process finds files and/or folders for which it cannot locate the directory entry so does not know what the name of the file/folder should be.

[bdcmrt07]

9 hours ago, JorgeB said:

Yep, rebuild it using the new disk.

Apologies, when you say "rebuild" is that a specific process or am I literally just stopping array, assigning the disk and starting it back? 

[Kevin T]

6 minutes ago, bdcmrt07 said:

Apologies, when you say "rebuild" is that a specific process or am I literally just stopping array, assigning the disk and starting it back? 

It will rebuild automatically when you restart the array.

[bdcmrt07]

Rebuild Complete! All is well again! I would like to express my sincere thanks to @JorgeB and everyone else that contributed to helping me resolve this. Was super stressed when this incident happened and i thought i lost everything but this is an awesome feeling today. First time ever posting in these forums and it's great to know such a helpful and wonderful community exists! Truly can't thank you guys enough! Cheers all around my friends!

[Ronan C]

Awesome, i appreciate the support here, this guys are awesome!

 

Enjoy your second chance and put an extra disk for parity 2 my friend, and think about change the parity 1 for larger ones, i think this can help to protect data on the disks isnt?!

[Kilrah]

3 hours ago, Ronan C said:

think about change the parity 1 for larger ones, i think this can help to protect data on the disks isnt?!

No.