2x data drive failure, 1x parity drive failure - can't start array


rawfuls

Recommended Posts

Parity: 2x 10TB

Data: 4x 4TB, 1x 5TB

 

Server has been remote up until this week, I have physical access for this week only.

Disk 1 had been showing bad sector counts, I unassigned but never removed the disk. Parity 2 failed tonight, showed disk disabled. Well, as good as time as ever to remove those drives. Shut down, unplugged the failing data disk 1 and was planning on rebooting with parity 2 still connected to do scans. Disk 3 (4tb Data) never came back online. Tested different connectors, same issue. Starting to look like a complete dead drive.

 

Wanted to start the array with 1 parity 10tb, and unassigned the 2x 4TB drives but it is saying too many wrong/missing drives.

Tried assigning the disabled parity 2 drive as a data drive, just so I can get the array to rebuild with the minimal disks, then rebuild array once again with a new parity when I get one- same error.

However, I am not trying 'new config' as I feel like I'm going to klutz something, so reaching out for help before I do that.

 

Local Best Buy has 1x 12TB (no 10TBs) and I have it on pickup for tomorrow, but that would need to go in as parity since it's larger since it's a larger disk? If so, can I do so without starting the array as I'm unable to now?

Looking for any insight here, lots of data here, thankfully nothing mission critical but would really prefer not to have to rebuild it all.

Link to comment

I'm not 100% sold that the parity 2 is truly failing. I do know that disk 3 has failed due to it no longer being recognized. Disk 1 shows bad sectors. 

 

I'm hopeful that the second parity isn't actually failed, it still reads and SMART short tests fine, though I know that's mostly useless. I don't see any extreme readings in the disk info section which tells me it should be okay. I'm hoping it only disabled that disk due to a temp power loss (loose connector, etc).

Is there any hope of setting a new config to make Unraid unmark that disk as disabled, bring back that second parity just enough to rebuild the array OR transfer the parity bits to a new drive?

Edited by rawfuls
Link to comment

Sorry, I meant reboot, start array with it unassigned, then reassign it. Let parity rebuild and the error go away.

 

Parity 2 is not showing any yellow abnormalities for disk attributes. I rebooted before I could find out what the error was on it getting disabled.

 

Disk 1 is showing available (it does power on and read), but it's error is UDMA CRC error count - this was a known issue, I was waiting for me to arrive home (today) to pull the drive. I should have unassigned the drive first and rebuilt the array around it, but alas.

 

Disk 3 isn't powering back up, at all, this seems like a catastrophic failure.

Screen Shot 2020-12-24 at 2.57.19 AM.png

Link to comment
7 hours ago, JorgeB said:

Not quite sure what you're asking, you can't transfer parity, you can replace it with another disk, but can only have one more disk disable.

Can I force-enable parity 2, then move forward with rebuilding disk 3?

Can I force-enable Unraid to see disk 1 as NOT a new device?

Link to comment

Believe my tentative plan (waiting for 10TB Parity 2 to finish SMART extended) will be to run New Config > Preserve All.

Assign the parity1&2 respectively, data respectively except exclude disk1/3 (since disk1 is outdated, disk3 is failed).

(So does this mean previous disk2 is now disk1? Or do I leave previous disk2 as new disk2? As in, do I have empty data slots in the new config?)

 

With the 2 parities, I'm hoping the array will start without the 2 other data disks.

Do I add the 12TB as an additional disk?

 

If in the event that I have to move forward with rebuilding the entire array, is there any point in pulling the existing data off the existing data drives? It's not mission critical, but would be nice...

 

EDIT: The SMART extended test finished and no errors were found, as expected moving the 12TB to parity 2 and 10TB to disk3 did not prompt an easy 'copy' selection. 

It's looking like a New Config > Preserve All, then force re-enable Parity 2 and praying it allows me to rebuild by throwing in the 12TB into the data array.

Edited by rawfuls
Link to comment
17 hours ago, rawfuls said:

Can I force-enable Unraid to see disk 1 as NOT a new device?

Yes, but

 

On 12/24/2020 at 11:22 AM, JorgeB said:

if the array was used without it parity won't be valid if you re-enabled it now

 

 

You can re-enable parity2 and rebuild both disk1 and disk3, use a new spare disk for disk1 in case the rebuild disk is corrupt, or just check the emulate disk before rebuilding on top.

 

Again this will only work if parity is still valid:

 

-Tools -> New Config -> Retain current configuration: All -> Apply
-Check all assignments and assign any missing disk(s) if needed, including the new disks 1 and 3, replacement disk should be same size or larger than the old ones
-IMPORTANT - Check both "parity is already valid" and "maintenance mode" and start the array
-Stop array
-Unassign both disks you want to rebuild
-Start array (in normal mode now), ideally the emulated disks will now mount and contents look correct, if it doesn't you should run a filesystem check on the emulated disk(s).
-If the emulated disks mounts and contents look correct stop the array
-Re-assign the disks to rebuild and start array to begin, (don't rebuild on top of old disk1 if the emulated disk isn't OK).

 

 

  • Thanks 1
Link to comment

Was finally able to throw in an extra 5TB and 4TB drive in place of disk1/disk3.

Upon doing so, I got the warning that starting the array would wipe out the existing data on parity1/2.

No combination of disk juggling would resolve this despite me thinking it would.

Does in fact look like data loss, so I went ahead and started the array and pulling whatever data I can to pull over to the new array.

 

I had my appdata/configs saved to my SSD cache drive, which is unaffected. I noticed my docker tab was empty (aren't these stored on the cache drive?) but when reinstalling some docker containers, the app would require setup parameters (edit menu) but would then reuse the appdata info, so this should make rebuild a bit easier.

 

But, why does the docker tab clear out if the docker file is located in cache?

Link to comment

I did have those checked and it still prompted those.

Although, now that I look at my data, it does look like my Media has more folders now.

Starting to believe that the parity-sync was able to move all files back to Disks2/4... love the self doubt at this point.

Edited by rawfuls
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.