Unmountable drive following parity upgrade


Recommended Posts

After upgrading parity a disk (md8) was returning red ball.

 

I tried to run xfs_repair and it says it couldnt find a valid secondary superblock. 

 

Any tips as to my next course of action?

 

I've attached the most recent diagnostics, but I have a few from the last few days, including one right after parity was rebuilt.

apollo-diagnostics-20191016-2013.zip

 

EDIT:

Looking back at an older syslog, it starts to get messy at the end.  The log just ends, but I did nothing but clean shutdowns on the array.

 

I also still have the old parity drive if theres some kind of rollback I can perform

 

apollo-diagnostics-20191015-1159.zip

Edited by Paulitix
Link to comment

Phase 1 - find and verify superblock...
couldn't verify primary superblock - not enough secondary superblocks with matching geometry !!!

attempting to find secondary superblock...
.found candidate secondary superblock...
unable to verify superblock, continuing...
....found candidate secondary superblock...
unable to verify superblock, continuing...
...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Sorry, could not find valid secondary superblock
Exiting now.

 

 

 

 

Edited by Paulitix
Link to comment

All 3 superblocks are corrupt, this means the rebuilt disk is likely generally corrupt, likely because of this:

 

Oct 14 02:47:33 Apollo kernel: md: disk7 read error, sector=17815552
Oct 14 02:47:33 Apollo kernel: md: recovery thread: multiple disk errors, sector=17815552
Oct 14 02:47:33 Apollo kernel: md: disk7 read error, sector=17815560
Oct 14 02:47:33 Apollo kernel: md: recovery thread: multiple disk errors, sector=17815560

 

There were errors on disk7 when syncing parity, this makes parity corrupt, and unable to properly rebuild a disk.

 

You were also having errors on disks 4 and 8 after that.

 

See if old disk8 is still usable and try to copy everything you from it to the new one.

 

 

Edited by johnnie.black
Link to comment

Ugh, the remaining disks are showing minimal writes, but writes nonetheless.

 

My array is currently sitting in maintenance mode.  How should I go about this copy?  What sould be my next steps to recover as much as possible (including kissing that drive's contents goodbye)?  Right now I have a fresh disk that I was going to add after the parity upgrade, as well as the old parity drive that was replaced.  Everything until then was in good wokring order, or so I thought.

 

Is there a way I can try to save any files off the disk by connecting elsewhere (say another PC/Windows), putting the disk back, re-formatting, and then rebuilding parity?  Or do disk 4 and 7 complicate that as well?

Edited by Paulitix
Link to comment
23 minutes ago, Paulitix said:

Ugh, the remaining disks are showing minimal writes, but writes nonetheless.

By zero writes I mean data writes from data you or some docker/vm copied, a very few writes will always happen due to filesystem housekeeping, and while not ideal they can usually recover from that, except btrfs since it keeps a generation id.

 

I you believe old parity was valid you can try using it, but with so many different disks having errors on previous syslog you might have other issues, if old disk is still usable probably easier to try and copy the data from there.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.