Paulitix Posted October 17, 2019 Share Posted October 17, 2019 (edited) After upgrading parity a disk (md8) was returning red ball. I tried to run xfs_repair and it says it couldnt find a valid secondary superblock. Any tips as to my next course of action? I've attached the most recent diagnostics, but I have a few from the last few days, including one right after parity was rebuilt. apollo-diagnostics-20191016-2013.zip EDIT: Looking back at an older syslog, it starts to get messy at the end. The log just ends, but I did nothing but clean shutdowns on the array. I also still have the old parity drive if theres some kind of rollback I can perform apollo-diagnostics-20191015-1159.zip Edited October 17, 2019 by Paulitix Quote Link to comment
Squid Posted October 17, 2019 Share Posted October 17, 2019 9 minutes ago, Paulitix said: I tried to run xfs_repair and it says it couldnt find a valid secondary superblock. Whats the full output of the command? Quote Link to comment
Paulitix Posted October 17, 2019 Author Share Posted October 17, 2019 (edited) Phase 1 - find and verify superblock... couldn't verify primary superblock - not enough secondary superblocks with matching geometry !!! attempting to find secondary superblock... .found candidate secondary superblock... unable to verify superblock, continuing... ....found candidate secondary superblock... unable to verify superblock, continuing... ...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Sorry, could not find valid secondary superblock Exiting now. Edited October 17, 2019 by Paulitix Quote Link to comment
JorgeB Posted October 17, 2019 Share Posted October 17, 2019 (edited) All 3 superblocks are corrupt, this means the rebuilt disk is likely generally corrupt, likely because of this: Oct 14 02:47:33 Apollo kernel: md: disk7 read error, sector=17815552 Oct 14 02:47:33 Apollo kernel: md: recovery thread: multiple disk errors, sector=17815552 Oct 14 02:47:33 Apollo kernel: md: disk7 read error, sector=17815560 Oct 14 02:47:33 Apollo kernel: md: recovery thread: multiple disk errors, sector=17815560 There were errors on disk7 when syncing parity, this makes parity corrupt, and unable to properly rebuild a disk. You were also having errors on disks 4 and 8 after that. See if old disk8 is still usable and try to copy everything you from it to the new one. Edited October 17, 2019 by johnnie.black Quote Link to comment
Paulitix Posted October 17, 2019 Author Share Posted October 17, 2019 Would it make any sense to put the original parity drive back and try to rebuild from there? Quote Link to comment
JorgeB Posted October 17, 2019 Share Posted October 17, 2019 Only if there were zero writes to all array disks since it was changed. Quote Link to comment
Paulitix Posted October 17, 2019 Author Share Posted October 17, 2019 (edited) Ugh, the remaining disks are showing minimal writes, but writes nonetheless. My array is currently sitting in maintenance mode. How should I go about this copy? What sould be my next steps to recover as much as possible (including kissing that drive's contents goodbye)? Right now I have a fresh disk that I was going to add after the parity upgrade, as well as the old parity drive that was replaced. Everything until then was in good wokring order, or so I thought. Is there a way I can try to save any files off the disk by connecting elsewhere (say another PC/Windows), putting the disk back, re-formatting, and then rebuilding parity? Or do disk 4 and 7 complicate that as well? Edited October 17, 2019 by Paulitix Quote Link to comment
JorgeB Posted October 17, 2019 Share Posted October 17, 2019 23 minutes ago, Paulitix said: Ugh, the remaining disks are showing minimal writes, but writes nonetheless. By zero writes I mean data writes from data you or some docker/vm copied, a very few writes will always happen due to filesystem housekeeping, and while not ideal they can usually recover from that, except btrfs since it keeps a generation id. I you believe old parity was valid you can try using it, but with so many different disks having errors on previous syslog you might have other issues, if old disk is still usable probably easier to try and copy the data from there. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.