phillipppp Posted December 15, 2023 Share Posted December 15, 2023 Had a power outage recently and had issues with Docker starting and killing all shares. After some looking around, I ran the `xfs_repair` on Disk 1 in my array and it reported issues. I re-ran without `-n` and cannot make it past the following: ... Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... rebuilding directory inode 12955326179 Metadata corruption detected at 0x46fa05, inode 0x38983bd88 dinode fatal error -- couldn't map inode 15192014216, err = 117 Starting the array outside maintenance mode now shows the disk with: Unmountable: Unsupported or no file system I am almost to the point of formatting the drive and rebuilding in place but want to check if there are any other options to try first. Diagnostics attached. tower-diagnostics-20231214-2011.zip Quote Link to comment
JorgeB Posted December 15, 2023 Share Posted December 15, 2023 Try again, xfs_repair should never abort, it should always finish even if it would require some data loss, suggesting a xfs_repair bug. Quote Link to comment
phillipppp Posted December 15, 2023 Author Share Posted December 15, 2023 (edited) Thanks @JorgeB . Unfortunately, I have tried multiple times all with the same result. Would formatting the drive and doing a rebuild return the array to normal operation? Edited December 15, 2023 by phillipppp Quote Link to comment
JorgeB Posted December 15, 2023 Share Posted December 15, 2023 Formatting the drive will delete all data and update parity, if there are no backups you can try a future release with newer xfsprogs in case that issue is fixed, or try something like UFS explorer to try and recover the data now. Quote Link to comment
phillipppp Posted December 15, 2023 Author Share Posted December 15, 2023 Alright. I am on 6.12.6. Would there be any newer version? Should I replace the drive with a new one instead of formatting? Quote Link to comment
JorgeB Posted December 15, 2023 Share Posted December 15, 2023 4 minutes ago, phillipppp said: Would there be any newer version? Not at the moment. 5 minutes ago, phillipppp said: Should I replace the drive with a new one instead of formatting? You can try to see if the emulated disk is any different, most likely it won't, and you will need to disabled the disk to try that. Quote Link to comment
phillipppp Posted December 15, 2023 Author Share Posted December 15, 2023 So, the only next steps are to wait for a newer version of unraid or restore the entire array from a backup? I was under the impression that, with my configuration, the array would be protected from loss of a single drive. I even keep a hot spare in the chassis for this. Are there no other options? Quote Link to comment
trurl Posted December 15, 2023 Share Posted December 15, 2023 How are you running xfs repair? It is easy to get the command line wrong so be sure you run it from the webUI Quote Link to comment
phillipppp Posted December 15, 2023 Author Share Posted December 15, 2023 I first ran from the web ui with `-n` and once it reported issues, I re-ran without the `-n`. I can provide the full output if it would help but it is quite large. Quote Link to comment
JorgeB Posted December 15, 2023 Share Posted December 15, 2023 1 hour ago, phillipppp said: version of unraid or restore the entire array from a backup? Not the entire array, that disk. 1 hour ago, phillipppp said: I was under the impression that, with my configuration, the array would be protected from loss of a single drive. I even keep a hot spare in the chassis for this. Disk failure and filesystem corruption are different things, parity can protect from a disk failure. Quote Link to comment
phillipppp Posted December 15, 2023 Author Share Posted December 15, 2023 2 minutes ago, JorgeB said: Not the entire array, that disk. Is it normal to keep disk-level backups and not data/share-based backups? Quote Link to comment
JorgeB Posted December 15, 2023 Share Posted December 15, 2023 I do because I use snapshots, and they can can only be disk by disk, but even if you have share backups you could use for example rsync and it would only restore the missing data. Quote Link to comment
phillipppp Posted December 23, 2023 Author Share Posted December 23, 2023 I was able to get an image of the drive and mounted in an Ubuntu box booted on a USB. UFS was able to identify all the files on the drive in the same structure as my shares. What’s the best way to replace the disk without rebuilding the back without carrying the bad filesystem? Quote Link to comment
trurl Posted December 24, 2023 Share Posted December 24, 2023 To get another disk into the array in place of that disk, you either have to rebuild onto the new disk, or New Config with the new disk in place of the old one, and rebuild parity instead. Either way, after rebuild you can format the new disk in the array then copy data you recover from UFS to it. Quote Link to comment
phillipppp Posted December 24, 2023 Author Share Posted December 24, 2023 Okay thanks, was curious if rebuilding could be avoided. Fingers crossed it goes smooth. Quote Link to comment
JorgeB Posted December 24, 2023 Share Posted December 24, 2023 If I understood correctly and the recovered data is outside the array, you can format that array disk and restore the data, no need to rebuild. Quote Link to comment
phillipppp Posted December 28, 2023 Author Share Posted December 28, 2023 While still unresolved, I've got a thread going with the xfs distro. There may be more information there in case someone comes across a similar problem in the future and finds this topic. https://lkml.kernel.org/linux-xfs/CACf8WVuHtDoJS1x5XzvBV99Z6Gp7bqbwDT4QsKsJrETtyyvFTg@mail.gmail.com/ 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.