FRX Posted September 25, 2022 Share Posted September 25, 2022 One of my drives (disk 5) was failing so I replaced it. When I started the server again I got a red X on disk 2. Even after doublechecking the cables it still said the same. Stupidly enough I unselected it and then reselected it on the disk selection dropdown, figuring it might help. Now I have two disks saying "Unmountable: No file system". I tried "xfs_repair -v /dev/md2" but got "Sorry, could not find valid secondary superblock" after a couple of hours. What can I do? Is my data beyond saving? file-server-diagnostics-20220925-0814.zip Quote Link to comment
JorgeB Posted September 25, 2022 Share Posted September 25, 2022 Disk2 looks healthy so it be easy to re-enable, is the old disk5 completely dead? If not connect it and post new diags. Quote Link to comment
FRX Posted September 25, 2022 Author Share Posted September 25, 2022 (edited) Thanks JorgeB! I switched back disk5, but cant start the array because it says "Too many wrong and/or missing disks!". Attached diagnostics anyway, if thats any help! file-server-diagnostics-20220925-1141.zip Edited September 25, 2022 by FRX Quote Link to comment
JorgeB Posted September 26, 2022 Share Posted September 26, 2022 Old disk5 appears to have really failed, but keep it intact for now, we can try re-enabling disk2 to rebuild disk5 again, connet new disk5 again, then: -Tools -> New Config -> Retain current configuration: All -> Apply -Check all assignments and assign any missing disk(s) if needed, including old disk2 and new disk5, replacement disk should be same size or larger than the old one -IMPORTANT - Check both "parity is already valid" and "maintenance mode" and start the array (note that the GUI will still show that data on parity disk(s) will be overwritten, this is normal as it doesn't account for the checkbox, but it won't be as long as it's checked) -Stop array -Unassign disk5 -Start array (in normal mode now), ideally the emulated disk will now mount and contents look correct, then post new diags Quote Link to comment
FRX Posted September 26, 2022 Author Share Posted September 26, 2022 Thanks for the instructions, and for helping out! I did as you said above, but disk2 still says "Unmountable: No file system". Posting diags! file-server-diagnostics-20220926-2012.zip Quote Link to comment
JorgeB Posted September 26, 2022 Share Posted September 26, 2022 Check filesystem on disks 2 and 5, note that you must remove -n or nothing will be done, and if it asks for -L use it. Quote Link to comment
FRX Posted September 27, 2022 Author Share Posted September 27, 2022 Same error on disk2 again, "Sorry, could not find valid secondary superblock". Quote Link to comment
JorgeB Posted September 27, 2022 Share Posted September 27, 2022 Please update to latest Unraid since it includes much newer xfs progrs and try running xfs_repair again on disk2. Quote Link to comment
FRX Posted September 27, 2022 Author Share Posted September 27, 2022 I updated Unraid now, should I start in maintenance mode when Im running xfs_repair? Quote Link to comment
trurl Posted September 27, 2022 Share Posted September 27, 2022 Just now, FRX said: start in maintenance mode when Im running xfs_repair No other way to do it. Your question does make me wonder though. Have you been doing check filesystem from the webUI or from the command line? Easy to get the command wrong, webUI will do it right. Quote Link to comment
FRX Posted September 27, 2022 Author Share Posted September 27, 2022 When I try to do it in the webUI I cant see "Check Filesystem Status" on drive2, its on the other drives though Quote Link to comment
JorgeB Posted September 27, 2022 Share Posted September 27, 2022 Click on disk2, set the fileystem to xfs and try again. Quote Link to comment
FRX Posted September 28, 2022 Author Share Posted September 28, 2022 Same thing again, "Sorry, could not find valid secondary superblock" Quote Link to comment
JorgeB Posted September 28, 2022 Share Posted September 28, 2022 And you are using the GUI to check filesystem? Also post new diags. Quote Link to comment
FRX Posted September 29, 2022 Author Share Posted September 29, 2022 Yeah Im using the GUI now file-server-diagnostics-20220929-1838.zip Quote Link to comment
JorgeB Posted September 29, 2022 Share Posted September 29, 2022 Diags after array start in normal mode please. Quote Link to comment
FRX Posted September 29, 2022 Author Share Posted September 29, 2022 Here it comes! And thanks again for helping out! file-server-diagnostics-20220929-2001.zip Quote Link to comment
JorgeB Posted September 29, 2022 Share Posted September 29, 2022 Start the array in maintenance mode and type in the console: xfs_repair -v /dev/md2 Then post the full output from that. Quote Link to comment
FRX Posted September 30, 2022 Author Share Posted September 30, 2022 This is what I got (but with thousands of periods) Phase 1 - find and verify superblock... couldn't verify primary superblock - not enough secondary superblocks with matching geometry !!! attempting to find secondary superblock... .......................................................................................................................................................................................................................................................................................................................................................................................................... Sorry, could not find valid secondary superblock Exiting now. Quote Link to comment
JorgeB Posted September 30, 2022 Share Posted September 30, 2022 This is very strange, clearly there's a xfs filesystem on disk2: ep 29 20:00:48 FILE-SERVER kernel: XFS (md2): Mounting V5 Filesystem Sep 29 20:00:48 FILE-SERVER root: mount: /mnt/disk2: mount(2) system call failed: Structure needs cleaning. Sep 29 20:00:48 FILE-SERVER root: dmesg(1) may have more information after failed mount system call. Sep 29 20:00:48 FILE-SERVER kernel: XFS (md2): Log inconsistent (didn't find previous header) Sep 29 20:00:48 FILE-SERVER kernel: XFS (md2): failed to find log head Sep 29 20:00:48 FILE-SERVER kernel: XFS (md2): log mount/recovery failed: error -117 Sep 29 20:00:48 FILE-SERVER kernel: XFS (md2): log mount failed But xfs_repair doesn't find main or a backup superblock, something weird is happening there, just one last shot in the dark, and to rule out any md driver issue, with the array stopped type in the console: xfs_repair -v /dev/sdc1 Quote Link to comment
FRX Posted October 1, 2022 Author Share Posted October 1, 2022 Same thing again Im afraid. Phase 1 - find and verify superblock... couldn't verify primary superblock - not enough secondary superblocks with matching geometry !!! attempting to find secondary superblock... .................. ..Sorry, could not find valid secondary superblock Exiting now. file-server-diagnostics-20221001-0812.zip Quote Link to comment
JorgeB Posted October 1, 2022 Share Posted October 1, 2022 It's strange but there's nothing else I can do for that filesystem, do you have backups? If not you can try posting the problem on the xfs mailing list, it's not Unraid related since running xfs_repair on the actual disk (bypassing the Unraid md driver) still cannot fix it. Quote Link to comment
FRX Posted October 1, 2022 Author Share Posted October 1, 2022 Thanks alot for trying! No backup unfortunately.. Im thinking I should disconnect disk2 and see if I can recover the data some other way. Whats the best solution to limit the damage on unraid? Remove disk2 (and replace it when I get a new disk), format the new disk5 and rebuild? Quote Link to comment
JorgeB Posted October 1, 2022 Share Posted October 1, 2022 Best to try and recover the data would be a xfs compatible file recovery util like UFS explorer. Quote Link to comment
FRX Posted October 1, 2022 Author Share Posted October 1, 2022 Yeah I heard about UFS explorer so might give that a go. If I somehow recover the data from disk2 and copy it to unraid, will everything be back? Or will some things be forever lost since two drives failed at once? Is it safe to remove disk2 and rebuild the array with one disk missing? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.