November 14, 20232 yr Hello, I recently had a disk (Disk 8, sdp, dm-7) fall out of my array with the error "Unmountable: Wrong or No File System". At first, I just attempted normal data rebuild after removing and re-adding the disk to the array, with the hopes that this would fix my problem. Unfortunately, even after the rebuild was successful, I get the same error: UNRAID emhttpd: shcmd (9411): mkdir -p /mnt/disk8 UNRAID emhttpd: shcmd (9412): mount -t xfs -o noatime,nouuid /dev/mapper/md8 /mnt/disk8 UNRAID kernel: XFS (dm-7): Mounting V5 Filesystem UNRAID kernel: XFS (dm-7): Corruption warning: Metadata has LSN (18:4) ahead of current LSN (15:2351740). Please unmount and run xfs_repair (>= v4.3) to resolve. UNRAID kernel: XFS (dm-7): log mount/recovery failed: error -22 UNRAID kernel: XFS (dm-7): log mount failed UNRAID root: mount: /mnt/disk8: wrong fs type, bad option, bad superblock on /dev/mapper/md8, missing codepage or helper program, or other error. UNRAID root: dmesg(1) may have more information after failed mount system call. UNRAID emhttpd: shcmd (9412): exit status: 32 UNRAID emhttpd: /mnt/disk8 mount error: Wrong or no file system I can tell that this probably has something to do with the xfs recovery logs being corrupt as if I bypass them I am able to mount it (I've only done ro mounts as to not further mess with the data). In doing my mount, I can see that my data appears intact on the disk: root@UNRAID:~# cryptsetup luksOpen /dev/sdp1 decrypted_disk Enter passphrase for /dev/sdp1: root@UNRAID:~# mount -o ro,norecovery /dev/mapper/decrypted_disk /tmp/recovery/ root@UNRAID:~# ls /tmp/recovery/ Lxxxxs/ Nxxxxxd/ Wxxxxs/ WxxxxM/ bxxxxs/ mxxc/ pxxxxs/ pxxxx/ sxxxxxxr/ system/ wxxxxs/ yxxxxxl/ I'm guessing the first thing I should do is make a copy of this disk. But I'm also wondering if there's a faster avenue I can take while still keeping my data intact. How do I get out of this state? It should be noted that I think this was caused by an unclean shutdown at some point, I had a couple as I had a bad NIC that was freezing up my system. Please ignore me messing with my new NIC this morning, I have that working at least. On top of that, my power went out last night but luckily I had syslog mirroring enabled as I expected issues. If this mirrored syslog is needed, let me know. Otherwise, I've provided my diagnostics. To be clear, the GUI does ask me if I want to format it, which I obviously do not want to do here. For future readers, do not format the disk if it's in this state, for two reasons: the data could still be there (as it is in my case) and a format would make parity invalid and thus make your disk not-recoverable. I wish Unraid was able to detect corrupt xfs logs and disable formatting for that reason. Anyway, thanks in advance for your help! ETA: I was curious to see how much data was on this disk, which helps me both feel better (since the data appears to be mostly there, if not all of it) and worse (since it's so much data): UNRAID:~# du -sh /tmp/recovery 13T /tmp/recovery Edited April 12, 20242 yr by RocketSLC added du command
November 14, 20232 yr Community Expert Solution Check filesystem on disk8, run it without -n, if it asks for -L use it.
November 22, 20232 yr Author On 11/14/2023 at 6:43 PM, JorgeB said: Check filesystem on disk8, run it without -n, if it asks for -L use it. Once again, you have saved me! I had to run without -n and with -L. Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being destroyed because the -L option was used. - scan filesystem freespace and inode maps... clearing needsrepair flag and regenerating metadata invalid start block 1746451168 in record 417 of cnt btree block 3/172 agf_freeblks 28702206, counted 28702198 in ag 3 sb_icount 292096, counted 283264 sb_ifree 14949, counted 30078 sb_fdblocks 89729384, counted 142158794 - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - agno = 11 - agno = 12 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 2 - agno = 5 - agno = 8 - agno = 11 - agno = 6 - agno = 12 - agno = 10 - agno = 9 - agno = 4 - agno = 3 - agno = 7 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (15:2351571) is ahead of log (1:2). Format log to cycle 18. done After that, the next test passed and I was able to stop the array (taking it out of maintenance mode) and it came back normally. Thanks!
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.