Disk 1 unmountable


Recommended Posts

My shares became unmounted a few minutes ago, and after a reboot, I'm getting errors that disk 1 is unmountable. The console is showing

XFS (md1): Internal error i !: 1 at line 2111 of file fs/xfs/libxfs/xfs_)alloc.c. Caller xfs_free_ag_extent+0x3b7/0x602 [xfs]

there are a few more lines, and it ends XFS (md1): Failed to recover intents

 

In the meantime, all of my containers have vanished, I suspect my docker image was on that drive, and for whatever reason it isn't reading from parity? 

 

Edit: Actually, looks like all of the data on disk1 just isn't being read from parity. Any ideas on why that data would just be missing?

 

Edit 2: I tried xfs_repair -n /dev/md1, and got:

 

Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - zero log...
ALERT: The filesystem has valuable metadata changes in a log which is being
ignored because the -n option was used.  Expect spurious inconsistencies
which may be resolved by first mounting the filesystem to replay the log.
        - scan filesystem freespace and inode maps...
agi unlinked bucket 48 is 126543024 in ag 0 (inode=126543024)
invalid start block 1481005391 in record 351 of cnt btree block 2/28863905
agf_freeblks 13906192, counted 13906190 in ag 2
sb_icount 73728, counted 49024
sb_ifree 8475, counted 13377
sb_fdblocks 252972481, counted 355543473
        - found root inode chunk
Phase 3 - for each AG...
        - scan (but don't clear) agi unlinked lists...
        - process known inodes and perform inode discovery...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - process newly discovered inodes...
Phase 4 - check for duplicate blocks...
        - setting up duplicate extent list...
free space (2,155157649-155157650) only seen by one free space btree
        - check for inodes claiming duplicate blocks...
        - agno = 3
        - agno = 2
        - agno = 1
        - agno = 5
        - agno = 4
        - agno = 6
        - agno = 0
        - agno = 7
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
        - traversing filesystem ...
        - traversal finished ...
        - moving disconnected inodes to lost+found ...
disconnected inode 126543024, would move to lost+found
Phase 7 - verify link counts...
would have reset inode 126543024 nlinks from 0 to 1
No modify flag set, skipping filesystem flush and exiting.

 

Then I tried JorgeB's advice from 

and when I mounted the disk using  mount -vt xfs -o noatime,nodiratime /dev/md1 /x, I got mount: /x: mount(2) system call failed: Structure needs cleaning.

 

I suppose at this point I either need to do -L, or reformat the disk and rebuild from parity. Any suggestions on which are more likely to minimize data loss?

 

The SMART data seems to indicate the drive is fine. Should I reformat the disk and rebuild from parity?

tower-diagnostics-20210423-1250.zip

Edited by vagrantprodigy
Link to comment
42 minutes ago, vagrantprodigy said:

Should I reformat the disk and rebuild from parity?

NO!    If you issue a format that will erase the contents of the disk and update parity to reflect this.

 

You should run the xfs_repair command without -n and with -L to repair the file system.

  • Like 1
Link to comment

xfs_repair -L /dev/md1
Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - zero log...
ALERT: The filesystem has valuable metadata changes in a log which is being
destroyed because the -L option was used.
        - scan filesystem freespace and inode maps...
agi unlinked bucket 48 is 126543024 in ag 0 (inode=126543024)
invalid start block 1481005391 in record 351 of cnt btree block 2/28863905
agf_freeblks 13906192, counted 13906190 in ag 2
sb_icount 73728, counted 49024
sb_ifree 8475, counted 13377
sb_fdblocks 252972481, counted 355543473
        - found root inode chunk
Phase 3 - for each AG...
        - scan and clear agi unlinked lists...
        - process known inodes and perform inode discovery...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - process newly discovered inodes...
Phase 4 - check for duplicate blocks...
        - setting up duplicate extent list...
        - check for inodes claiming duplicate blocks...
        - agno = 1
        - agno = 2
        - agno = 0
        - agno = 3
        - agno = 6
        - agno = 7
        - agno = 5
        - agno = 4
Phase 5 - rebuild AG headers and trees...
        - reset superblock...
Phase 6 - check inode connectivity...
        - resetting contents of realtime bitmap and summary inodes
        - traversing filesystem ...
        - traversal finished ...
        - moving disconnected inodes to lost+found ...
disconnected inode 126543024, moving to lost+found
Phase 7 - verify and correct link counts...
Maximum metadata LSN (78:3556288) is ahead of log (1:2).
Format log to cycle 81.
done

Link to comment
1 hour ago, vagrantprodigy said:

The disk has remounted, and I was able to get my old docker image to work again. Is there any way to tell what, if anything, was lost?


Not for certain if you do not have a list of what was there before.     However, if no lost+found folder was created on the drive then it is highly likely that nothing was lost.

Link to comment
13 minutes ago, itimpi said:


Not for certain if you do not have a list of what was there before.     However, if no lost+found folder was created on the drive then it is highly likely that nothing was lost.

I do have a lost+found folder, with 1 item in it. I don't have an exact list of what was on the disk previously.

Link to comment
9 minutes ago, vagrantprodigy said:

I do have a lost+found folder, with 1 item in it. I don't have an exact list of what was on the disk previously.


no way to know for certain then, but typically the list+found folder are where folders/files end up if their directory entry cannot be found.   With only one file there it does not sound as if you had serious corruption.     You can use the Linux ‘file’ command on the one in lost+found to help identify its type and thus what program to check it with.

 

do you have backups you can use to check against with what is on unRaid?

Link to comment
50 minutes ago, itimpi said:


no way to know for certain then, but typically the list+found folder are where folders/files end up if their directory entry cannot be found.   With only one file there it does not sound as if you had serious corruption.     You can use the Linux ‘file’ command on the one in lost+found to help identify its type and thus what program to check it with.

 

do you have backups you can use to check against with what is on unRaid?

Only of Appdata, Docs, stuff like that. It's an 8TB drive, so I don't have a backup of all of it.

 

I ran du against the file, and it shows a size of 0, and the file command returns the value empty, so I'm assuming everything is there. Thank you for all of your help.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.