paululibro Posted October 22, 2021 Share Posted October 22, 2021 (edited) Hi. I just noticed that my array is missing about 8TB of space so I checked it and one of the drives shows "Unmountable: not mounted" message. It was fine a few hours ago, I didn't change anything on the server, no reboots, no power outage etc. I restarted the array but it didn't change anything. I restarted the server, but it didn't help either. So I followed this guide: Check Disk Filesystems up to the "Running the test" section. I started array in the Maintenance mode, checked fs of my drive (xfs) and ran the test with -nv flags. Test results: Spoiler Phase 1 - find and verify superblock... - block cache size set to 305152 entries Phase 2 - using internal log - zero log... zero_log: head block 2683033 tail block 2682495 ALERT: The filesystem has valuable metadata changes in a log which is being ignored because the -n option was used. Expect spurious inconsistencies which may be resolved by first mounting the filesystem to replay the log. - scan filesystem freespace and inode maps... agf_freeblks 1871568, counted 1871016 in ag 0 sb_ifree 1309, counted 1315 sb_fdblocks 13794311, counted 13351159 - found root inode chunk Phase 3 - for each AG... - scan (but don't clear) agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 imap claims a free inode 139 is in use, would correct imap and clear inode imap claims a free inode 28892283 is in use, would correct imap and clear inode imap claims a free inode 28892285 is in use, would correct imap and clear inode imap claims a free inode 1582104777 is in use, would correct imap and clear inode imap claims a free inode 1583459335 is in use, would correct imap and clear inode - agno = 1 imap claims in-use inode 2963299145 is free, correcting imap - agno = 2 imap claims a free inode 4332396100 is in use, would correct imap and clear inode imap claims a free inode 4332614211 is in use, would correct imap and clear inode imap claims a free inode 4332614213 is in use, would correct imap and clear inode imap claims a free inode 4332614224 is in use, would correct imap and clear inode - agno = 3 - agno = 4 imap claims in-use inode 8594108589 is free, correcting imap imap claims in-use inode 8635454101 is free, correcting imap data fork in ino 8635454102 claims free block 1079431756 imap claims in-use inode 8635454102 is free, correcting imap - agno = 5 - agno = 6 data fork in ino 12886092072 claims free block 131802630 data fork in ino 12886092072 claims free block 277947012 - agno = 7 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... free space (0,197722233-197724352) only seen by one free space btree - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 2 - agno = 3 entry "Foundation.2021.S01E05.Upon.Awakening.1080p.mkv" at block 0 offset 2144 in directory inode 6442637206 references free inode 2170362988 would clear inode number in entry at offset 2144... entry "9-1-1.2018.S05E05.Peer.Pressure.1080p.mkv" at block 0 offset 2224 in directory inode 6442637206 references free inode 6442637218 would clear inode number in entry at offset 2224... entry "Ice.Age.2002.720p.BluRay.DD5.1.x264.mkv" at block 0 offset 2448 in directory inode 6442637206 references free inode 4332614220 would clear inode number in entry at offset 2448... - agno = 4 - agno = 5 - agno = 6 - agno = 7 entry "Sample" in shortform directory 15059048199 references free inode 1583459335 would have junked entry "Sample" in directory inode 15059048199 entry "Sample" in shortform directory 15059048210 references free inode 139 would have junked entry "Sample" in directory inode 15059048210 entry "channels" in shortform directory 10755051349 references free inode 2170362991 would have junked entry "channels" in directory inode 10755051349 Missing reference count record for (1/116439153) len 2083453 count 2 setting reflink flag on inode 2963299145 setting reflink flag on inode 6442637214 setting reflink flag on inode 6442637217 setting reflink flag on inode 12886092071 No modify flag set, skipping phase 5 Phase 6 - check inode connectivity... - traversing filesystem ... - agno = 0 - agno = 1 - agno = 2 - agno = 3 entry "Foundation.2021.S01E05.Upon.Awakening.1080p.mkv" in directory inode 6442637206 points to free inode 2170362988, would junk entry entry "9-1-1.2018.S05E05.Peer.Pressure.1080p.mkv" in directory inode 6442637206 points to free inode 6442637218, would junk entry entry "Ice.Age.2002.720p.BluRay.DD5.1.x264.mkv" in directory inode 6442637206 points to free inode 4332614220, would junk entry bad hash table for directory inode 6442637206 (no data entry): would rebuild would rebuild directory inode 6442637206 - agno = 4 - agno = 5 entry "channels" in shortform directory inode 10755051349 points to free inode 2170362991 would junk entry - agno = 6 - agno = 7 entry "Sample" in shortform directory inode 15059048199 points to free inode 1583459335 would junk entry entry "Sample" in shortform directory inode 15059048210 points to free inode 139 would junk entry - traversal finished ... - moving disconnected inodes to lost+found ... disconnected inode 2963299145, would move to lost+found disconnected inode 4394877331, would move to lost+found disconnected inode 4394877332, would move to lost+found disconnected dir inode 8594108589, would move to lost+found Phase 7 - verify link counts... would have reset inode 6442637206 nlinks from 2 to 6 would have reset inode 10755051349 nlinks from 4 to 3 would have reset inode 15059048199 nlinks from 3 to 2 would have reset inode 15059048210 nlinks from 3 to 2 Maximum metadata LSN (1:2685632) is ahead of log (1:2683033). Would format log to cycle 4. No modify flag set, skipping filesystem flush and exiting. XFS_REPAIR Summary Fri Oct 22 22:32:29 2021 Phase Start End Duration Phase 1: 10/22 22:32:09 10/22 22:32:09 Phase 2: 10/22 22:32:09 10/22 22:32:10 1 second Phase 3: 10/22 22:32:10 10/22 22:32:26 16 seconds Phase 4: 10/22 22:32:26 10/22 22:32:26 Phase 5: Skipped Phase 6: 10/22 22:32:26 10/22 22:32:29 3 seconds Phase 7: 10/22 22:32:29 10/22 22:32:29 Total run time: 20 seconds What should I do next? Edited October 22, 2021 by paululibro Quote Link to comment
Spazilton Posted October 23, 2021 Share Posted October 23, 2021 (edited) Had this exact same issue but with much more serious corruption. Try restarting the array, see if the disc comes back. If not stop the array, maintenance mode and run xfs_repair again without the -n. Here was my issue. Edited October 23, 2021 by Spazilton Quote Link to comment
paululibro Posted October 23, 2021 Author Share Posted October 23, 2021 Thanks. I had to run it with additional -L flag so it was xfs_repair -v -L /dev/md3. Disk is back on and it also created new share with some files without extensions. I already identified two of them but can't figure one what the third one is. Quote Link to comment
Spazilton Posted October 23, 2021 Share Posted October 23, 2021 (edited) 48 minutes ago, paululibro said: Thanks. I had to run it with additional -L flag so it was xfs_repair -v -L /dev/md3. Disk is back on and it also created new share with some files without extensions. I already identified two of them but can't figure one what the third one is. You got off easier than I did. I had several thousand files to go through and figure out what goes where. At least 90% of them were in named subfolders so I could figure out what it was. Edited October 23, 2021 by Spazilton Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.