hermy65 Posted December 7, 2023 Share Posted December 7, 2023 Tuesday i started to hear a sound from my server that sounded like a fan going bad then about 5 minutes later i got notification that some drives were hot so i pulled the lid off to check the fan and it was good. Put things back together then went back to the dashboard and was missing 4 drives from one backplane. Shut down, reseated the 4 drives and powered back on, 2 came back and 2 were marked as missing. Thought maybe a backplane went bad so i powered down, swapped a new one in and then the 2 missing drives showed back up but were disabled so i kicked off a rebuild on the first one. Today the rebuild finished but now it says its unmountable, same with the other drive that was missing. What do i do now? Diagnostics are attached storage-diagnostics-20231207-0936.zip Quote Link to comment
JorgeB Posted December 7, 2023 Share Posted December 7, 2023 Check filesystem on disks 1 and 8, run it without -n Quote Link to comment
hermy65 Posted December 7, 2023 Author Share Posted December 7, 2023 @JorgeB im getting this on both Phase 1 - find and verify superblock... bad primary superblock - bad CRC in superblock !!! attempting to find secondary superblock... .found candidate secondary superblock... verified secondary superblock... writing modified primary superblock Phase 2 - using internal log - zero log... ERROR: The filesystem has valuable metadata changes in a log which needs to be replayed. Mount the filesystem to replay the log, and unmount it before re-running xfs_repair. If you are unable to mount the filesystem, then use the -L option to destroy the log and attempt a repair. Note that destroying the log may cause corruption -- please attempt a mount of the filesystem before doing this. Quote Link to comment
hermy65 Posted December 7, 2023 Author Share Posted December 7, 2023 @JorgeB Done - logs are below, what should i do next? disk 1 with -L did this: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being destroyed because the -L option was used. - scan filesystem freespace and inode maps... clearing needsrepair flag and regenerating metadata sb_icount 23808, counted 132032 sb_ifree 2334, counted 35228 sb_fdblocks 10484039, counted 33519164 - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 2 - agno = 9 - agno = 3 - agno = 1 - agno = 6 - agno = 7 - agno = 5 - agno = 8 - agno = 4 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (3:1193189) is ahead of log (1:2). Format log to cycle 6. done Disk 8 did: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being destroyed because the -L option was used. - scan filesystem freespace and inode maps... clearing needsrepair flag and regenerating metadata sb_icount 61696, counted 312768 sb_ifree 9362, counted 47709 sb_fdblocks 47064461, counted 92818335 - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - agno = 11 - agno = 12 - agno = 13 - agno = 14 - agno = 15 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 7 - agno = 12 - agno = 5 - agno = 3 - agno = 6 - agno = 8 - agno = 2 - agno = 9 - agno = 11 - agno = 10 - agno = 13 - agno = 14 - agno = 15 - agno = 4 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (7:742200) is ahead of log (1:2). Format log to cycle 10. done Quote Link to comment
itimpi Posted December 7, 2023 Share Posted December 7, 2023 Restart the array in normal mode and the drives should now mount OK. Quote Link to comment
hermy65 Posted December 7, 2023 Author Share Posted December 7, 2023 @JorgeB @itimpi The array started fine and Disk 1 is back and operational. Disk 8 though is still marked as disabled and emulated . I assume my next step is to force a rebuild on that disk? storage-diagnostics-20231207-1045.zip Quote Link to comment
JorgeB Posted December 7, 2023 Share Posted December 7, 2023 22 minutes ago, hermy65 said: I assume my next step is to force a rebuild on that disk? Disk looks healthy, so if the emulated disk contents look correct you can rebuild on top: https://docs.unraid.net/unraid-os/manual/storage-management#rebuilding-a-drive-onto-itself Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.