December 7, 20232 yr Tuesday i started to hear a sound from my server that sounded like a fan going bad then about 5 minutes later i got notification that some drives were hot so i pulled the lid off to check the fan and it was good. Put things back together then went back to the dashboard and was missing 4 drives from one backplane. Shut down, reseated the 4 drives and powered back on, 2 came back and 2 were marked as missing. Thought maybe a backplane went bad so i powered down, swapped a new one in and then the 2 missing drives showed back up but were disabled so i kicked off a rebuild on the first one. Today the rebuild finished but now it says its unmountable, same with the other drive that was missing. What do i do now? Diagnostics are attached storage-diagnostics-20231207-0936.zip
December 7, 20232 yr Author @JorgeB im getting this on both Phase 1 - find and verify superblock... bad primary superblock - bad CRC in superblock !!! attempting to find secondary superblock... .found candidate secondary superblock... verified secondary superblock... writing modified primary superblock Phase 2 - using internal log - zero log... ERROR: The filesystem has valuable metadata changes in a log which needs to be replayed. Mount the filesystem to replay the log, and unmount it before re-running xfs_repair. If you are unable to mount the filesystem, then use the -L option to destroy the log and attempt a repair. Note that destroying the log may cause corruption -- please attempt a mount of the filesystem before doing this.
December 7, 20232 yr Author @JorgeB Done - logs are below, what should i do next? disk 1 with -L did this: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being destroyed because the -L option was used. - scan filesystem freespace and inode maps... clearing needsrepair flag and regenerating metadata sb_icount 23808, counted 132032 sb_ifree 2334, counted 35228 sb_fdblocks 10484039, counted 33519164 - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 2 - agno = 9 - agno = 3 - agno = 1 - agno = 6 - agno = 7 - agno = 5 - agno = 8 - agno = 4 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (3:1193189) is ahead of log (1:2). Format log to cycle 6. done Disk 8 did: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being destroyed because the -L option was used. - scan filesystem freespace and inode maps... clearing needsrepair flag and regenerating metadata sb_icount 61696, counted 312768 sb_ifree 9362, counted 47709 sb_fdblocks 47064461, counted 92818335 - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - agno = 11 - agno = 12 - agno = 13 - agno = 14 - agno = 15 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 7 - agno = 12 - agno = 5 - agno = 3 - agno = 6 - agno = 8 - agno = 2 - agno = 9 - agno = 11 - agno = 10 - agno = 13 - agno = 14 - agno = 15 - agno = 4 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (7:742200) is ahead of log (1:2). Format log to cycle 10. done
December 7, 20232 yr Community Expert Restart the array in normal mode and the drives should now mount OK.
December 7, 20232 yr Author @JorgeB @itimpi The array started fine and Disk 1 is back and operational. Disk 8 though is still marked as disabled and emulated . I assume my next step is to force a rebuild on that disk? storage-diagnostics-20231207-1045.zip
December 7, 20232 yr Community Expert 22 minutes ago, hermy65 said: I assume my next step is to force a rebuild on that disk? Disk looks healthy, so if the emulated disk contents look correct you can rebuild on top: https://docs.unraid.net/unraid-os/manual/storage-management#rebuilding-a-drive-onto-itself
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.