August 31, 20241 yr I find myself in a pretty horrible situation and need some help. I'm on 7 beta 2 with 6 SATA disks attached to an ASMEDIA AS1166 PCIe card. File system is xsf. A few days ago I found that the server had locked up and I had to hard reset it. Since then, I noticed one or more disks show errors like "failed command: READ FPDMA QUEUED" - "hard resetting link". This included the parity disk. I then proceed to attach that directly to the Mobo and Unraid didn't find that disk on next reboot and I had to reassign that same disk as parity disk, losing parity. I tried to rebuild parity but the system repeatedly had mini freezes when another disk hard resetted its controller link. I let that run over night but the whole system froze again, presumably because the log overran with controller messages. On another attempt, the system got unresponsive and I short-pressed the power button. The system shut down after a while but after the next boot, one of the data disks is now unmountable. I ran a filesystem check from the GUI and xfs_repair_status tells me: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ERROR: The filesystem has valuable metadata changes in a log which needs to be replayed. Mount the filesystem to replay the log, and unmount it before re-running xfs_repair. If you are unable to mount the filesystem, then use the -L option to destroy the log and attempt a repair. Note that destroying the log may cause corruption -- please attempt a mount of the filesystem before doing this. My first priority before tackling the other problems is to restore the data on this disk and back it up but I am unclear on how to proceed. The Checker button now says "ZERO LOG"... should I do that or something else? Thanks in advance!
August 31, 20241 yr Author Well, I started the array again normally, then stopped and started in maintenance mode again and ran another check on the unmountable disk. Now the result is: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ALERT: The filesystem has valuable metadata changes in a log which is being ignored because the -n option was used. Expect spurious inconsistencies which may be resolved by first mounting the filesystem to replay the log. - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan (but don't clear) agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - agno = 8 - agno = 9 - agno = 10 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 3 - agno = 5 - agno = 8 - agno = 2 - agno = 7 - agno = 4 - agno = 9 - agno = 10 - agno = 6 No modify flag set, skipping phase 5 Phase 6 - check inode connectivity... - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify link counts... Maximum metadata LSN (1:3393433) is ahead of log (1:3393409). Would format log to cycle 4. No modify flag set, skipping filesystem flush and exiting. The check button now offers FIX. Should I do that or something else?
August 31, 20241 yr Community Expert Run it again without -n, and if it asks for it, use -L, next time please post the diagnostics.
September 15, 20241 yr Author Just a heads up. My problems seem to be fixed after I have replaced ALL cabling with fabric-covered, tight-locking versions, including Molex and SATA-cables. It seems to have been a mix of bad cabling, loose connections and case vibrations which made the error hard to track to a single source. Funny how this only showed up after almost a year of seemingly reliable operation. I came out of it with minimal data loss and learned a lot, although I don't know how much creeping corruption may have ocurred during the last year.
September 15, 20241 yr 8 hours ago, Jorgo said: Funny how this only showed up after almost a year of seemingly reliable operation. SATA connections are crappy designs. The don't positively lock, and have poor contact patches that are prone to oxidation over time. More expensive cables, read ones that actually meet the written specs instead of engineered to a bottom price, can help somewhat, but every once in a while you just need to pull and reseat the cables to ensure good clean metal on metal contact for all the many pins.
September 15, 20241 yr Author Hmm.... now the drives are working well but I am stuck with (old) smart warnings about high UDMA CRC error counts. The numbers aren't rising... is there any way to get rid of those?
September 15, 20241 yr 29 minutes ago, Jorgo said: Hmm.... now the drives are working well but I am stuck with (old) smart warnings about high UDMA CRC error counts. The numbers aren't rising... is there any way to get rid of those? Nope. Those only increment upwards, the only way to reset them is with double top secret expensive stuff that theoretically only manufacturers can get. It's kind of like an odometer in a car, only I'm not aware of any laws specifically keeping them from being reset.
September 15, 20241 yr Author OK, I phrased that wrong... I just looked for a possiblity to acknowledge the smart warnings in the Unraid GUI (dashboard) but that is already possible through the context menu. Edited September 15, 20241 yr by Jorgo
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.