April 10, 20215 yr After a reboot & starting the array, I had an issue with a drive showing a few read errors. Thinking this wasn't right, I noticed that a parity drive also showed errors. Both of these drives are were plugged into the same LSI card. This gave me a clue. I replaced said card, after double-checking all my cables were good, and started the array in maintenance mode. I was able to rebuild the contents of the erroring drive, but after starting the array normally, I have two disks that are "unmountable: not mounted" on the main screen. quizzleunraid-diagnostics-20210410-1802.zip
April 11, 20215 yr Community Expert 3 hours ago, TQ said: rebuild the contents Was the disk disabled? You didn't mention it.
April 11, 20215 yr Community Expert Also please post new diags, syslog is empty in the ones posted, reboot if needed.
April 11, 20215 yr Author 11 hours ago, trurl said: Was the disk disabled? You didn't mention it. As with all read errors, they were indeed. Booted in maintenance mode and parity rebuild commenced on the disabled Disk 8.
April 11, 20215 yr Community Expert 13 hours ago, trurl said: Was the disk disabled? You didn't mention it. 2 hours ago, TQ said: As with all read errors, they were indeed. Unraid only disables a disk when a write to it fails. The failed write makes the disk out-of-sync with the array so it has to be rebuilt. Read errors won't disable a disk because a failed read doesn't make the disk out-of-sync with the array. But it is possible that a failed read will cause Unraid to get the data from the parity calculation and try to write it back to the disk. If that write fails the disk is disabled.
April 11, 20215 yr Author Rebooted after figuring out why syslog was blank. I see errors now below: # grep -C 3 -i error /var/log/syslog -- Apr 11 12:09:05 QuizzleUNRAID emhttpd: shcmd (11375): mount -t xfs -o noatime /dev/md7 /mnt/disk7 Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): Mounting V5 Filesystem Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): Starting recovery (logdev: internal) Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): Metadata CRC error detected at xfs_inobt_read_verify+0xd/0x3a [xfs], xfs_inobt block 0xaea86678 Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): Unmount and run xfs_repair Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): First 128 bytes of corrupted metadata buffer: Apr 11 12:09:05 QuizzleUNRAID kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ -- Apr 11 12:09:05 QuizzleUNRAID kernel: 00000050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Apr 11 12:09:05 QuizzleUNRAID kernel: 00000060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Apr 11 12:09:05 QuizzleUNRAID kernel: 00000070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): metadata I/O error in "xfs_btree_read_buf_block.constprop.0+0x75/0xc1 [xfs]" at daddr 0xaea86678 len 8 error 74 Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): xfs_do_force_shutdown(0x1) called from line 296 of file fs/xfs/xfs_trans_buf.c. Return address = 00000000d5f03c72 Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): I/O Error Detected. Shutting down filesystem Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): Please unmount the filesystem and rectify the problem(s) Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): xfs_difree_inobt: xfs_inobt_lookup() returned error -117. Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): Ending recovery (logdev: internal) Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md7): Error -5 reserving per-AG metadata reserve pool. Apr 11 12:09:05 QuizzleUNRAID root: mount: /mnt/disk7: can't read superblock on /dev/md7. Apr 11 12:09:05 QuizzleUNRAID emhttpd: shcmd (11375): exit status: 32 Apr 11 12:09:05 QuizzleUNRAID emhttpd: /mnt/disk7 mount error: not mounted Apr 11 12:09:05 QuizzleUNRAID emhttpd: shcmd (11376): umount /mnt/disk7 Apr 11 12:09:05 QuizzleUNRAID root: umount: /mnt/disk7: not mounted. Apr 11 12:09:05 QuizzleUNRAID emhttpd: shcmd (11376): exit status: 32 -- Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md8): Mounting V5 Filesystem Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md8): Log inconsistent (didn't find previous header) Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md8): failed to find log head Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md8): log mount/recovery failed: error -117 Apr 11 12:09:05 QuizzleUNRAID kernel: XFS (md8): log mount failed Apr 11 12:09:05 QuizzleUNRAID root: mount: /mnt/disk8: mount(2) system call failed: Structure needs cleaning. Apr 11 12:09:05 QuizzleUNRAID emhttpd: shcmd (11379): exit status: 32 Apr 11 12:09:05 QuizzleUNRAID emhttpd: /mnt/disk8 mount error: not mounted Apr 11 12:09:05 QuizzleUNRAID emhttpd: shcmd (11380): umount /mnt/disk8 Apr 11 12:09:05 QuizzleUNRAID root: umount: /mnt/disk8: not mounted. Apr 11 12:09:05 QuizzleUNRAID emhttpd: shcmd (11380): exit status: 32 Should I start back in maintenance mode and run file checks on both disks? syslog.txt
April 11, 20215 yr 14 minutes ago, TQ said: Should I start back in maintenance mode and run file checks on both disks? Yes. Don't forget to run them without the -n option. Both file systems are damaged, which is why they are not mountable. If it says you need to run the check again with the -L option, then do so.
April 12, 20215 yr Community Expert Use -L for disk7, disk8 on the other hand appears unfixable, likely there were read errors on a different disk during the rebuild.
April 13, 20215 yr Author On 4/12/2021 at 3:44 AM, JorgeB said: Use -L for disk7, disk8 on the other hand appears unfixable, likely there were read errors on a different disk during the rebuild. Thanks @JorgeB. Disk7 back in action. What should I do about disk8. I attempted to remove it from the array, and rebuild it's contents again, and that "succeeded" but the disk is still unmountable. Since I have dual parity and one of those is also disabled, should I attempt the rebuild of that one first and then again for disk8? I'm just at a loss (and I feel possibly data loss) as to what my next steps should be.
April 13, 20215 yr Community Expert 9 minutes ago, TQ said: What should I do about disk8. On 4/12/2021 at 9:44 AM, JorgeB said: likely there were read errors on a different disk during the rebuild. but unless you saved the syslog it's just a guess.
April 13, 20215 yr Author 2 minutes ago, JorgeB said: but unless you saved the syslog it's just a guess. Unfortunately, all syslogs prior to the second posting of this thread are blank.
April 13, 20215 yr Community Expert The filesystem appears to be damage beyond recovery, and like mentioned that suggests a corrupt rebuild, not much more I can suggest then restore the data from backups, if available.
April 14, 20215 yr Author Alright. Assuming I have backups of the data and can accept the loss of data from this disk, how do I get disk8 back to a working filesystem and available to take data again?
April 14, 20215 yr Community Expert No, that is a screenshot of Main - Array Devices. 5 minutes ago, trurl said: Post a screenshot of Main - Array Operation
April 14, 20215 yr Author 2 minutes ago, trurl said: No, that is a screenshot of Main - Array Devices. I promise I can read. ...and I guess the answer is right there in front of me.
April 14, 20215 yr Community Expert 9 minutes ago, TQ said: answer is right there in front of me yes it is
Archived
This topic is now archived and is closed to further replies.