December 12, 201510 yr Was getting a couple odd I/O error messages in Windows when attempting to write to a couple of my user shares, so I thought a reboot might be in order... After reboot, I'm unable to access WebGUI at all, I can Ping and Telnet into the console remotely (no local physical monitor/kb on my server). Attached is the ZIP file from running "Diagnostics" on Telnet session but here is the end part of Syslog where stuff hit's the fan apparently! Dec 12 13:38:46 media kernel: XFS (md9): xfs_do_force_shutdown(0x8) called from line 1008 of file fs/xfs/xfs_trans.c. Return address = 0xffffffff812743f4 Dec 12 13:38:46 media kernel: XFS (md9): Corruption of in-memory data detected. Shutting down filesystem Dec 12 13:38:46 media kernel: XFS (md9): Please umount the filesystem and rectify the problem(s) Dec 12 13:38:46 media kernel: XFS (md9): Failed to recover EFIs Dec 12 13:38:46 media kernel: XFS (md9): log mount finish failed Dec 12 13:38:46 media kernel: XFS (md9): xfs_log_force: error -5 returned. Dec 12 13:38:46 media kernel: XFS (md9): xfs_log_force: error -5 returned. <<--- (and hundreds/thousands more of these) HELP! Please and thank you! -Sw2 media-diagnostics-20151212-1459.zip
December 12, 201510 yr Edit /boot/config/disk.cfg. Change startArray="yes" to be startArray="no" and reboot. That should get you access to the webUI. Then start the array in maintenance mode, and run the XFS checks on disk9. Also wouldn't hurt to run a memtest either, since its talking about in-memory corruption.
December 12, 201510 yr Author Edit /boot/config/disk.cfg. Change startArray="yes" to be startArray="no" and reboot. That should get you access to the webUI. Then start the array in maintenance mode, and run the XFS checks on disk9. Also wouldn't hurt to run a memtest either, since its talking about in-memory corruption. Thanks that allowed me to start the WebGUI... Checked "Maintenance Mode" box and then Start... It Immediately started a Parity Check... assume I need to let that finish? How do I start an XFS check on disk9? Also, might be a silly question, but I don't see a Memtest option in the GUI or know how to run from console. -Wade
December 12, 201510 yr Edit /boot/config/disk.cfg. Change startArray="yes" to be startArray="no" and reboot. That should get you access to the webUI. Then start the array in maintenance mode, and run the XFS checks on disk9. Also wouldn't hurt to run a memtest either, since its talking about in-memory corruption. Thanks that allowed me to start the WebGUI... Checked "Maintenance Mode" box and then Start... It Immediately started a Parity Check... assume I need to let that finish? How do I start an XFS check on disk9? Also, might be a silly question, but I don't see a Memtest option in the GUI or know how to run from console. -Wade I'm surprised that it started a parity check even in maintenance mode. I assume that you wound up doing an unclean shutdown. Shouldn't hurt to stop it (you can always run it again). Memtest you can only get to locally with a keyboard and mouse attached. Right when it starts booting, you'll see an option to run memtest
December 12, 201510 yr I would actually run the memtest first if only to rule it out. Run at least 1 pass (ideally though you should let it run for a day)
December 12, 201510 yr Author Edit /boot/config/disk.cfg. Change startArray="yes" to be startArray="no" and reboot. That should get you access to the webUI. Then start the array in maintenance mode, and run the XFS checks on disk9. Also wouldn't hurt to run a memtest either, since its talking about in-memory corruption. Thanks that allowed me to start the WebGUI... Checked "Maintenance Mode" box and then Start... It Immediately started a Parity Check... assume I need to let that finish? How do I start an XFS check on disk9? Also, might be a silly question, but I don't see a Memtest option in the GUI or know how to run from console. -Wade I'm surprised that it started a parity check even in maintenance mode. I assume that you wound up doing an unclean shutdown. Shouldn't hurt to stop it (you can always run it again). Memtest you can only get to locally with a keyboard and mouse attached. Right when it starts booting, you'll see an option to run memtest Ah yah, remember the Memtest on the boot, but it's been a couple years since I've booted with a monitor attached...
December 13, 201510 yr Author Running Memtest now... And 1 pass, no errors... I can let it run overnight, but I really need to figure out what's up with me Server first! (never had anything that led me to believe that i'd ever had any memory issues). -SW
December 13, 201510 yr Author Looking at the XFS repair on Disk 9... No clue what this all means, but it certainly "sounds" bad... Phase 1 - find and verify superblock... Phase 2 - using internal log - scan filesystem freespace and inode maps... out-of-order bno btree record 91 (72556437 131314) block 0/1 out-of-order bno btree record 102 (83548277 2354) block 0/1 out-of-order cnt btree record 20 (18038001 1) block 0/2 out-of-order cnt btree record 21 (50010339 1) block 0/2 out-of-order cnt btree record 22 (50010343 1) block 0/2 out-of-order cnt btree record 23 (70860475 1) block 0/2 out-of-order cnt btree record 24 (70860527 1) block 0/2 out-of-order cnt btree record 25 (73211771 1) block 0/2 out-of-order cnt btree record 26 (73226357 1) block 0/2 out-of-order cnt btree record 27 (73685354 1) block 0/2 out-of-order cnt btree record 28 (77686589 1) block 0/2 out-of-order cnt btree record 85 (1261386 23) block 0/2 block (0,1261400-1261400) multiply claimed by cnt space tree, state - 2 agf_freeblks 26931021, counted 27667652 in ag 0 ir_freecount/free mismatch, inode chunk 0/40785952, freecount 12 nfree 4 inode chunk claims used block, inobt block - agno 0, bno 18020273, inopb 8 agi_freecount 385, counted 393 in ag 0 agi_freecount 385, counted 377 in ag 0 finobt agi unlinked bucket 0 is 122490240 in ag 0 (inode=122490240) sb_ifree 7838, counted 7849 sb_fdblocks 65573345, counted 65581531 - found root inode chunk Phase 3 - for each AG... - scan (but don't clear) agi unlinked lists... found inodes not in the inode allocation tree - process known inodes and perform inode discovery... - agno = 0 imap claims in-use inode 40785991 is free, would correct imap imap claims in-use inode 40785992 is free, would correct imap imap claims in-use inode 40785993 is free, would correct imap imap claims in-use inode 40785994 is free, would correct imap data fork in ino 77200554 claims free block 9650102 data fork in ino 566881623 claims free block 72556437 data fork in ino 623478740 claims free block 83590138 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 1 - agno = 0 - agno = 2 - agno = 5 - agno = 7 - agno = 6 - agno = 3 - agno = 4 No modify flag set, skipping phase 5 Inode allocation btrees are too corrupted, skipping phases 6 and 7 No modify flag set, skipping filesystem flush and exiting.
December 13, 201510 yr Author Then I tried to do a repair (by blanking out the options in the "Check Filesystem Status" on the Disk 9 Page and get the following... Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... ERROR: The filesystem has valuable metadata changes in a log which needs to be replayed. Mount the filesystem to replay the log, and unmount it before re-running xfs_repair. If you are unable to mount the filesystem, then use the -L option to destroy the log and attempt a repair. Note that destroying the log may cause corruption -- please attempt a mount of the filesystem before doing this. When I try to mount the filesystem, the system just HANG'S and won't start... SO. I guess I have to try the -L and then repair??? Help!
December 25, 201510 yr Merry Christmas to all! I'm in the same position. Should I try -L? Help please?
Archived
This topic is now archived and is closed to further replies.