6.7.2: Lost disk 7 and emulated disk has problems too!

[spants]

So I have noticed a disk problem just before it completely failed.

 

I decided to do a Plan with unBalance (no writing to disks) to see if I could move the files from the emulated disk to real ones..... it started reading then the disk 7 disappeared completely...

What is the best way to proceed without losing the data on the emulated disk?

 

This is the log for the disk 7 error. I am now on 6.7.2

Nov 10 23:55:20 Tower kernel: XFS (md7): Mounting V4 Filesystem
Nov 10 23:55:20 Tower kernel: XFS (md7): Starting recovery (logdev: internal)
Nov 10 23:55:21 Tower kernel: XFS (md7): Internal error XFS_WANT_CORRUPTED_GOTO at line 1862 of file fs/xfs/libxfs/xfs_alloc.c.  Caller __xfs_free_extent+0xdf/0x146 [xfs]
Nov 10 23:55:21 Tower kernel: CPU: 3 PID: 14896 Comm: mount Not tainted 4.19.56-Unraid #1
Nov 10 23:55:21 Tower kernel: Hardware name: System manufacturer System Product Name/P8Z68-V PRO, BIOS 3603 11/09/2012
Nov 10 23:55:21 Tower kernel: Call Trace:
Nov 10 23:55:21 Tower kernel: dump_stack+0x5d/0x79
Nov 10 23:55:21 Tower kernel: xfs_free_ag_extent+0x3d2/0x5ff [xfs]
Nov 10 23:55:21 Tower kernel: __xfs_free_extent+0xdf/0x146 [xfs]
Nov 10 23:55:21 Tower kernel: xfs_trans_free_extent+0x27/0x5d [xfs]
Nov 10 23:55:21 Tower kernel: xfs_efi_recover+0x14b/0x199 [xfs]
Nov 10 23:55:21 Tower kernel: xlog_recover_process_efi+0x2d/0x43 [xfs]
Nov 10 23:55:21 Tower kernel: xlog_recover_process_intents+0xa6/0x1b0 [xfs]
Nov 10 23:55:21 Tower kernel: xlog_recover_finish+0x13/0x80 [xfs]
Nov 10 23:55:21 Tower kernel: xfs_log_mount_finish+0x5a/0xc3 [xfs]
Nov 10 23:55:21 Tower kernel: xfs_mountfs+0x50d/0x72f [xfs]
Nov 10 23:55:21 Tower kernel: ? xfs_mru_cache_create+0x12b/0x151 [xfs]
Nov 10 23:55:21 Tower kernel: xfs_fs_fill_super+0x448/0x527 [xfs]
Nov 10 23:55:21 Tower kernel: ? xfs_test_remount_options+0x53/0x53 [xfs]
Nov 10 23:55:21 Tower kernel: mount_bdev+0x12f/0x17c
Nov 10 23:55:21 Tower kernel: mount_fs+0x10/0x6b
Nov 10 23:55:21 Tower kernel: vfs_kern_mount+0x62/0xfa
Nov 10 23:55:21 Tower kernel: do_mount+0x7b3/0xa2f
Nov 10 23:55:21 Tower kernel: ? __kmalloc_track_caller+0x65/0x122
Nov 10 23:55:21 Tower kernel: ? _copy_from_user+0x2f/0x4d
Nov 10 23:55:21 Tower kernel: ? memdup_user+0x39/0x55
Nov 10 23:55:21 Tower kernel: ksys_mount+0x6d/0x92
Nov 10 23:55:21 Tower kernel: __x64_sys_mount+0x1c/0x1f
Nov 10 23:55:21 Tower kernel: do_syscall_64+0x57/0xf2
Nov 10 23:55:21 Tower kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Nov 10 23:55:21 Tower kernel: RIP: 0033:0x14ee3912bfca
Nov 10 23:55:21 Tower kernel: Code: 48 8b 0d c9 7e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 96 7e 0c 00 f7 d8 64 89 01 48
Nov 10 23:55:21 Tower kernel: RSP: 002b:00007ffe41fa7648 EFLAGS: 00000206 ORIG_RAX: 00000000000000a5
Nov 10 23:55:21 Tower kernel: RAX: ffffffffffffffda RBX: 000000000040d2b0 RCX: 000014ee3912bfca
Nov 10 23:55:21 Tower kernel: RDX: 000000000040d4c0 RSI: 000000000040d540 RDI: 000000000040d520
Nov 10 23:55:21 Tower kernel: RBP: 000014ee392baf24 R08: 0000000000000000 R09: 0000000000000000
Nov 10 23:55:21 Tower kernel: R10: 0000000000000c00 R11: 0000000000000206 R12: 0000000000000000
Nov 10 23:55:21 Tower kernel: R13: 0000000000000c00 R14: 000000000040d520 R15: 000000000040d4c0
Nov 10 23:55:21 Tower kernel: XFS (md7): Internal error xfs_trans_cancel at line 1041 of file fs/xfs/xfs_trans.c.  Caller xfs_efi_recover+0x15e/0x199 [xfs]
Nov 10 23:55:21 Tower kernel: CPU: 3 PID: 14896 Comm: mount Not tainted 4.19.56-Unraid #1
Nov 10 23:55:21 Tower kernel: Hardware name: System manufacturer System Product Name/P8Z68-V PRO, BIOS 3603 11/09/2012
Nov 10 23:55:21 Tower kernel: Call Trace:
Nov 10 23:55:21 Tower kernel: dump_stack+0x5d/0x79
Nov 10 23:55:21 Tower kernel: xfs_trans_cancel+0x52/0xcd [xfs]
Nov 10 23:55:21 Tower kernel: xfs_efi_recover+0x15e/0x199 [xfs]
Nov 10 23:55:21 Tower kernel: xlog_recover_process_efi+0x2d/0x43 [xfs]
Nov 10 23:55:21 Tower kernel: xlog_recover_process_intents+0xa6/0x1b0 [xfs]
Nov 10 23:55:21 Tower kernel: xlog_recover_finish+0x13/0x80 [xfs]
Nov 10 23:55:21 Tower kernel: xfs_log_mount_finish+0x5a/0xc3 [xfs]
Nov 10 23:55:21 Tower kernel: xfs_mountfs+0x50d/0x72f [xfs]
Nov 10 23:55:21 Tower kernel: ? xfs_mru_cache_create+0x12b/0x151 [xfs]
Nov 10 23:55:21 Tower kernel: xfs_fs_fill_super+0x448/0x527 [xfs]
Nov 10 23:55:21 Tower kernel: ? xfs_test_remount_options+0x53/0x53 [xfs]
Nov 10 23:55:21 Tower kernel: mount_bdev+0x12f/0x17c
Nov 10 23:55:21 Tower kernel: mount_fs+0x10/0x6b
Nov 10 23:55:21 Tower kernel: vfs_kern_mount+0x62/0xfa
Nov 10 23:55:21 Tower kernel: do_mount+0x7b3/0xa2f
Nov 10 23:55:21 Tower kernel: ? __kmalloc_track_caller+0x65/0x122
Nov 10 23:55:21 Tower kernel: ? _copy_from_user+0x2f/0x4d
Nov 10 23:55:21 Tower kernel: ? memdup_user+0x39/0x55
Nov 10 23:55:21 Tower kernel: ksys_mount+0x6d/0x92
Nov 10 23:55:21 Tower kernel: __x64_sys_mount+0x1c/0x1f
Nov 10 23:55:21 Tower kernel: do_syscall_64+0x57/0xf2
Nov 10 23:55:21 Tower kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Nov 10 23:55:21 Tower kernel: RIP: 0033:0x14ee3912bfca
Nov 10 23:55:21 Tower kernel: Code: 48 8b 0d c9 7e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 96 7e 0c 00 f7 d8 64 89 01 48
Nov 10 23:55:21 Tower kernel: RSP: 002b:00007ffe41fa7648 EFLAGS: 00000206 ORIG_RAX: 00000000000000a5
Nov 10 23:55:21 Tower kernel: RAX: ffffffffffffffda RBX: 000000000040d2b0 RCX: 000014ee3912bfca
Nov 10 23:55:21 Tower kernel: RDX: 000000000040d4c0 RSI: 000000000040d540 RDI: 000000000040d520
Nov 10 23:55:21 Tower kernel: RBP: 000014ee392baf24 R08: 0000000000000000 R09: 0000000000000000
Nov 10 23:55:21 Tower kernel: R10: 0000000000000c00 R11: 0000000000000206 R12: 0000000000000000
Nov 10 23:55:21 Tower kernel: R13: 0000000000000c00 R14: 000000000040d520 R15: 000000000040d4c0
Nov 10 23:55:21 Tower kernel: XFS (md7): xfs_do_force_shutdown(0x8) called from line 1042 of file fs/xfs/xfs_trans.c.  Return address = 000000007285336a
Nov 10 23:55:21 Tower kernel: XFS (md7): Corruption of in-memory data detected.  Shutting down filesystem
Nov 10 23:55:21 Tower kernel: XFS (md7): Please umount the filesystem and rectify the problem(s)
Nov 10 23:55:21 Tower kernel: XFS (md7): Failed to recover intents
Nov 10 23:55:21 Tower kernel: XFS (md7): log mount finish failed
Nov 10 23:55:21 Tower root: mount: /mnt/disk7: mount(2) system call failed: Structure needs cleaning.
Nov 10 23:55:21 Tower emhttpd: shcmd (71): exit status: 32
Nov 10 23:55:21 Tower emhttpd: /mnt/disk7 mount error: No file system
Nov 10 23:55:21 Tower emhttpd: shcmd (72): umount /mnt/disk7
Nov 10 23:55:21 Tower root: umount: /mnt/disk7: not mounted.

 

Edited November 11, 2019 by spants
mistake

[spants]

Managed to get the disk mounted by running the xfs_repair with -L parameter. Lots of data in lost/found....
Will move the data off the disks and restore the important files from the backup.