I have about 500 miles of the first two errors in the logs before the server hard locked. What am I looking at here? I have additional complete log files for the last day or so if those are needed as well. Just let me know.
Additional diag attached.
Jan 25 13:32:54 Goliath kernel: critical medium error, dev sda, sector 5110141 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
Jan 25 13:32:54 Goliath kernel: FAT-fs (sda1): Directory bread(block 5108093) failed
Jan 25 13:32:54 Goliath kernel: sd 0:0:0:0: [sda] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s
Jan 25 13:32:54 Goliath kernel: sd 0:0:0:0: [sda] tag#0 Sense Key : 0x3 [current]
Jan 25 13:32:54 Goliath kernel: sd 0:0:0:0: [sda] tag#0 ASC=0x11 ASCQ=0x0
Jan 25 13:32:54 Goliath kernel: sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x28 28 00 00 4d f9 7e 00 00 01 00
Jan 25 13:32:54 Goliath kernel: critical medium error, dev sda, sector 5110142 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
Jan 25 13:32:54 Goliath kernel: FAT-fs (sda1): Directory bread(block 5108094) failed
Jan 25 13:32:54 Goliath kernel: sd 0:0:0:0: [sda] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s
Jan 25 13:32:54 Goliath kernel: sd 0:0:0:0: [sda] tag#0 Sense Key : 0x3 [current]
Jan 25 13:32:54 Goliath kernel: sd 0:0:0:0: [sda] tag#0 ASC=0x11 ASCQ=0x0
Jan 25 13:32:54 Goliath kernel: sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x28 28 00 00 4d f9 7f 00 00 01 00
Jan 25 13:32:54 Goliath kernel: critical medium error, dev sda, sector 5110143 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
Jan 25 13:32:54 Goliath kernel: FAT-fs (sda1): Directory bread(block 5108095) failed
Jan 25 13:33:08 Goliath kernel: Invalid SPTE change: cannot replace a present leaf
Jan 25 13:33:08 Goliath kernel: SPTE with another present leaf SPTE mapping a
Jan 25 13:33:08 Goliath kernel: different PFN!
Jan 25 13:33:08 Goliath kernel: as_id: 0 gfn: 2c5c33 old_spte: ffff88814b626a48 new_spte: 6000004a9033b77 level: 1
Jan 25 13:33:08 Goliath kernel: ------------[ cut here ]------------
Jan 25 13:33:08 Goliath kernel: kernel BUG at arch/x86/kvm/mmu/tdp_mmu.c:485!
Jan 25 13:33:08 Goliath kernel: invalid opcode: 0000 [#1] PREEMPT SMP PTI
Jan 25 13:33:09 Goliath kernel: CPU: 2 PID: 19792 Comm: CPU 4/KVM Tainted: P W O 6.1.64-Unraid #1
Jan 25 13:33:09 Goliath kernel: Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 10/21/2019
Jan 25 13:33:09 Goliath kernel: RIP: 0010:__handle_changed_spte+0x11b/0x4ce [kvm]
Jan 25 13:33:09 Goliath kernel: Code: a8 01 74 28 80 7c 24 3e 00 74 21 48 8b 0c 24 45 89 e1 49 89 d8 48 c7 c7 92 7d aa a0 48 8b 54 24 10 8b 74 24 24 e8 46 56 dd e0 <0f> 0b 48 39 1c 24 0f 84 94 03 00 00 66 90 eb 4d 65 8b 3d cb b2 58
Jan 25 13:33:09 Goliath kernel: RSP: 0018:ffffc9000bc17aa8 EFLAGS: 00010246
Jan 25 13:33:09 Goliath kernel: RAX: 00000000000000c2 RBX: 06000004a9033b77 RCX: 0000000000000000
Jan 25 13:33:09 Goliath kernel: RDX: 0000000000000000 RSI: ffffffff820d7e01 RDI: 00000000ffffffff
Jan 25 13:33:09 Goliath kernel: RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000031
Jan 25 13:33:09 Goliath kernel: R10: 0000000000000010 R11: ffff88907ff8eb0c R12: 0000000000000001
Jan 25 13:33:09 Goliath kernel: R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000800
Jan 25 13:33:09 Goliath kernel: FS: 000014648f7ff6c0(0000) GS:ffff88885f880000(0000) knlGS:ffffc500dd1e9000
Jan 25 13:33:09 Goliath kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 25 13:33:09 Goliath kernel: CR2: 000000008c6ce000 CR3: 00000008d8c00005 CR4: 00000000001726e0
Jan 25 13:33:09 Goliath kernel: DR0: 00000000042e1e67 DR1: 0000000000000000 DR2: 0000000000000000
Jan 25 13:33:09 Goliath kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jan 25 13:33:09 Goliath kernel: Call Trace:
Jan 25 13:33:09 Goliath kernel: <TASK>
Jan 25 13:33:09 Goliath kernel: ? __die_body+0x1a/0x5c
Jan 25 13:33:09 Goliath kernel: ? die+0x30/0x49
Jan 25 13:33:09 Goliath kernel: ? do_trap+0x7b/0xfe
Jan 25 13:33:09 Goliath kernel: ? __handle_changed_spte+0x11b/0x4ce [kvm]
Jan 25 13:33:09 Goliath kernel: ? __handle_changed_spte+0x11b/0x4ce [kvm]
Jan 25 13:33:09 Goliath kernel: ? do_error_trap+0x6e/0x98
Jan 25 13:33:09 Goliath kernel: ? __handle_changed_spte+0x11b/0x4ce [kvm]
Jan 25 13:33:09 Goliath kernel: ? exc_invalid_op+0x4c/0x60
Jan 25 13:33:09 Goliath kernel: ? __handle_changed_spte+0x11b/0x4ce [kvm]
Jan 25 13:33:09 Goliath kernel: ? asm_exc_invalid_op+0x16/0x20
Jan 25 13:33:09 Goliath kernel: ? __handle_changed_spte+0x11b/0x4ce [kvm]
Jan 25 13:33:09 Goliath kernel: ? __handle_changed_spte+0x11b/0x4ce [kvm]
Jan 25 13:33:09 Goliath kernel: tdp_mmu_set_spte_atomic+0x4b/0x64 [kvm]
Jan 25 13:33:09 Goliath kernel: kvm_tdp_mmu_map+0x27c/0x3c9 [kvm]
Jan 25 13:33:09 Goliath kernel: direct_page_fault+0x39f/0x581 [kvm]
Jan 25 13:33:09 Goliath kernel: ? try_to_wake_up+0x20e/0x248
Jan 25 13:33:09 Goliath kernel: kvm_mmu_do_page_fault+0xcf/0x12d [kvm]
Jan 25 13:33:09 Goliath kernel: kvm_mmu_page_fault+0x392/0x4a0 [kvm]
Jan 25 13:33:09 Goliath kernel: ? vmx_vmexit+0x79/0x8bd [kvm_intel]
Jan 25 13:33:09 Goliath kernel: ? vmx_vmexit+0x73/0x8bd [kvm_intel]
Jan 25 13:33:09 Goliath kernel: ? vmx_vmexit+0x8d/0x8bd [kvm_intel]
Jan 25 13:33:09 Goliath kernel: vmx_handle_exit+0x58d/0x667 [kvm_intel]
Jan 25 13:33:09 Goliath kernel: kvm_arch_vcpu_ioctl_run+0x133f/0x15db [kvm]
Jan 25 13:33:09 Goliath kernel: ? kvm_vm_ioctl_irq_line+0x23/0x37 [kvm]
Jan 25 13:33:09 Goliath kernel: kvm_vcpu_ioctl+0x192/0x5b4 [kvm]
Jan 25 13:33:09 Goliath kernel: ? __note_gp_changes+0x12f/0x13b
Jan 25 13:33:09 Goliath kernel: ? _raw_spin_unlock_irqrestore+0x24/0x3a
Jan 25 13:33:09 Goliath kernel: ? note_gp_changes+0x62/0x78
Jan 25 13:33:09 Goliath kernel: ? __seccomp_filter+0x89/0x2ff
Jan 25 13:33:09 Goliath kernel: ? rcu_core+0x27f/0x2ac
Jan 25 13:33:09 Goliath kernel: vfs_ioctl+0x1e/0x2f
Jan 25 13:33:09 Goliath kernel: __do_sys_ioctl+0x52/0x78
Jan 25 13:33:09 Goliath kernel: do_syscall_64+0x6b/0x81
Jan 25 13:33:09 Goliath kernel: entry_SYSCALL_64_after_hwframe+0x64/0xce
Jan 25 13:33:09 Goliath kernel: RIP: 0033:0x146894cad4e8
Jan 25 13:33:09 Goliath kernel: Code: 00 00 48 8d 44 24 08 48 89 54 24 e0 48 89 44 24 c0 48 8d 44 24 d0 48 89 44 24 c8 b8 10 00 00 00 c7 44 24 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 07 89 d0 c3 0f 1f 40 00 48 8b 15 f9 e8 0d
Jan 25 13:33:09 Goliath kernel: RSP: 002b:000014648f7fdc48 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Jan 25 13:33:09 Goliath kernel: RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 0000146894cad4e8
Jan 25 13:33:09 Goliath kernel: RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 000000000000001a
Jan 25 13:33:09 Goliath kernel: RBP: 0000146492738fc0 R08: 00005636dc7c9fe0 R09: 000000000000ffff
Jan 25 13:33:09 Goliath kernel: R10: 0000000000000007 R11: 0000000000000246 R12: 0000000000000000
Jan 25 13:33:09 Goliath kernel: R13: 0000000000000002 R14: 0000000000000001 R15: 0000000000000000
Jan 25 13:33:09 Goliath kernel: </TASK>
Jan 25 13:33:09 Goliath kernel: Modules linked in: veth xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap xt_nat xt_tcpudp macvlan xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype br_netfilter nvidia_uvm(PO) xfs nfsd auth_rpcgss oid_registry lockd grace sunrpc md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) tcp_diag inet_diag nvidia_modeset(PO) video nvidia(PO) ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp bridge stp llc bonding tls tg3 ipmi_ssif intel_rapl_msr intel_rapl_common iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm mgag200 drm_shmem_helper i2c_algo_bit crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel ghash_clmulni_intel sha512_ssse3 sha256_ssse3 sha1_ssse3 aesni_intel crypto_simd drm cryptd rapl intel_cstate hpsa
Jan 25 13:33:09 Goliath kernel: backlight i2c_i801 syscopyarea cp210x acpi_ipmi i2c_smbus sysfillrect sysimgblt intel_uncore i2c_core usbserial fb_sys_fops scsi_transport_sas wmi ipmi_si acpi_tad button unix [last unloaded: acpi_power_meter]
Jan 25 13:33:09 Goliath kernel: ---[ end trace 0000000000000000 ]---
Jan 25 13:33:09 Goliath kernel: RIP: 0010:__handle_changed_spte+0x11b/0x4ce [kvm]
Jan 25 13:33:09 Goliath kernel: Code: a8 01 74 28 80 7c 24 3e 00 74 21 48 8b 0c 24 45 89 e1 49 89 d8 48 c7 c7 92 7d aa a0 48 8b 54 24 10 8b 74 24 24 e8 46 56 dd e0 <0f> 0b 48 39 1c 24 0f 84 94 03 00 00 66 90 eb 4d 65 8b 3d cb b2 58
Jan 25 13:33:09 Goliath kernel: RSP: 0018:ffffc9000bc17aa8 EFLAGS: 00010246
Jan 25 13:33:09 Goliath kernel: RAX: 00000000000000c2 RBX: 06000004a9033b77 RCX: 0000000000000000
Jan 25 13:33:09 Goliath kernel: RDX: 0000000000000000 RSI: ffffffff820d7e01 RDI: 00000000ffffffff
Jan 25 13:33:09 Goliath kernel: RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000031
Jan 25 13:33:09 Goliath kernel: R10: 0000000000000010 R11: ffff88907ff8eb0c R12: 0000000000000001
Jan 25 13:33:09 Goliath kernel: R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000800
Jan 25 13:33:09 Goliath kernel: FS: 000014648f7ff6c0(0000) GS:ffff88885f880000(0000) knlGS:ffffc500dd1e9000
Jan 25 13:33:09 Goliath kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 25 13:33:09 Goliath kernel: CR2: 000000008c6ce000 CR3: 00000008d8c00005 CR4: 00000000001726e0
Jan 25 13:33:09 Goliath kernel: DR0: 00000000042e1e67 DR1: 0000000000000000 DR2: 0000000000000000
Jan 25 13:33:09 Goliath kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jan 25 13:33:09 Goliath kernel: note: CPU 4/KVM[19792] exited with preempt_count 1
Jan 25 13:33:48 Goliath nmbd[13543]: [2024/01/25 13:33:48.081639, 0] ../../source3/nmbd/nmbd_become_lmb.c:398(become_local_master_stage2)
Jan 25 13:33:48 Goliath nmbd[13543]: *****
Jan 25 13:33:48 Goliath nmbd[13543]:
Jan 25 13:33:48 Goliath nmbd[13543]: Samba name server GOLIATH is now a local master browser for workgroup WORKGROUP on subnet 192.168.1.207
Jan 25 13:33:48 Goliath nmbd[13543]:
Jan 25 13:33:48 Goliath nmbd[13543]: *****
goliath-diagnostics-20240125-1604.zip