May 13, 20233 yr Recently I upgraded my my RAM, Motherboard, and CPU from DDR4->DDR5, i7-12700k->i7-13700k. The unraid setup ran fine for a couple of days, but this morning, I woke up and found that the server crashed. Every time I start it up, it crashes again within 30 minutes. I ran memtest and it passed. I upgraded BIOS, disabled docker, scrubbed btrfs cache pool but it's still crashing. XMP is disabled, and nothing is overclocked. Currently I'm testing disabling wireguard to see if that helps. Below is the error that I am getting. The errors seems to be a bit different each crash. Any help or suggestions would be appreciated. 7:44 AM Crash May 13 07:44:12 daniel-nas kernel: invalid opcode: 0000 [#2] PREEMPT SMP NOPTI May 13 07:44:12 daniel-nas kernel: CPU: 8 PID: 7800 Comm: find Tainted: G D 5.19.17-Unraid #2 May 13 07:44:12 daniel-nas kernel: Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023 May 13 07:44:12 daniel-nas kernel: RIP: 0010:get_cached_acl+0x1c/0x63 May 13 07:44:12 daniel-nas kernel: Code: 01 00 00 00 e9 79 43 19 00 c3 cc cc cc cc 0f 1f 44 00 00 55 53 e8 fb f8 ff ff 48 89 c5 e8 dd 44 e5 ff 48 8b 5d 00 48 85 db 74 <37> f6 c3 01 75 32 8b 13 85 d2 74 23 8d 4a 01 89 d0 f0 0f b1 0b 74 May 13 07:44:12 daniel-nas kernel: RSP: 0018:ffffc90008d27c18 EFLAGS: 00010297 May 13 07:44:12 daniel-nas kernel: RAX: 0000000000000000 RBX: ffff888196cc8c48 RCX: 0000000000038900 May 13 07:44:12 daniel-nas kernel: RDX: ffff888101e58000 RSI: 0000000000008000 RDI: ffff888196cc8c48 May 13 07:44:12 daniel-nas kernel: RBP: ffff888196cc8c58 R08: 0000000000000002 R09: 0000ffffffffffff May 13 07:44:12 daniel-nas kernel: R10: 281d2f771de95511 R11: 0000000000000fe0 R12: 00000000000041ff May 13 07:44:12 daniel-nas kernel: R13: ffff888196cc8c48 R14: 0000000000008000 R15: 0000000000000000 May 13 07:44:12 daniel-nas kernel: FS: 000014f559c41740(0000) GS:ffff88885f400000(0000) knlGS:0000000000000000 May 13 07:44:12 daniel-nas kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 13 07:44:12 daniel-nas kernel: CR2: 000000000045e048 CR3: 000000011cd2c004 CR4: 0000000000770ee0 May 13 07:44:12 daniel-nas kernel: PKRU: 55555554 May 13 07:44:12 daniel-nas kernel: Call Trace: May 13 07:44:12 daniel-nas kernel: <TASK> May 13 07:44:12 daniel-nas kernel: get_acl+0x1a/0xcc May 13 07:44:12 daniel-nas kernel: generic_permission+0xd7/0x1d2 May 13 07:44:12 daniel-nas kernel: inode_permission+0xbe/0x131 May 13 07:44:12 daniel-nas kernel: may_open+0xbf/0x105 May 13 07:44:12 daniel-nas kernel: path_openat+0x921/0xaa9 May 13 07:44:12 daniel-nas kernel: do_filp_open+0x55/0xb8 May 13 07:44:12 daniel-nas kernel: ? cp_new_stat+0x144/0x174 May 13 07:44:12 daniel-nas kernel: ? getname_flags+0x29/0x152 May 13 07:44:12 daniel-nas kernel: ? kmem_cache_alloc+0x11a/0x143 May 13 07:44:12 daniel-nas kernel: do_sys_openat2+0x6c/0xd9 May 13 07:44:12 daniel-nas kernel: do_sys_open+0x3a/0x5a May 13 07:44:12 daniel-nas kernel: do_syscall_64+0x68/0x81 May 13 07:44:12 daniel-nas kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd May 13 07:44:12 daniel-nas kernel: RIP: 0033:0x14f559d4923f May 13 07:44:12 daniel-nas kernel: Code: 89 4c 24 58 f6 c2 40 75 32 89 d0 45 31 d2 25 00 00 41 00 3d 00 00 41 00 74 21 80 3d a2 b3 0e 00 00 74 45 b8 01 01 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 85 00 00 00 48 83 c4 78 c3 48 8d 84 24 80 May 13 07:44:12 daniel-nas kernel: RSP: 002b:00007fff8e790260 EFLAGS: 00000202 ORIG_RAX: 0000000000000101 May 13 07:44:12 daniel-nas kernel: RAX: ffffffffffffffda RBX: 00007fff8e7903dc RCX: 000014f559d4923f May 13 07:44:12 daniel-nas kernel: RDX: 00000000000b0900 RSI: 0000000000458a00 RDI: 000000000000000a May 13 07:44:12 daniel-nas kernel: RBP: 0000000000458900 R08: 0000000000444d20 R09: 0000000000000073 May 13 07:44:12 daniel-nas kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000000 May 13 07:44:12 daniel-nas kernel: R13: 0000000000000000 R14: 0000000000444c90 R15: 0000000000000003 May 13 07:44:12 daniel-nas kernel: </TASK> May 13 07:44:12 daniel-nas kernel: Modules linked in: udp_diag xt_connmark xt_comment iptable_raw xt_mark ipvlan xt_nat veth xt_CHECKSUM ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter xfs md_mod tcp_diag inet_diag nct6775 nct6775_core hwmon_vid efivarfs iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp bridge stp llc mlx4_en mlx4_core r8169 realtek wmi_bmof x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd intel_cstate intel_uncore i915 iosf_mbi drm_buddy i2c_algo_bit ttm drm_display_helper i2c_i801 May 13 07:44:12 daniel-nas kernel: i2c_smbus drm_kms_helper btusb nvme btrtl ahci btbcm nvme_core btintel libahci drm bluetooth mpt3sas intel_gtt agpgart ecdh_generic raid_class input_leds ecc joydev led_class i2c_core scsi_transport_sas syscopyarea sysfillrect vmd sysimgblt fb_sys_fops thermal fan wmi video backlight tpm_crb tpm_tis tpm_tis_core tpm acpi_tad acpi_pad button unix [last unloaded: md_mod] May 13 07:44:12 daniel-nas kernel: ---[ end trace 0000000000000000 ]--- May 13 07:44:12 daniel-nas kernel: RIP: 0010:__btrfs_qgroup_reserve_meta+0x30/0x69 May 13 07:44:12 daniel-nas kernel: Code: 56 45 89 c6 41 55 44 0f b6 e9 41 54 44 89 e9 41 89 d4 55 89 f5 53 48 89 fb e8 14 ff ff ff 85 c0 0f 9e c1 83 f8 86 0f 95 c2 84 <d1> 75 29 45 84 f6 75 24 48 89 df e8 7f b1 ff ff 85 c0 78 18 44 89 May 13 07:44:12 daniel-nas kernel: RSP: 0018:ffffc900047c3c38 EFLAGS: 00010282 May 13 07:44:12 daniel-nas kernel: RAX: 0000000000000000 RBX: ffff8881aa3c8800 RCX: 0000000000000001 May 13 07:44:12 daniel-nas kernel: RDX: 0000000013730be0 RSI: ffff8881016b9800 RDI: ffff8881aa3c8800 May 13 07:44:12 daniel-nas kernel: RBP: 0000000000004000 R08: 0000000000000000 R09: 0000000000000000 May 13 07:44:12 daniel-nas kernel: R10: 0000000000001000 R11: 0000000000004000 R12: 0000000000000002 May 13 07:44:12 daniel-nas kernel: R13: 0000000000000001 R14: 0000000000000000 R15: ffff88819c4ae358 May 13 07:44:12 daniel-nas kernel: FS: 000014f559c41740(0000) GS:ffff88885f400000(0000) knlGS:0000000000000000 May 13 07:44:12 daniel-nas kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 13 07:44:12 daniel-nas kernel: CR2: 000000000045e048 CR3: 000000011cd2c004 CR4: 0000000000770ee0 May 13 07:44:12 daniel-nas kernel: PKRU: 55555554 May 13 07:46:16 daniel-nas kernel: BUG: kernel NULL pointer dereference, address: 0000000000000020 May 13 07:46:16 daniel-nas kernel: #PF: supervisor read access in kernel mode May 13 07:46:16 daniel-nas kernel: #PF: error_code(0x0000) - not-present page May 13 07:46:16 daniel-nas kernel: PGD 20a8df067 P4D 20a8df067 PUD 2cfe1c067 PMD 0 May 13 07:46:16 daniel-nas kernel: Oops: 0000 [#3] PREEMPT SMP NOPTI May 13 07:46:16 daniel-nas kernel: CPU: 8 PID: 10702 Comm: find Tainted: G D 5.19.17-Unraid #2 May 13 07:46:16 daniel-nas kernel: Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023 May 13 07:46:16 daniel-nas kernel: RIP: 0010:xfs_btree_get_block+0x5/0x37 [xfs] May 13 07:46:16 daniel-nas kernel: Code: 48 8b 40 38 c3 cc cc cc cc 48 8b 40 40 c3 cc cc cc cc 0f 1f 44 00 00 e8 b5 ff ff ff 48 8b 40 08 c3 cc cc cc cc 0f 1f 44 00 00 <f6> 47 20 02 74 11 0f b6 47 48 ff c8 39 f0 75 07 31 c0 48 89 02 eb May 13 07:46:16 daniel-nas kernel: RSP: 0018:ffffc9000b347c40 EFLAGS: 00010256 May 13 07:46:16 daniel-nas kernel: RAX: ffff88886a547c68 RBX: 0000000000000000 RCX: 0000000000000000 May 13 07:46:16 daniel-nas kernel: RDX: ffffc9000b347c88 RSI: 0000000000000000 RDI: 0000000000000000 May 13 07:46:16 daniel-nas kernel: RBP: ffffc9000b347cd0 R08: ffffc9000b347cf8 R09: ffffc9000b347cec May 13 07:46:16 daniel-nas kernel: R10: ffffc9000b347d00 R11: 0000000000000000 R12: ffff88815eb50000 May 13 07:46:16 daniel-nas kernel: R13: ffffc9000b347d00 R14: 0000000000000000 R15: ffffffffffffffff May 13 07:46:16 daniel-nas kernel: FS: 00001542d3294740(0000) GS:ffff88885f400000(0000) knlGS:0000000000000000 May 13 07:46:16 daniel-nas kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 13 07:46:16 daniel-nas kernel: CR2: 0000000000000020 CR3: 0000000361baa002 CR4: 0000000000770ee0 May 13 07:46:16 daniel-nas kernel: PKRU: 55555554 May 13 07:46:16 daniel-nas kernel: Call Trace: May 13 07:46:16 daniel-nas kernel: <TASK> May 13 07:46:16 daniel-nas kernel: xfs_btree_insrec+0xfb/0x448 [xfs] May 13 07:46:16 daniel-nas kernel: ? xfs_dabuf_map.constprop.0+0x56/0x299 [xfs] May 13 07:46:16 daniel-nas kernel: ? path_put+0x12/0x1b May 13 07:46:16 daniel-nas kernel: ? terminate_walk+0x20/0x6e May 13 07:46:16 daniel-nas kernel: ? xfs_da_read_buf+0x66/0xff [xfs] May 13 07:46:16 daniel-nas kernel: ? xfs_dir3_block_read+0x33/0xa6 [xfs] May 13 07:46:16 daniel-nas kernel: ? xfs_dir2_block_getdents+0x74/0x21c [xfs] May 13 07:46:16 daniel-nas kernel: ? xfs_readdir+0x137/0x190 [xfs] May 13 07:46:16 daniel-nas kernel: ? iterate_dir+0x94/0x146 May 13 07:46:16 daniel-nas kernel: ? __do_sys_getdents64+0x6b/0xd8 May 13 07:46:16 daniel-nas kernel: ? compat_filldir+0x1b2/0x1b2 May 13 07:46:16 daniel-nas kernel: ? do_syscall_64+0x68/0x81 May 13 07:46:16 daniel-nas kernel: ? entry_SYSCALL_64_after_hwframe+0x63/0xcd May 13 07:46:16 daniel-nas kernel: </TASK> May 13 07:46:16 daniel-nas kernel: Modules linked in: udp_diag xt_connmark xt_comment iptable_raw xt_mark ipvlan xt_nat veth xt_CHECKSUM ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter xfs md_mod tcp_diag inet_diag nct6775 nct6775_core hwmon_vid efivarfs iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp bridge stp llc mlx4_en mlx4_core r8169 realtek wmi_bmof x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd intel_cstate intel_uncore i915 iosf_mbi drm_buddy i2c_algo_bit ttm drm_display_helper i2c_i801 May 13 07:46:16 daniel-nas kernel: i2c_smbus drm_kms_helper btusb nvme btrtl ahci btbcm nvme_core btintel libahci drm bluetooth mpt3sas intel_gtt agpgart ecdh_generic raid_class input_leds ecc joydev led_class i2c_core scsi_transport_sas syscopyarea sysfillrect vmd sysimgblt fb_sys_fops thermal fan wmi video backlight tpm_crb tpm_tis tpm_tis_core tpm acpi_tad acpi_pad button unix [last unloaded: md_mod] May 13 07:46:16 daniel-nas kernel: CR2: 0000000000000020 May 13 07:46:16 daniel-nas kernel: ---[ end trace 0000000000000000 ]--- May 13 07:46:16 daniel-nas kernel: RIP: 0010:__btrfs_qgroup_reserve_meta+0x30/0x69 May 13 07:46:16 daniel-nas kernel: Code: 56 45 89 c6 41 55 44 0f b6 e9 41 54 44 89 e9 41 89 d4 55 89 f5 53 48 89 fb e8 14 ff ff ff 85 c0 0f 9e c1 83 f8 86 0f 95 c2 84 <d1> 75 29 45 84 f6 75 24 48 89 df e8 7f b1 ff ff 85 c0 78 18 44 89 May 13 07:46:16 daniel-nas kernel: RSP: 0018:ffffc900047c3c38 EFLAGS: 00010282 May 13 07:46:16 daniel-nas kernel: RAX: 0000000000000000 RBX: ffff8881aa3c8800 RCX: 0000000000000001 May 13 07:46:16 daniel-nas kernel: RDX: 0000000013730be0 RSI: ffff8881016b9800 RDI: ffff8881aa3c8800 May 13 07:46:16 daniel-nas kernel: RBP: 0000000000004000 R08: 0000000000000000 R09: 0000000000000000 May 13 07:46:16 daniel-nas kernel: R10: 0000000000001000 R11: 0000000000004000 R12: 0000000000000002 May 13 07:46:16 daniel-nas kernel: R13: 0000000000000001 R14: 0000000000000000 R15: ffff88819c4ae358 May 13 07:46:16 daniel-nas kernel: FS: 00001542d3294740(0000) GS:ffff88885f400000(0000) knlGS:0000000000000000 May 13 07:46:16 daniel-nas kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 13 07:46:16 daniel-nas kernel: CR2: 0000000000000020 CR3: 0000000361baa002 CR4: 0000000000770ee0 May 13 07:46:16 daniel-nas kernel: PKRU: 55555554 May 13 07:46:20 daniel-nas kernel: smartctl_type[10761]: segfault at ffffffffffffff8d ip 000014dc4fd5d931 sp 00007ffed3c52b70 error 7 in ld-2.36.so[14dc4fd4b000+27000] May 13 07:46:20 daniel-nas kernel: Code: ff 0f 84 72 03 00 00 89 75 90 45 8b 77 14 4d 8d 6f 30 48 85 db 0f 84 19 08 00 00 48 8d 35 f4 03 02 00 48 89 df e8 ef 02 01 00 <48> b9 00 00 00 00 00 00 04 00 48 89 8d 78 ff ff ff 85 c0 74 2d 48 12:19 PM Crash May 13 12:19:44 daniel-nas kernel: BUG: kernel NULL pointer dereference, address: 0000000000000004 May 13 12:19:44 daniel-nas kernel: #PF: supervisor read access in kernel mode May 13 12:19:44 daniel-nas kernel: #PF: error_code(0x0000) - not-present page May 13 12:19:44 daniel-nas kernel: PGD 1ab3eb067 P4D 1ab3eb067 PUD 1ab3ed067 PMD 0 May 13 12:19:44 daniel-nas kernel: Oops: 0000 [#1] PREEMPT SMP NOPTI May 13 12:19:44 daniel-nas kernel: CPU: 8 PID: 12423 Comm: dockerd Not tainted 5.19.17-Unraid #2 May 13 12:19:44 daniel-nas kernel: Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023 May 13 12:19:44 daniel-nas kernel: RIP: 0010:folio_migrate_flags+0xa/0x111 May 13 12:19:44 daniel-nas kernel: Code: f8 03 75 03 83 cb 02 49 8b 4c 24 08 89 ea 89 df e8 e7 ec fd ff 5b 5d 41 5c 41 5d c3 cc cc cc cc 0f 1f 44 00 00 55 48 89 fd 53 <48> 8b 06 48 89 f3 0f ba e0 08 73 05 f0 80 4f 01 01 48 8b 03 a8 02 May 13 12:19:44 daniel-nas kernel: RSP: 0018:ffffc90001d0fa48 EFLAGS: 00010246 May 13 12:19:44 daniel-nas kernel: RAX: ffffea000937f880 RBX: ffffffff811a67b9 RCX: 0000000000000000 May 13 12:19:44 daniel-nas kernel: RDX: ffffea000937f848 RSI: 0000000000000004 RDI: ffffea000937f880 May 13 12:19:44 daniel-nas kernel: RBP: ffffea000937f880 R08: 0000000000000000 R09: 0000000000000000 May 13 12:19:44 daniel-nas kernel: R10: 0000000000002498 R11: ffff8881ace770d0 R12: 0000000000000001 May 13 12:19:44 daniel-nas kernel: R13: 0000000000000000 R14: ffff8881c3605800 R15: ffffc90001d0fc07 May 13 12:19:44 daniel-nas kernel: FS: 000014c7579bc700(0000) GS:ffff88885f400000(0000) knlGS:0000000000000000 May 13 12:19:44 daniel-nas kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 13 12:19:44 daniel-nas kernel: CR2: 0000000000000004 CR3: 00000001a0296005 CR4: 0000000000770ee0 May 13 12:19:44 daniel-nas kernel: PKRU: 55555554 May 13 12:19:44 daniel-nas kernel: Call Trace: May 13 12:19:44 daniel-nas kernel: <TASK> May 13 12:19:44 daniel-nas kernel: ? mark_page_accessed+0xa/0x12 May 13 12:19:44 daniel-nas kernel: ? mark_extent_buffer_accessed+0x41/0x51 May 13 12:19:44 daniel-nas kernel: ? find_extent_buffer+0x3a/0x44 May 13 12:19:44 daniel-nas kernel: ? read_block_for_search+0xb3/0x27d May 13 12:19:44 daniel-nas kernel: ? btrfs_search_slot+0x6f7/0x7c5 May 13 12:19:44 daniel-nas kernel: ? btrfs_lookup_match_dir+0x35/0x6c May 13 12:19:44 daniel-nas kernel: ? btrfs_lookup_xattr+0x76/0xab May 13 12:19:44 daniel-nas kernel: ? btrfs_getxattr+0x6c/0x113 May 13 12:19:44 daniel-nas kernel: ? vfs_getxattr_alloc+0x96/0x117 May 13 12:19:44 daniel-nas kernel: ? cap_inode_getsecurity+0x92/0x25d May 13 12:19:44 daniel-nas kernel: ? vfs_getxattr+0xa5/0x138 May 13 12:19:44 daniel-nas kernel: ? do_getxattr+0x39/0x10f May 13 12:19:44 daniel-nas kernel: ? getxattr+0x72/0xb3 May 13 12:19:44 daniel-nas kernel: ? virt_to_slab+0x5/0x19 May 13 12:19:44 daniel-nas kernel: ? memcg_slab_free_hook+0x4b/0xf9 May 13 12:19:44 daniel-nas kernel: ? user_path_at_empty+0x42/0x4f May 13 12:19:44 daniel-nas kernel: ? kmem_cache_free+0xdb/0x176 May 13 12:19:44 daniel-nas kernel: ? user_path_at_empty+0x42/0x4f May 13 12:19:44 daniel-nas kernel: ? path_getxattr+0x6a/0xb1 May 13 12:19:44 daniel-nas kernel: ? do_syscall_64+0x68/0x81 May 13 12:19:44 daniel-nas kernel: ? entry_SYSCALL_64_after_hwframe+0x63/0xcd May 13 12:19:44 daniel-nas kernel: </TASK> May 13 12:19:44 daniel-nas kernel: Modules linked in: udp_diag xt_connmark xt_comment iptable_raw xt_mark ipvlan veth xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter xfs md_mod tcp_diag inet_diag nct6775 nct6775_core hwmon_vid efivarfs iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp bridge stp llc mlx4_en mlx4_core r8169 realtek wmi_bmof x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd intel_cstate intel_uncore i2c_i801 i2c_smbus i915 nvme nvme_core btusb btrtl ahci btbcm libahci btintel May 13 12:19:44 daniel-nas kernel: iosf_mbi bluetooth drm_buddy i2c_algo_bit ttm drm_display_helper drm_kms_helper ecdh_generic ecc input_leds drm joydev mpt3sas led_class intel_gtt agpgart i2c_core raid_class scsi_transport_sas syscopyarea sysfillrect sysimgblt vmd fb_sys_fops fan thermal wmi video backlight tpm_crb tpm_tis tpm_tis_core tpm acpi_tad acpi_pad button unix [last unloaded: mlx4_core] May 13 12:19:44 daniel-nas kernel: CR2: 0000000000000004 May 13 12:19:44 daniel-nas kernel: ---[ end trace 0000000000000000 ]--- May 13 12:19:44 daniel-nas kernel: RIP: 0010:folio_migrate_flags+0xa/0x111 May 13 12:19:44 daniel-nas kernel: Code: f8 03 75 03 83 cb 02 49 8b 4c 24 08 89 ea 89 df e8 e7 ec fd ff 5b 5d 41 5c 41 5d c3 cc cc cc cc 0f 1f 44 00 00 55 48 89 fd 53 <48> 8b 06 48 89 f3 0f ba e0 08 73 05 f0 80 4f 01 01 48 8b 03 a8 02 May 13 12:19:44 daniel-nas kernel: RSP: 0018:ffffc90001d0fa48 EFLAGS: 00010246 May 13 12:19:44 daniel-nas kernel: RAX: ffffea000937f880 RBX: ffffffff811a67b9 RCX: 0000000000000000 May 13 12:19:44 daniel-nas kernel: RDX: ffffea000937f848 RSI: 0000000000000004 RDI: ffffea000937f880 May 13 12:19:44 daniel-nas kernel: RBP: ffffea000937f880 R08: 0000000000000000 R09: 0000000000000000 May 13 12:19:44 daniel-nas kernel: R10: 0000000000002498 R11: ffff8881ace770d0 R12: 0000000000000001 May 13 12:19:44 daniel-nas kernel: R13: 0000000000000000 R14: ffff8881c3605800 R15: ffffc90001d0fc07 May 13 12:19:44 daniel-nas kernel: FS: 000014c7579bc700(0000) GS:ffff88885f400000(0000) knlGS:0000000000000000 May 13 12:19:44 daniel-nas kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 13 12:19:44 daniel-nas kernel: CR2: 0000000000000004 CR3: 00000001a0296006 CR4: 0000000000770ee0 May 13 12:19:44 daniel-nas kernel: PKRU: 55555554 4:55 PM Crash May 13 16:55:39 daniel-nas kernel: traps: smartctl[31759] general protection fault ip:14f579809496 sp:7ffdb96eba60 error:0 in libc-2.36.so[14f5797ab000+16b000] May 13 16:56:09 daniel-nas kernel: traps: smartctl_type[313] general protection fault ip:150cb512ffa8 sp:7ffe75298bd0 error:0 in libc-2.36.so[150cb4fee000+16b000] May 13 16:57:29 daniel-nas kernel: BUG: kernel NULL pointer dereference, address: 0000000000000000 May 13 16:57:29 daniel-nas kernel: #PF: supervisor read access in kernel mode May 13 16:57:29 daniel-nas kernel: #PF: error_code(0x0000) - not-present page May 13 16:57:29 daniel-nas kernel: PGD 0 P4D 0 May 13 16:57:29 daniel-nas kernel: Oops: 0000 [#1] PREEMPT SMP NOPTI May 13 16:57:29 daniel-nas kernel: CPU: 9 PID: 18760 Comm: shfs Not tainted 5.19.17-Unraid #2 May 13 16:57:29 daniel-nas kernel: Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1002 03/31/2023 May 13 16:57:29 daniel-nas kernel: RIP: 0010:fuse_get_req+0x1b/0x1f7 May 13 16:57:29 daniel-nas kernel: Code: fb 5a 57 00 48 83 c4 18 5b 5d c3 cc cc cc cc 0f 1f 44 00 00 41 56 41 89 f6 41 55 49 89 fd 41 54 41 89 f4 55 53 48 83 ec 30 48 <8b> 2f 65 48 8b 04 25 28 00 00 00 48 89 44 24 28 31 c0 f0 ff 85 34 May 13 16:57:29 daniel-nas kernel: RSP: 0018:ffffc9000150bbf8 EFLAGS: 00010246 May 13 16:57:29 daniel-nas kernel: RAX: 0000000000000000 RBX: ffffc9000150bd00 RCX: 0000000000000001 May 13 16:57:29 daniel-nas kernel: RDX: 0000000000000000 RSI: ffffc9000150be30 RDI: 0000000000000000 May 13 16:57:29 daniel-nas kernel: RBP: 0000000000000000 R08: 0000000000000068 R09: 0000000000000000 May 13 16:57:29 daniel-nas kernel: R10: ffffc9000150be30 R11: ffffc9000150bc00 R12: ffffc9000150be08 May 13 16:57:29 daniel-nas kernel: R13: ffff888161fc9400 R14: ffff88815ecfbe00 R15: ffff88818af32100 May 13 16:57:29 daniel-nas kernel: FS: 0000149fac7c56c0(0000) GS:ffff88885f440000(0000) knlGS:0000000000000000 May 13 16:57:29 daniel-nas kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 13 16:57:29 daniel-nas kernel: CR2: 0000000000000000 CR3: 00000001a19c8005 CR4: 0000000000770ee0 May 13 16:57:29 daniel-nas kernel: PKRU: 55555554 May 13 16:57:29 daniel-nas kernel: Call Trace: May 13 16:57:29 daniel-nas kernel: <TASK> May 13 16:57:29 daniel-nas kernel: ? fuse_copy_one+0x3f/0x51 May 13 16:57:29 daniel-nas kernel: ? fuse_dev_do_write+0x57/0xaa1 May 13 16:57:29 daniel-nas kernel: mntget+0x1c/0x25 May 13 16:57:29 daniel-nas kernel: dget+0x12/0x1b May 13 16:57:29 daniel-nas kernel: ? path_put+0x12/0x1b May 13 16:57:29 daniel-nas kernel: ? terminate_walk+0x20/0x6e May 13 16:57:29 daniel-nas kernel: ? path_lookupat+0xec/0xfe May 13 16:57:29 daniel-nas kernel: ? fuse_dev_write+0x60/0x7e May 13 16:57:29 daniel-nas kernel: ? do_iter_readv_writev+0x93/0xdd May 13 16:57:29 daniel-nas kernel: ? do_iter_write+0x80/0xc4 May 13 16:57:29 daniel-nas kernel: ? vfs_writev+0x79/0xbd May 13 16:57:29 daniel-nas kernel: ? futex_wake+0x7d/0x149 May 13 16:57:29 daniel-nas kernel: ? __fget+0x33/0x41 May 13 16:57:29 daniel-nas kernel: ? __fget_light+0x3d/0x4c May 13 16:57:29 daniel-nas kernel: ? do_writev+0x7f/0xf0 May 13 16:57:29 daniel-nas kernel: ? do_syscall_64+0x68/0x81 May 13 16:57:29 daniel-nas kernel: ? entry_SYSCALL_64_after_hwframe+0x63/0xcd May 13 16:57:29 daniel-nas kernel: </TASK> May 13 16:57:29 daniel-nas kernel: Modules linked in: xt_connmark xt_comment iptable_raw xt_mark udp_diag veth xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap ipvlan xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter xfs md_mod tcp_diag inet_diag nct6775 nct6775_core hwmon_vid efivarfs iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet 8021q garp mrp bridge stp llc mlx4_en mlx4_core r8169 realtek wmi_bmof x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd intel_cstate intel_uncore nvme ahci i2c_i801 i2c_smbus nvme_core libahci btusb i915 btrtl btbcm btintel May 13 16:57:29 daniel-nas kernel: bluetooth iosf_mbi drm_buddy i2c_algo_bit ttm drm_display_helper ecdh_generic ecc drm_kms_helper input_leds drm joydev mpt3sas led_class intel_gtt agpgart vmd raid_class i2c_core scsi_transport_sas syscopyarea sysfillrect sysimgblt fb_sys_fops fan thermal wmi video backlight tpm_crb tpm_tis tpm_tis_core tpm acpi_tad acpi_pad button unix [last unloaded: mlx4_core] May 13 16:57:29 daniel-nas kernel: CR2: 0000000000000000 May 13 16:57:29 daniel-nas kernel: ---[ end trace 0000000000000000 ]--- May 13 16:57:29 daniel-nas kernel: RIP: 0010:fuse_get_req+0x1b/0x1f7 May 13 16:57:29 daniel-nas kernel: Code: fb 5a 57 00 48 83 c4 18 5b 5d c3 cc cc cc cc 0f 1f 44 00 00 41 56 41 89 f6 41 55 49 89 fd 41 54 41 89 f4 55 53 48 83 ec 30 48 <8b> 2f 65 48 8b 04 25 28 00 00 00 48 89 44 24 28 31 c0 f0 ff 85 34 May 13 16:57:29 daniel-nas kernel: RSP: 0018:ffffc9000150bbf8 EFLAGS: 00010246 May 13 16:57:29 daniel-nas kernel: RAX: 0000000000000000 RBX: ffffc9000150bd00 RCX: 0000000000000001 May 13 16:57:29 daniel-nas kernel: RDX: 0000000000000000 RSI: ffffc9000150be30 RDI: 0000000000000000 May 13 16:57:29 daniel-nas kernel: RBP: 0000000000000000 R08: 0000000000000068 R09: 0000000000000000 May 13 16:57:29 daniel-nas kernel: R10: ffffc9000150be30 R11: ffffc9000150bc00 R12: ffffc9000150be08 May 13 16:57:29 daniel-nas kernel: R13: ffff888161fc9400 R14: ffff88815ecfbe00 R15: ffff88818af32100 May 13 16:57:29 daniel-nas kernel: FS: 0000149fac7c56c0(0000) GS:ffff88885f440000(0000) knlGS:0000000000000000 May 13 16:57:29 daniel-nas kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 13 16:57:29 daniel-nas kernel: CR2: 0000000000000000 CR3: 00000001a19c8006 CR4: 0000000000770ee0 May 13 16:57:29 daniel-nas kernel: PKRU: 55555554 May 13 16:57:40 daniel-nas kernel: traps: qbittorrent-nox[20426] trap int3 ip:152f0fd5d290 sp:152f0b9fbdd0 error:0 in libtorrent-rasterbar.so.2.0.8[152f0fa59000+39b000] May 13 16:57:40 daniel-nas kernel: traps: sh[3717] general protection fault ip:42fec0 sp:7ffe4b587580 error:0 in bash[426000+c5000] May 13 16:57:40 daniel-nas kernel: mce: [Hardware Error]: Machine check events logged
May 14, 20233 yr Author I think my disklocation plugin got corrupted somehow and was causing problems. The plugin settings page itself couldn't even open. I deleted the disklocation database and reinstalled the plugin, currently testing.
May 14, 20233 yr Author So after testing, I'm not 100% disklocation is the root cause. I think something else caused the disklocation plugin to get corrupted. I'm still seeing a bunch of errors in my logs, and just an hour ago, my server completely froze without a trace of any error logs, and the console screen was still stuck on login. These are the recent errrors that I am seeing. daniel-nas-syslog-20230514-2030.zip
May 14, 20233 yr Author I just checked /var/log/mcelog and I see a bunch of errors. Is there any way to determine whether this is a CPU or Motherboard issue? mcelog: failed to prefill DIMM database from DMI data Kernel does not support page offline interface Hardware event. This is not a software error. MCE 0 CPU 8 BANK 0 TSC c9fa45464b ADDR 149a9c617b87 TIME 1684100730 Sun May 14 17:45:30 2023 MCG status: MCi status: Corrected error MCi_ADDR register valid MCA: Instruction CACHE Level-1 Instruction-Fetch Error STATUS 8400004000040150 MCGSTATUS 0 MCGCAP c16 APICID 20 SOCKETID 0 MICROCODE 113 CPUID Vendor Intel Family 6 Model 183 Step 1 Hardware event. This is not a software error. MCE 0 CPU 8 BANK 0 TSC c9fa486d08 ADDR 3ffffff811e7fe0 TIME 1684100730 Sun May 14 17:45:30 2023 MCG status: MCi status: Corrected error MCi_ADDR register valid MCA: Instruction CACHE Level-1 Instruction-Fetch Error STATUS 8400008000040150 MCGSTATUS 0 MCGCAP c16 APICID 20 SOCKETID 0 MICROCODE 113 CPUID Vendor Intel Family 6 Model 183 Step 1 Hardware event. This is not a software error. MCE 0 CPU 8 BANK 0 TSC cacc37c105 ADDR 149a9c60d3a4 TIME 1684100731 Sun May 14 17:45:31 2023 MCG status: MCi status: Corrected error MCi_ADDR register valid MCA: Instruction CACHE Level-1 Instruction-Fetch Error STATUS 8400008000040150 MCGSTATUS 0 MCGCAP c16 APICID 20 SOCKETID 0 MICROCODE 113 CPUID Vendor Intel Family 6 Model 183 Step 1 Hardware event. This is not a software error. MCE 0 CPU 8 BANK 0 TSC 38baf480b69 TIME 1684101616 Sun May 14 18:00:16 2023 MCG status: MCi status: Corrected error MCA: Instruction CACHE Level-1 Instruction-Fetch Error STATUS 8000004000020150 MCGSTATUS 0 MCGCAP c16 APICID 20 SOCKETID 0 MICROCODE 113 CPUID Vendor Intel Family 6 Model 183 Step 1
May 15, 20233 yr Author I think I may just go exchange the CPU and motherboard for a new one at micro center. Hoping they will let me since I already threw away the CPU box.
May 15, 20233 yr I am having this exact same issue. SMARTCTL faults constantly then a crash. I do not have the disk location plugin. I did not thing to check the other logs really. I was so focused on the syslog. I went from AMD 3900x to an intel 12700k platform. I am in the same boat as you. I am exchanging my mobo and cpu Monday to microcenter. Good news after speaking with them over text they have stated that they will take the cpu and motherboard even without a box. ***Edit*** Just to add. I am on unraid 6.12.0-rc5 I have rolled back to RC2 as well to check as on the 3900x platform I did not have issues and same problem. I have also reinstalled unraid as a trail version on a new flash drive to test and errors still occur with version 6.11.5 Edited May 15, 20233 yr by soultaco83 Update to note unraid version
May 15, 20233 yr Author 13 minutes ago, soultaco83 said: I am having this exact same issue. SMARTCTL faults constantly then a crash. I do not have the disk location plugin. I did not thing to check the other logs really. I was so focused on the syslog. I went from AMD 3900x to an intel 12700k platform. I am in the same boat as you. I am exchanging my mobo and cpu Monday to microcenter. Good news after speaking with them over text they have stated that they will take the cpu and motherboard even without a box. ***Edit*** Just to add. I am on unraid 6.12.0-rc5 I have rolled back to RC2 as well to check as on the 3900x platform I did not have issues and same problem. I have also reinstalled unraid as a trail version on a new flash drive to test and errors still occur with version 6.11.5 Good to know that Micro Center is flexible with returns. I originally had the issues with 6.11.5 which threw kernel errors in syslog before crashing, then I upgraded to 6.12.0-rc5 and now I still get mce errors here and there, but the hard crash usually has no errors associated with it.
May 15, 20233 yr Author Solution Good news, exchanged both the motherboard and the processor and I no longer see those errors anymore.
May 16, 20233 yr That is indeed good news. I'm still seeing errors after my exchange. So still poking at it all.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.