Meehan

TL;DR: Server will intermittently lock up requiring a hardreset. It may then run for a few days without issue, or it might immediatly lockup again upon reboot, requiring multiple powercycles to regain functionality. Are Null pointers, CMCI Storms and tainted kernals indicitave of an old CPU dying? The server was built using my old gaming PC parts - specifically an i5 3570k and a MSI - Z77A-GD65 motherboard which I have added drives to using the mothboards SATA headers (no HBA cards for me). The CPU and Mobo are from 2012 so they're certainly some pretty old hardward that have seen extensive use over the last decade+. I have run a 10 hour Memtest84 load on the 16GB of RAM and they have reported no errors. I am not super "syslog literate" but at lines 1333 - 1385 & 1548 - 1626: it looks like something is going wrong. Lines 1548 - 1626 are the exact moment that the server locked up and would not respond to pings, ssh logins, web-ui etc. and can only be recovered by repeatedly powercycling the system. I'm hoping some of the more experienced members of the forum might have an idea of what's going on or where I should start looking. ______ Syslog exerpts Lines 1333 - 1385: Aug 18 23:06:56 Zeus kernel: BUG: unable to handle page fault for address: 0000000000032190 Aug 18 23:06:56 Zeus kernel: #PF: supervisor write access in kernel mode Aug 18 23:06:56 Zeus kernel: #PF: error_code(0x0002) - not-present page Aug 18 23:06:56 Zeus kernel: PGD 8000000145484067 P4D 8000000145484067 PUD 13f5b6067 PMD 0 Aug 18 23:06:56 Zeus kernel: Oops: 0002 [#1] PREEMPT SMP PTI Aug 18 23:06:56 Zeus kernel: CPU: 0 PID: 207 Comm: kworker/u8:4 Tainted: P O 6.1.38-Unraid #2 Aug 18 23:06:56 Zeus kernel: Hardware name: MSI MS-7751/Z77A-GD65 GAMING (MS-7751), BIOS V25.0 03/20/2013 Aug 18 23:06:56 Zeus kernel: Workqueue: btrfs-endio btrfs_end_bio_work Aug 18 23:06:56 Zeus kernel: RIP: 0010:endio_readpage_release_extent+0xb0/0xb7 Aug 18 23:06:56 Zeus kernel: Code: 89 6b 08 48 89 6b 10 44 88 63 18 48 8b 44 24 08 65 48 2b 04 25 28 00 00 00 74 05 e8 0c cb 51 00 48 83 c4 10 5b 5d 41 5c 41 5d <41> 5e c3 cc cc cc cc e8 de e2 ff ff f0 48 0f ba 28 00 0f 93 c0 0f Aug 18 23:06:56 Zeus kernel: RSP: 0018:ffffc9000046bdb0 EFLAGS: 00010286 Aug 18 23:06:56 Zeus kernel: RAX: 0000000000000000 RBX: 0000000000001000 RCX: 0000000000000000 Aug 18 23:06:56 Zeus kernel: RDX: 0000000001203400 RSI: ffffc9000046bcd8 RDI: 0000000000032190 Aug 18 23:06:56 Zeus kernel: RBP: ffffc9000046be90 R08: ffff8881615011f8 R09: ffffffff813b89df Aug 18 23:06:56 Zeus kernel: R10: ffff88815a36f750 R11: fefefefefefefeff R12: 00000001ea9e3000 Aug 18 23:06:56 Zeus kernel: R13: 0000000500000000 R14: 0000000000000000 R15: ffff8881357e8ae8 Aug 18 23:06:56 Zeus kernel: FS: 0000000000000000(0000) GS:ffff88840f600000(0000) knlGS:0000000000000000 Aug 18 23:06:56 Zeus kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 18 23:06:56 Zeus kernel: CR2: 0000000000032190 CR3: 0000000104cb0001 CR4: 00000000001706f0 Aug 18 23:06:56 Zeus kernel: Call Trace: Aug 18 23:06:56 Zeus kernel: <TASK> Aug 18 23:06:56 Zeus kernel: ? __die_body+0x1a/0x5c Aug 18 23:06:56 Zeus kernel: ? page_fault_oops+0x329/0x376 Aug 18 23:06:56 Zeus kernel: ? fixup_exception+0x22/0x24b Aug 18 23:06:56 Zeus kernel: ? exc_page_fault+0xfb/0x11d Aug 18 23:06:56 Zeus kernel: ? asm_exc_page_fault+0x22/0x30 Aug 18 23:06:56 Zeus kernel: ? __clear_extent_bit+0x314/0x329 Aug 18 23:06:56 Zeus kernel: ? endio_readpage_release_extent+0xb0/0xb7 Aug 18 23:06:56 Zeus kernel: end_bio_extent_readpage+0x4a5/0x4fe Aug 18 23:06:56 Zeus kernel: ? process_one_work+0x1ab/0x295 Aug 18 23:06:56 Zeus kernel: process_one_work+0x1ab/0x295 Aug 18 23:06:56 Zeus kernel: worker_thread+0x18b/0x244 Aug 18 23:06:56 Zeus kernel: ? rescuer_thread+0x281/0x281 Aug 18 23:06:56 Zeus kernel: kthread+0xe7/0xef Aug 18 23:06:56 Zeus kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 18 23:06:56 Zeus kernel: ret_from_fork+0x22/0x30 Aug 18 23:06:56 Zeus kernel: </TASK> Aug 18 23:06:56 Zeus kernel: Modules linked in: macvlan veth xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter xfs md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) f71882fg iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs af_packet 8021q garp mrp bridge stp llc bonding tls i915 intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm iosf_mbi drm_buddy i2c_algo_bit ttm drm_display_helper drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel sha512_ssse3 drm aesni_intel crypto_simd cryptd rapl intel_cstate mei_hdcp mei_pxp mxm_wmi intel_gtt i2c_i801 intel_uncore agpgart mei_me i2c_smbus i2c_core syscopyarea Aug 18 23:06:56 Zeus kernel: sysfillrect sysimgblt firewire_ohci ahci alx firewire_core libahci mdio mei fb_sys_fops fan thermal video wmi backlight button unix Aug 18 23:06:56 Zeus kernel: CR2: 0000000000032190 Aug 18 23:06:56 Zeus kernel: ---[ end trace 0000000000000000 ]--- Aug 18 23:06:56 Zeus kernel: RIP: 0010:endio_readpage_release_extent+0xb0/0xb7 Aug 18 23:06:56 Zeus kernel: Code: 89 6b 08 48 89 6b 10 44 88 63 18 48 8b 44 24 08 65 48 2b 04 25 28 00 00 00 74 05 e8 0c cb 51 00 48 83 c4 10 5b 5d 41 5c 41 5d <41> 5e c3 cc cc cc cc e8 de e2 ff ff f0 48 0f ba 28 00 0f 93 c0 0f Aug 18 23:06:56 Zeus kernel: RSP: 0018:ffffc9000046bdb0 EFLAGS: 00010286 Aug 18 23:06:56 Zeus kernel: RAX: 0000000000000000 RBX: 0000000000001000 RCX: 0000000000000000 Aug 18 23:06:56 Zeus kernel: RDX: 0000000001203400 RSI: ffffc9000046bcd8 RDI: 0000000000032190 Aug 18 23:06:56 Zeus kernel: RBP: ffffc9000046be90 R08: ffff8881615011f8 R09: ffffffff813b89df Aug 18 23:06:56 Zeus kernel: R10: ffff88815a36f750 R11: fefefefefefefeff R12: 00000001ea9e3000 Aug 18 23:06:56 Zeus kernel: R13: 0000000500000000 R14: 0000000000000000 R15: ffff8881357e8ae8 Aug 18 23:06:56 Zeus kernel: FS: 0000000000000000(0000) GS:ffff88840f600000(0000) knlGS:0000000000000000 Aug 18 23:06:56 Zeus kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 18 23:06:56 Zeus kernel: CR2: 0000000000032190 CR3: 0000000104cb0001 CR4: 00000000001706f0 Aug 18 23:06:56 Zeus kernel: note: kworker/u8:4[207] exited with irqs disabled 1548 - 1626: Aug 18 23:07:16 Zeus kernel: BUG: kernel NULL pointer dereference, address: 0000000000000065 Aug 18 23:07:16 Zeus kernel: #PF: supervisor read access in kernel mode Aug 18 23:07:16 Zeus kernel: #PF: error_code(0x0000) - not-present page Aug 18 23:07:16 Zeus kernel: PGD 80000002f3602067 P4D 80000002f3602067 PUD 2f357b067 PMD 0 Aug 18 23:07:16 Zeus kernel: Oops: 0000 [#2] PREEMPT SMP PTI Aug 18 23:07:16 Zeus kernel: CPU: 0 PID: 10285 Comm: influxd Tainted: P D O 6.1.38-Unraid #2 Aug 18 23:07:16 Zeus kernel: Hardware name: MSI MS-7751/Z77A-GD65 GAMING (MS-7751), BIOS V25.0 03/20/2013 Aug 18 23:07:16 Zeus kernel: RIP: 0010:_btrfs_printk+0x6f/0x154 Aug 18 23:07:16 Zeus kernel: Code: 48 89 54 24 40 31 d2 48 89 74 24 18 48 8d 74 24 48 66 c7 44 24 2c 00 00 c6 44 24 2e 00 c7 44 24 10 10 00 00 00 48 89 74 24 20 <80> 38 01 75 4b 8a 48 01 84 c9 74 44 80 f9 37 0f be d1 7f 31 80 f9 Aug 18 23:07:16 Zeus kernel: RSP: 0018:ffffc9000a1ef920 EFLAGS: 00010246 Aug 18 23:07:16 Zeus kernel: RAX: 0000000000000065 RBX: 0000000045879000 RCX: 0000000000004000 Aug 18 23:07:16 Zeus kernel: RDX: 0000000000000000 RSI: ffffc9000a1ef968 RDI: ffffffff822fc5c0 Aug 18 23:07:16 Zeus kernel: RBP: ffffc9000a1ef9a8 R08: 0000000000000000 R09: 0000000000000011 Aug 18 23:07:16 Zeus kernel: R10: 00000000000071b4 R11: ffff8881fff82f68 R12: ffffffff8207bfb6 Aug 18 23:07:16 Zeus kernel: R13: ffff8881a91c5000 R14: 0000000000000000 R15: ffffc9000a1efaf7 Aug 18 23:07:16 Zeus kernel: FS: 0000145ffdff3b20(0000) GS:ffff88840f600000(0000) knlGS:0000000000000000 Aug 18 23:07:16 Zeus kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 18 23:07:16 Zeus kernel: CR2: 0000000000000065 CR3: 00000002ecfda003 CR4: 00000000001706f0 Aug 18 23:07:16 Zeus kernel: Call Trace: Aug 18 23:07:16 Zeus kernel: <TASK> Aug 18 23:07:16 Zeus kernel: ? __die_body+0x1a/0x5c Aug 18 23:07:16 Zeus kernel: ? page_fault_oops+0x329/0x376 Aug 18 23:07:16 Zeus kernel: ? fixup_exception+0x22/0x24b Aug 18 23:07:16 Zeus kernel: ? exc_page_fault+0xfb/0x11d Aug 18 23:07:16 Zeus kernel: ? asm_exc_page_fault+0x22/0x30 Aug 18 23:07:16 Zeus kernel: ? _btrfs_printk+0x6f/0x154 Aug 18 23:07:16 Zeus kernel: check_eb_range+0x2f/0x39 Aug 18 23:07:16 Zeus kernel: read_extent_buffer+0x22/0x9b Aug 18 23:07:16 Zeus kernel: btrfs_verify_level_key+0x100/0x175 Aug 18 23:07:16 Zeus kernel: read_block_for_search+0x10a/0x2a1 Aug 18 23:07:16 Zeus kernel: btrfs_next_old_leaf+0x1bf/0x343 Aug 18 23:07:16 Zeus kernel: ? kmem_cache_alloc+0x122/0x14d Aug 18 23:07:16 Zeus kernel: btrfs_load_inode_props+0x9c/0x300 Aug 18 23:07:16 Zeus kernel: ? __set_extent_bit+0x19f/0x499 Aug 18 23:07:16 Zeus kernel: ? acls_after_inode_item+0x34/0x113 Aug 18 23:07:16 Zeus kernel: ? btrfs_get_64+0x72/0xdf Aug 18 23:07:16 Zeus kernel: btrfs_iget_path+0x45a/0x57b Aug 18 23:07:16 Zeus kernel: ? btrfs_lookup_dentry+0xfd/0x442 Aug 18 23:07:16 Zeus kernel: btrfs_lookup_dentry+0x16e/0x442 Aug 18 23:07:16 Zeus kernel: ? __d_lookup+0x8b/0x9d Aug 18 23:07:16 Zeus kernel: btrfs_lookup+0xf/0x24 Aug 18 23:07:16 Zeus kernel: path_openat+0x515/0xa4d Aug 18 23:07:16 Zeus kernel: do_filp_open+0x55/0xb8 Aug 18 23:07:16 Zeus kernel: ? btrfs_real_readdir+0x36d/0x39e Aug 18 23:07:16 Zeus kernel: ? getname_flags+0x29/0x152 Aug 18 23:07:16 Zeus kernel: ? kmem_cache_alloc+0x122/0x14d Aug 18 23:07:16 Zeus kernel: ? _raw_spin_unlock+0x14/0x29 Aug 18 23:07:16 Zeus kernel: do_sys_openat2+0x6c/0xd9 Aug 18 23:07:16 Zeus kernel: do_sys_open+0x3a/0x5a Aug 18 23:07:16 Zeus kernel: do_syscall_64+0x6b/0x81 Aug 18 23:07:16 Zeus kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd Aug 18 23:07:16 Zeus kernel: RIP: 0033:0x489d6a Aug 18 23:07:16 Zeus kernel: Code: e8 1b ae fe ff 48 8b 7c 24 10 48 8b 74 24 18 48 8b 54 24 20 4c 8b 54 24 28 4c 8b 44 24 30 4c 8b 4c 24 38 48 8b 44 24 08 0f 05 <48> 3d 01 f0 ff ff 76 20 48 c7 44 24 40 ff ff ff ff 48 c7 44 24 48 Aug 18 23:07:16 Zeus kernel: RSP: 002b:000000c000d78ec8 EFLAGS: 00000216 ORIG_RAX: 0000000000000101 Aug 18 23:07:16 Zeus kernel: RAX: ffffffffffffffda RBX: 000000c000063800 RCX: 0000000000489d6a Aug 18 23:07:16 Zeus kernel: RDX: 0000000000080000 RSI: 000000c001b672c0 RDI: ffffffffffffff9c Aug 18 23:07:16 Zeus kernel: RBP: 000000c000d78f58 R08: 0000000000000000 R09: 0000000000000000 Aug 18 23:07:16 Zeus kernel: R10: 00000000000001b6 R11: 0000000000000216 R12: 000000c001b672c0 Aug 18 23:07:16 Zeus kernel: R13: 0000000000000001 R14: 000000c0016d9380 R15: ffffffffffffffff Aug 18 23:07:16 Zeus kernel: </TASK> Aug 18 23:07:16 Zeus kernel: Modules linked in: xt_mark tun macvlan veth xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter xfs md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) f71882fg iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs af_packet 8021q garp mrp bridge stp llc bonding tls i915 intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm iosf_mbi drm_buddy i2c_algo_bit ttm drm_display_helper drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel sha512_ssse3 drm aesni_intel crypto_simd cryptd rapl intel_cstate mei_hdcp mei_pxp mxm_wmi intel_gtt i2c_i801 intel_uncore agpgart mei_me i2c_smbus i2c_core Aug 18 23:07:16 Zeus kernel: syscopyarea sysfillrect sysimgblt firewire_ohci ahci alx firewire_core libahci mdio mei fb_sys_fops fan thermal video wmi backlight button unix Aug 18 23:07:16 Zeus kernel: CR2: 0000000000000065 Aug 18 23:07:16 Zeus kernel: ---[ end trace 0000000000000000 ]--- Aug 18 23:07:16 Zeus kernel: RIP: 0010:endio_readpage_release_extent+0xb0/0xb7 Aug 18 23:07:16 Zeus kernel: Code: 89 6b 08 48 89 6b 10 44 88 63 18 48 8b 44 24 08 65 48 2b 04 25 28 00 00 00 74 05 e8 0c cb 51 00 48 83 c4 10 5b 5d 41 5c 41 5d <41> 5e c3 cc cc cc cc e8 de e2 ff ff f0 48 0f ba 28 00 0f 93 c0 0f Aug 18 23:07:16 Zeus kernel: RSP: 0018:ffffc9000046bdb0 EFLAGS: 00010286 Aug 18 23:07:16 Zeus kernel: RAX: 0000000000000000 RBX: 0000000000001000 RCX: 0000000000000000 Aug 18 23:07:16 Zeus kernel: RDX: 0000000001203400 RSI: ffffc9000046bcd8 RDI: 0000000000032190 Aug 18 23:07:16 Zeus kernel: RBP: ffffc9000046be90 R08: ffff8881615011f8 R09: ffffffff813b89df Aug 18 23:07:16 Zeus kernel: R10: ffff88815a36f750 R11: fefefefefefefeff R12: 00000001ea9e3000 Aug 18 23:07:16 Zeus kernel: R13: 0000000500000000 R14: 0000000000000000 R15: ffff8881357e8ae8 Aug 18 23:07:16 Zeus kernel: FS: 0000145ffdff3b20(0000) GS:ffff88840f600000(0000) knlGS:0000000000000000 Aug 18 23:07:16 Zeus kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 18 23:07:16 Zeus kernel: CR2: 0000000000000065 CR3: 00000002ecfda003 CR4: 00000000001706f0 Aug 18 23:07:23 Zeus kernel: mce: CMCI storm detected: switching to poll mode Aug 18 23:07:23 Zeus kernel: celery[10404]: segfault at 0 ip 0000146edc78617d sp 00007fff06ad8370 error 6 in libpython3.9.so.1.0[146edc661000+198000] likely on CPU 0 (core 0, socket 0) Aug 18 23:07:23 Zeus kernel: Code: a8 00 00 00 25 00 02 00 00 c3 0f 1f 00 41 55 41 54 48 83 ec 28 4c 8b 25 41 7a 16 00 64 48 8b 04 25 28 00 00 00 48 89 44 24 18 <31> c0 4c 39 66 08 75 6b 80 7e 20 00 79 65 48 83 7e 10 64 7f 5e f6 Aug 18 23:08:55 Zeus sshd[4518]: fatal: Timeout before authentication for 10.10.10.167 port 53301 syslog.txt

I too would love to hear if anyone has a better solution than Filezilla.

Hi there, would you be able to make an icon of my old Fractal Design Arc XL? I really appreciate the little touches like this that separate UnRaid from the competitors!