fraserallanuk

Hello. Hoping someone can help me. My unRAID Server periodically locks up and/or crashes. I've managed to rather the following from a syslog server that I have setup. Currently, the unRAID webui in unreachable (although nginx and other docker containers are), and I am able to get to the username SSH prompt, however, after entering this, it doesnt proceed to the password prompt and eventually times out. Hopefully the below log might help point in the right direction of the issue. Thanks

Hello, Since replacing Docker.img, the crash happened again. Logs from syslog show the following: Jan 7 13:58:20 Dougie-unRAID kernel: BUG: kernel NULL pointer dereference, address: 0000000000000080 Jan 7 13:58:20 Dougie-unRAID kernel: #PF: supervisor read access in kernel mode Jan 7 13:58:20 Dougie-unRAID kernel: #PF: error_code(0x0000) - not-present page Jan 7 13:58:20 Dougie-unRAID kernel: PGD 0 P4D 0 Jan 7 13:58:20 Dougie-unRAID kernel: Oops: 0000 [#1] PREEMPT SMP PTI Jan 7 13:58:20 Dougie-unRAID kernel: CPU: 0 PID: 9211 Comm: Plex Transcoder Tainted: P O 6.1.64-Unraid #1 Jan 7 13:58:20 Dougie-unRAID kernel: Hardware name: Dell Inc. OptiPlex 5060/0654JC, BIOS 1.2.22 11/01/2018 Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0010:pick_next_task_fair+0x90/0x2a7 Jan 7 13:58:20 Dougie-unRAID kernel: Code: ed 48 89 ef e8 84 dd ff ff 84 c0 74 12 83 bb 90 00 00 00 00 0f 84 4f 01 00 00 e9 8f 00 00 00 4c 89 ee 48 89 ef e8 df b6 ff ff <48> 8b a8 80 00 00 00 49 89 c5 48 85 ed 75 ab 48 89 c7 e8 c4 6e ff Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 0018:ffffc9000f65fcf8 EFLAGS: 00010046 Jan 7 13:58:20 Dougie-unRAID kernel: RAX: 0000000000000000 RBX: ffff888a1442d440 RCX: 000000000000011c Jan 7 13:58:20 Dougie-unRAID kernel: RDX: ffffc9000f65fd40 RSI: 0000000000000000 RDI: ffff88815a9d4a00 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: ffff88815a9d4a00 R08: 000000000000001c R09: 0000000000000aab Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff88822591d140 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000000 R14: ffff888a1442d4c0 R15: ffffc9000f65fd40 Jan 7 13:58:20 Dougie-unRAID kernel: FS: 0000150090de7b38(0000) GS:ffff888a14400000(0000) knlGS:0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 7 13:58:20 Dougie-unRAID kernel: CR2: 0000000000000080 CR3: 00000001e2be8005 CR4: 00000000003706f0 Jan 7 13:58:20 Dougie-unRAID kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Jan 7 13:58:20 Dougie-unRAID kernel: Call Trace: Jan 7 13:58:20 Dougie-unRAID kernel: <TASK> Jan 7 13:58:20 Dougie-unRAID kernel: ? __die_body+0x1a/0x5c Jan 7 13:58:20 Dougie-unRAID kernel: ? page_fault_oops+0x329/0x376 Jan 7 13:58:20 Dougie-unRAID kernel: ? do_user_addr_fault+0x12e/0x48d Jan 7 13:58:20 Dougie-unRAID kernel: ? exc_page_fault+0xfb/0x11d Jan 7 13:58:20 Dougie-unRAID kernel: ? asm_exc_page_fault+0x22/0x30 Jan 7 13:58:20 Dougie-unRAID kernel: ? pick_next_task_fair+0x90/0x2a7 Jan 7 13:58:20 Dougie-unRAID kernel: ? pick_next_task_fair+0x90/0x2a7 Jan 7 13:58:20 Dougie-unRAID kernel: __schedule+0x1c4/0x612 Jan 7 13:58:20 Dougie-unRAID kernel: schedule+0x8e/0xcc Jan 7 13:58:20 Dougie-unRAID kernel: futex_wait_queue+0x4e/0x82 Jan 7 13:58:20 Dougie-unRAID kernel: futex_wait+0xb7/0x17e Jan 7 13:58:20 Dougie-unRAID kernel: ? xfd_validate_state+0x1d/0x71 Jan 7 13:58:20 Dougie-unRAID kernel: do_futex+0xcd/0x143 Jan 7 13:58:20 Dougie-unRAID kernel: __do_sys_futex+0x127/0x150 Jan 7 13:58:20 Dougie-unRAID kernel: do_syscall_64+0x68/0x81 Jan 7 13:58:20 Dougie-unRAID kernel: entry_SYSCALL_64_after_hwframe+0x64/0xce Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0033:0x150098beb739 Jan 7 13:58:20 Dougie-unRAID kernel: Code: c0 0f 85 24 00 00 00 49 89 fb 48 89 f0 48 89 d7 48 89 ce 4c 89 c2 4d 89 ca 4c 8b 44 24 08 4c 8b 4c 24 10 4c 89 5c 24 08 0f 05 <c3> e9 d5 cb ff ff 41 57 41 56 53 48 81 ec 90 00 00 00 49 89 f6 b8 Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 002b:0000150090de7968 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca Jan 7 13:58:20 Dougie-unRAID kernel: RAX: ffffffffffffffda RBX: 00000000000000ca RCX: 0000150098beb739 Jan 7 13:58:20 Dougie-unRAID kernel: RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000150090de7a44 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: 0000150090de7a44 R08: 0000000000000000 R09: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000150090de7b38 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000002 R14: 0000000000000000 R15: 0000150090de7b74 Jan 7 13:58:20 Dougie-unRAID kernel: </TASK> Jan 7 13:58:20 Dougie-unRAID kernel: Modules linked in: cmac cifs asn1_decoder cifs_arc4 cifs_md4 oid_registry dns_resolver af_packet bluetooth ecdh_generic ecc xt_nat xt_CHECKSUM ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap ipvlan xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype br_netfilter xfs md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs bridge 8021q garp mrp stp llc bonding tls intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp i915 kvm_intel kvm iosf_mbi crct10dif_pclmul drm_buddy crc32_pclmul crc32c_intel i2c_algo_bit ghash_clmulni_intel sha512_ssse3 ttm sha256_ssse3 sha1_ssse3 drm_display_helper aesni_intel drm_kms_helper crypto_simd cryptd tpm_crb mei_hdcp mei_pxp wmi_bmof i2c_i801 rapl Jan 7 13:58:20 Dougie-unRAID kernel: intel_cstate drm nvme ahci e1000e intel_uncore input_leds intel_gtt led_class mei_me i2c_smbus agpgart nvme_core mei i2c_core video libahci intel_pch_thermal syscopyarea sysfillrect sysimgblt fb_sys_fops tpm_tis thermal fan wmi tpm_tis_core backlight tpm acpi_pad intel_pmc_core button unix Jan 7 13:58:20 Dougie-unRAID kernel: CR2: 0000000000000080 Jan 7 13:58:20 Dougie-unRAID kernel: ---[ end trace 0000000000000000 ]--- Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0010:pick_next_task_fair+0x90/0x2a7 Jan 7 13:58:20 Dougie-unRAID kernel: Code: ed 48 89 ef e8 84 dd ff ff 84 c0 74 12 83 bb 90 00 00 00 00 0f 84 4f 01 00 00 e9 8f 00 00 00 4c 89 ee 48 89 ef e8 df b6 ff ff <48> 8b a8 80 00 00 00 49 89 c5 48 85 ed 75 ab 48 89 c7 e8 c4 6e ff Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 0018:ffffc9000f65fcf8 EFLAGS: 00010046 Jan 7 13:58:20 Dougie-unRAID kernel: RAX: 0000000000000000 RBX: ffff888a1442d440 RCX: 000000000000011c Jan 7 13:58:20 Dougie-unRAID kernel: RDX: ffffc9000f65fd40 RSI: 0000000000000000 RDI: ffff88815a9d4a00 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: ffff88815a9d4a00 R08: 000000000000001c R09: 0000000000000aab Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff88822591d140 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000000 R14: ffff888a1442d4c0 R15: ffffc9000f65fd40 Jan 7 13:58:20 Dougie-unRAID kernel: FS: 0000150090de7b38(0000) GS:ffff888a14400000(0000) knlGS:0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 7 13:58:20 Dougie-unRAID kernel: CR2: 0000000000000080 CR3: 00000001e2be8005 CR4: 00000000003706f0 Jan 7 13:58:20 Dougie-unRAID kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Jan 7 13:58:20 Dougie-unRAID kernel: note: Plex Transcoder[9211] exited with irqs disabled Jan 7 13:58:20 Dougie-unRAID kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 0-...D } 21169 jiffies s: 356197 root: 0x1/. Jan 7 13:58:20 Dougie-unRAID kernel: rcu: blocking rcu_node structures (internal RCU debug): Jan 7 13:58:20 Dougie-unRAID kernel: Sending NMI from CPU 2 to CPUs 0: Jan 7 13:58:20 Dougie-unRAID kernel: NMI backtrace for cpu 0 Jan 7 13:58:20 Dougie-unRAID kernel: CPU: 0 PID: 9211 Comm: Plex Transcoder Tainted: P D O 6.1.64-Unraid #1 Jan 7 13:58:20 Dougie-unRAID kernel: Hardware name: Dell Inc. OptiPlex 5060/0654JC, BIOS 1.2.22 11/01/2018 Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: Code: c2 0f b6 d2 c1 e2 08 30 e4 09 d0 3d ff 00 00 00 76 0c 0f ba e0 08 72 1e c6 43 01 00 eb 18 85 c0 74 0a 8b 03 84 c0 74 04 f3 90 <eb> f6 66 c7 03 01 00 e9 32 01 00 00 e8 bd 40 ff ff 49 c7 c4 80 e1 Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 0018:ffffc90000003f50 EFLAGS: 00000002 Jan 7 13:58:20 Dougie-unRAID kernel: RAX: 0000000000080101 RBX: ffff888a1442d440 RCX: 000000000000080b Jan 7 13:58:20 Dougie-unRAID kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff888a1442d440 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: ffff888a1442d440 R08: 0000000000000000 R09: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: ffffc90000003ff8 R12: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: FS: 0000150090de7b38(0000) GS:ffff888a14400000(0000) knlGS:0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 7 13:58:20 Dougie-unRAID kernel: CR2: 0000000000000080 CR3: 00000001e2be8005 CR4: 00000000003706f0 Jan 7 13:58:20 Dougie-unRAID kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Jan 7 13:58:20 Dougie-unRAID kernel: Call Trace: Jan 7 13:58:20 Dougie-unRAID kernel: <NMI> Jan 7 13:58:20 Dougie-unRAID kernel: ? nmi_cpu_backtrace+0xd3/0x104 Jan 7 13:58:20 Dougie-unRAID kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Jan 7 13:58:20 Dougie-unRAID kernel: ? nmi_handle+0x54/0x131 Jan 7 13:58:20 Dougie-unRAID kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: ? default_do_nmi+0x66/0x15b Jan 7 13:58:20 Dougie-unRAID kernel: ? exc_nmi+0xbf/0x130 Jan 7 13:58:20 Dougie-unRAID kernel: ? end_repeat_nmi+0x16/0x67 Jan 7 13:58:20 Dougie-unRAID kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: </NMI> Jan 7 13:58:20 Dougie-unRAID kernel: <IRQ> Jan 7 13:58:20 Dougie-unRAID kernel: do_raw_spin_lock+0x14/0x1a Jan 7 13:58:20 Dougie-unRAID kernel: raw_spin_rq_lock_nested+0x1b/0x26 Jan 7 13:58:20 Dougie-unRAID kernel: _raw_spin_rq_lock_irqsave+0x15/0x1e Jan 7 13:58:20 Dougie-unRAID kernel: rq_lock_irqsave+0x12/0x27 Jan 7 13:58:20 Dougie-unRAID kernel: sched_ttwu_pending+0x53/0xe0 Jan 7 13:58:20 Dougie-unRAID kernel: ? __flush_smp_call_function_queue+0x43/0x1a5 Jan 7 13:58:20 Dougie-unRAID kernel: __sysvec_call_function_single+0x59/0xa4 Jan 7 13:58:20 Dougie-unRAID kernel: sysvec_call_function_single+0x80/0xa6 Jan 7 13:58:20 Dougie-unRAID kernel: </IRQ> Jan 7 13:58:20 Dougie-unRAID kernel: <TASK> Jan 7 13:58:20 Dougie-unRAID kernel: asm_sysvec_call_function_single+0x16/0x20 Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0010:make_task_dead+0x69/0x11c Jan 7 13:58:20 Dougie-unRAID kernel: Code: 58 0f 1f 40 00 0f ba e0 09 72 1f 8b 93 88 04 00 00 48 8d b3 80 06 00 00 48 c7 c7 82 24 0c 82 e8 a9 d6 7e 00 fb 0f 1f 44 00 00 <65> 8b 0d 58 a4 fa 7e 81 e1 ff ff ff 7f 74 3f 65 48 8b 34 25 c0 cb Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 0018:ffffc9000f65ff40 EFLAGS: 00000246 Jan 7 13:58:20 Dougie-unRAID kernel: RAX: 0000000000000035 RBX: ffff88822591d140 RCX: 0000000000000027 Jan 7 13:58:20 Dougie-unRAID kernel: RDX: 0000000000000000 RSI: ffffffff820d7e01 RDI: 00000000ffffffff Jan 7 13:58:20 Dougie-unRAID kernel: RBP: 0000000000000009 R08: 0000000000000000 R09: 0000000000aaaaaa Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000080 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: rewind_stack_and_make_dead+0x17/0x17 Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0033:0x150098beb739 Jan 7 13:58:20 Dougie-unRAID kernel: Code: c0 0f 85 24 00 00 00 49 89 fb 48 89 f0 48 89 d7 48 89 ce 4c 89 c2 4d 89 ca 4c 8b 44 24 08 4c 8b 4c 24 10 4c 89 5c 24 08 0f 05 <c3> e9 d5 cb ff ff 41 57 41 56 53 48 81 ec 90 00 00 00 49 89 f6 b8 Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 002b:0000150090de7968 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca Jan 7 13:58:20 Dougie-unRAID kernel: RAX: ffffffffffffffda RBX: 00000000000000ca RCX: 0000150098beb739 Jan 7 13:58:20 Dougie-unRAID kernel: RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000150090de7a44 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: 0000150090de7a44 R08: 0000000000000000 R09: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000150090de7b38 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000002 R14: 0000000000000000 R15: 0000150090de7b74 Jan 7 13:58:20 Dougie-unRAID kernel: </TASK> Full syslog attached. syslog syslog-previous

Ahh sorry, I didnt realise it wasnt included (im fairly new to unRAID) I hooked a monitor up to the unRAID box and noticed some BTRFS errors (attached) so I followed a guide to replace the docker.img. This was only this morning, when I made this post so still too early to see if that has resolved the problem yet. Jan 3 08:11:40 Dougie-unRAID root: Creating new image file: '/mnt/cache/system/docker/docker.img' size: 20G This machine used to run uBuntu server without any issues. The lockup issues only started since I reformatted and moved to unRAID. syslog syslog-previous

Hello, I have a small unRAID server which seems to locking up and I'm hoping someone can point me in the direction on how to diagnose. When it locks up, I loose access to the unRAID web ui. If I SSH to the box, or physically login, then I can enter the username but unable to proceed to the actual command prompt. I have enabled syslog writing to USB disk. This unRAID box runs a few docker containers (on various VLANs). There are no Virtual Machines. I have replaced the USB flash but this didnt resolve my issue. Many thanks, Fraser dougie-unraid-diagnostics-20240103-0802.zip.crdownload