January 3, 20242 yr Hello, I have a small unRAID server which seems to locking up and I'm hoping someone can point me in the direction on how to diagnose. When it locks up, I loose access to the unRAID web ui. If I SSH to the box, or physically login, then I can enter the username but unable to proceed to the actual command prompt. I have enabled syslog writing to USB disk. This unRAID box runs a few docker containers (on various VLANs). There are no Virtual Machines. I have replaced the USB flash but this didnt resolve my issue. Many thanks, Fraser dougie-unraid-diagnostics-20240103-0802.zip.crdownload
January 3, 20242 yr 51 minutes ago, fraserallanuk said: I have enabled syslog writing to USB disk. Hi, can you also share that file if it covers a crash ? It is not included in the diagnostics.
January 3, 20242 yr Author 5 hours ago, ChatNoir said: Hi, can you also share that file if it covers a crash ? It is not included in the diagnostics. Ahh sorry, I didnt realise it wasnt included (im fairly new to unRAID) I hooked a monitor up to the unRAID box and noticed some BTRFS errors (attached) so I followed a guide to replace the docker.img. This was only this morning, when I made this post so still too early to see if that has resolved the problem yet. Jan 3 08:11:40 Dougie-unRAID root: Creating new image file: '/mnt/cache/system/docker/docker.img' size: 20G This machine used to run uBuntu server without any issues. The lockup issues only started since I reformatted and moved to unRAID. syslog syslog-previous Edited January 3, 20242 yr by fraserallanuk
January 7, 20242 yr Author Hello, Since replacing Docker.img, the crash happened again. Logs from syslog show the following: Jan 7 13:58:20 Dougie-unRAID kernel: BUG: kernel NULL pointer dereference, address: 0000000000000080 Jan 7 13:58:20 Dougie-unRAID kernel: #PF: supervisor read access in kernel mode Jan 7 13:58:20 Dougie-unRAID kernel: #PF: error_code(0x0000) - not-present page Jan 7 13:58:20 Dougie-unRAID kernel: PGD 0 P4D 0 Jan 7 13:58:20 Dougie-unRAID kernel: Oops: 0000 [#1] PREEMPT SMP PTI Jan 7 13:58:20 Dougie-unRAID kernel: CPU: 0 PID: 9211 Comm: Plex Transcoder Tainted: P O 6.1.64-Unraid #1 Jan 7 13:58:20 Dougie-unRAID kernel: Hardware name: Dell Inc. OptiPlex 5060/0654JC, BIOS 1.2.22 11/01/2018 Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0010:pick_next_task_fair+0x90/0x2a7 Jan 7 13:58:20 Dougie-unRAID kernel: Code: ed 48 89 ef e8 84 dd ff ff 84 c0 74 12 83 bb 90 00 00 00 00 0f 84 4f 01 00 00 e9 8f 00 00 00 4c 89 ee 48 89 ef e8 df b6 ff ff <48> 8b a8 80 00 00 00 49 89 c5 48 85 ed 75 ab 48 89 c7 e8 c4 6e ff Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 0018:ffffc9000f65fcf8 EFLAGS: 00010046 Jan 7 13:58:20 Dougie-unRAID kernel: RAX: 0000000000000000 RBX: ffff888a1442d440 RCX: 000000000000011c Jan 7 13:58:20 Dougie-unRAID kernel: RDX: ffffc9000f65fd40 RSI: 0000000000000000 RDI: ffff88815a9d4a00 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: ffff88815a9d4a00 R08: 000000000000001c R09: 0000000000000aab Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff88822591d140 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000000 R14: ffff888a1442d4c0 R15: ffffc9000f65fd40 Jan 7 13:58:20 Dougie-unRAID kernel: FS: 0000150090de7b38(0000) GS:ffff888a14400000(0000) knlGS:0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 7 13:58:20 Dougie-unRAID kernel: CR2: 0000000000000080 CR3: 00000001e2be8005 CR4: 00000000003706f0 Jan 7 13:58:20 Dougie-unRAID kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Jan 7 13:58:20 Dougie-unRAID kernel: Call Trace: Jan 7 13:58:20 Dougie-unRAID kernel: <TASK> Jan 7 13:58:20 Dougie-unRAID kernel: ? __die_body+0x1a/0x5c Jan 7 13:58:20 Dougie-unRAID kernel: ? page_fault_oops+0x329/0x376 Jan 7 13:58:20 Dougie-unRAID kernel: ? do_user_addr_fault+0x12e/0x48d Jan 7 13:58:20 Dougie-unRAID kernel: ? exc_page_fault+0xfb/0x11d Jan 7 13:58:20 Dougie-unRAID kernel: ? asm_exc_page_fault+0x22/0x30 Jan 7 13:58:20 Dougie-unRAID kernel: ? pick_next_task_fair+0x90/0x2a7 Jan 7 13:58:20 Dougie-unRAID kernel: ? pick_next_task_fair+0x90/0x2a7 Jan 7 13:58:20 Dougie-unRAID kernel: __schedule+0x1c4/0x612 Jan 7 13:58:20 Dougie-unRAID kernel: schedule+0x8e/0xcc Jan 7 13:58:20 Dougie-unRAID kernel: futex_wait_queue+0x4e/0x82 Jan 7 13:58:20 Dougie-unRAID kernel: futex_wait+0xb7/0x17e Jan 7 13:58:20 Dougie-unRAID kernel: ? xfd_validate_state+0x1d/0x71 Jan 7 13:58:20 Dougie-unRAID kernel: do_futex+0xcd/0x143 Jan 7 13:58:20 Dougie-unRAID kernel: __do_sys_futex+0x127/0x150 Jan 7 13:58:20 Dougie-unRAID kernel: do_syscall_64+0x68/0x81 Jan 7 13:58:20 Dougie-unRAID kernel: entry_SYSCALL_64_after_hwframe+0x64/0xce Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0033:0x150098beb739 Jan 7 13:58:20 Dougie-unRAID kernel: Code: c0 0f 85 24 00 00 00 49 89 fb 48 89 f0 48 89 d7 48 89 ce 4c 89 c2 4d 89 ca 4c 8b 44 24 08 4c 8b 4c 24 10 4c 89 5c 24 08 0f 05 <c3> e9 d5 cb ff ff 41 57 41 56 53 48 81 ec 90 00 00 00 49 89 f6 b8 Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 002b:0000150090de7968 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca Jan 7 13:58:20 Dougie-unRAID kernel: RAX: ffffffffffffffda RBX: 00000000000000ca RCX: 0000150098beb739 Jan 7 13:58:20 Dougie-unRAID kernel: RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000150090de7a44 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: 0000150090de7a44 R08: 0000000000000000 R09: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000150090de7b38 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000002 R14: 0000000000000000 R15: 0000150090de7b74 Jan 7 13:58:20 Dougie-unRAID kernel: </TASK> Jan 7 13:58:20 Dougie-unRAID kernel: Modules linked in: cmac cifs asn1_decoder cifs_arc4 cifs_md4 oid_registry dns_resolver af_packet bluetooth ecdh_generic ecc xt_nat xt_CHECKSUM ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap ipvlan xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype br_netfilter xfs md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs bridge 8021q garp mrp stp llc bonding tls intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp i915 kvm_intel kvm iosf_mbi crct10dif_pclmul drm_buddy crc32_pclmul crc32c_intel i2c_algo_bit ghash_clmulni_intel sha512_ssse3 ttm sha256_ssse3 sha1_ssse3 drm_display_helper aesni_intel drm_kms_helper crypto_simd cryptd tpm_crb mei_hdcp mei_pxp wmi_bmof i2c_i801 rapl Jan 7 13:58:20 Dougie-unRAID kernel: intel_cstate drm nvme ahci e1000e intel_uncore input_leds intel_gtt led_class mei_me i2c_smbus agpgart nvme_core mei i2c_core video libahci intel_pch_thermal syscopyarea sysfillrect sysimgblt fb_sys_fops tpm_tis thermal fan wmi tpm_tis_core backlight tpm acpi_pad intel_pmc_core button unix Jan 7 13:58:20 Dougie-unRAID kernel: CR2: 0000000000000080 Jan 7 13:58:20 Dougie-unRAID kernel: ---[ end trace 0000000000000000 ]--- Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0010:pick_next_task_fair+0x90/0x2a7 Jan 7 13:58:20 Dougie-unRAID kernel: Code: ed 48 89 ef e8 84 dd ff ff 84 c0 74 12 83 bb 90 00 00 00 00 0f 84 4f 01 00 00 e9 8f 00 00 00 4c 89 ee 48 89 ef e8 df b6 ff ff <48> 8b a8 80 00 00 00 49 89 c5 48 85 ed 75 ab 48 89 c7 e8 c4 6e ff Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 0018:ffffc9000f65fcf8 EFLAGS: 00010046 Jan 7 13:58:20 Dougie-unRAID kernel: RAX: 0000000000000000 RBX: ffff888a1442d440 RCX: 000000000000011c Jan 7 13:58:20 Dougie-unRAID kernel: RDX: ffffc9000f65fd40 RSI: 0000000000000000 RDI: ffff88815a9d4a00 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: ffff88815a9d4a00 R08: 000000000000001c R09: 0000000000000aab Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff88822591d140 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000000 R14: ffff888a1442d4c0 R15: ffffc9000f65fd40 Jan 7 13:58:20 Dougie-unRAID kernel: FS: 0000150090de7b38(0000) GS:ffff888a14400000(0000) knlGS:0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 7 13:58:20 Dougie-unRAID kernel: CR2: 0000000000000080 CR3: 00000001e2be8005 CR4: 00000000003706f0 Jan 7 13:58:20 Dougie-unRAID kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Jan 7 13:58:20 Dougie-unRAID kernel: note: Plex Transcoder[9211] exited with irqs disabled Jan 7 13:58:20 Dougie-unRAID kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 0-...D } 21169 jiffies s: 356197 root: 0x1/. Jan 7 13:58:20 Dougie-unRAID kernel: rcu: blocking rcu_node structures (internal RCU debug): Jan 7 13:58:20 Dougie-unRAID kernel: Sending NMI from CPU 2 to CPUs 0: Jan 7 13:58:20 Dougie-unRAID kernel: NMI backtrace for cpu 0 Jan 7 13:58:20 Dougie-unRAID kernel: CPU: 0 PID: 9211 Comm: Plex Transcoder Tainted: P D O 6.1.64-Unraid #1 Jan 7 13:58:20 Dougie-unRAID kernel: Hardware name: Dell Inc. OptiPlex 5060/0654JC, BIOS 1.2.22 11/01/2018 Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: Code: c2 0f b6 d2 c1 e2 08 30 e4 09 d0 3d ff 00 00 00 76 0c 0f ba e0 08 72 1e c6 43 01 00 eb 18 85 c0 74 0a 8b 03 84 c0 74 04 f3 90 <eb> f6 66 c7 03 01 00 e9 32 01 00 00 e8 bd 40 ff ff 49 c7 c4 80 e1 Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 0018:ffffc90000003f50 EFLAGS: 00000002 Jan 7 13:58:20 Dougie-unRAID kernel: RAX: 0000000000080101 RBX: ffff888a1442d440 RCX: 000000000000080b Jan 7 13:58:20 Dougie-unRAID kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff888a1442d440 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: ffff888a1442d440 R08: 0000000000000000 R09: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: ffffc90000003ff8 R12: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: FS: 0000150090de7b38(0000) GS:ffff888a14400000(0000) knlGS:0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 7 13:58:20 Dougie-unRAID kernel: CR2: 0000000000000080 CR3: 00000001e2be8005 CR4: 00000000003706f0 Jan 7 13:58:20 Dougie-unRAID kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Jan 7 13:58:20 Dougie-unRAID kernel: Call Trace: Jan 7 13:58:20 Dougie-unRAID kernel: <NMI> Jan 7 13:58:20 Dougie-unRAID kernel: ? nmi_cpu_backtrace+0xd3/0x104 Jan 7 13:58:20 Dougie-unRAID kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Jan 7 13:58:20 Dougie-unRAID kernel: ? nmi_handle+0x54/0x131 Jan 7 13:58:20 Dougie-unRAID kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: ? default_do_nmi+0x66/0x15b Jan 7 13:58:20 Dougie-unRAID kernel: ? exc_nmi+0xbf/0x130 Jan 7 13:58:20 Dougie-unRAID kernel: ? end_repeat_nmi+0x16/0x67 Jan 7 13:58:20 Dougie-unRAID kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf Jan 7 13:58:20 Dougie-unRAID kernel: </NMI> Jan 7 13:58:20 Dougie-unRAID kernel: <IRQ> Jan 7 13:58:20 Dougie-unRAID kernel: do_raw_spin_lock+0x14/0x1a Jan 7 13:58:20 Dougie-unRAID kernel: raw_spin_rq_lock_nested+0x1b/0x26 Jan 7 13:58:20 Dougie-unRAID kernel: _raw_spin_rq_lock_irqsave+0x15/0x1e Jan 7 13:58:20 Dougie-unRAID kernel: rq_lock_irqsave+0x12/0x27 Jan 7 13:58:20 Dougie-unRAID kernel: sched_ttwu_pending+0x53/0xe0 Jan 7 13:58:20 Dougie-unRAID kernel: ? __flush_smp_call_function_queue+0x43/0x1a5 Jan 7 13:58:20 Dougie-unRAID kernel: __sysvec_call_function_single+0x59/0xa4 Jan 7 13:58:20 Dougie-unRAID kernel: sysvec_call_function_single+0x80/0xa6 Jan 7 13:58:20 Dougie-unRAID kernel: </IRQ> Jan 7 13:58:20 Dougie-unRAID kernel: <TASK> Jan 7 13:58:20 Dougie-unRAID kernel: asm_sysvec_call_function_single+0x16/0x20 Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0010:make_task_dead+0x69/0x11c Jan 7 13:58:20 Dougie-unRAID kernel: Code: 58 0f 1f 40 00 0f ba e0 09 72 1f 8b 93 88 04 00 00 48 8d b3 80 06 00 00 48 c7 c7 82 24 0c 82 e8 a9 d6 7e 00 fb 0f 1f 44 00 00 <65> 8b 0d 58 a4 fa 7e 81 e1 ff ff ff 7f 74 3f 65 48 8b 34 25 c0 cb Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 0018:ffffc9000f65ff40 EFLAGS: 00000246 Jan 7 13:58:20 Dougie-unRAID kernel: RAX: 0000000000000035 RBX: ffff88822591d140 RCX: 0000000000000027 Jan 7 13:58:20 Dougie-unRAID kernel: RDX: 0000000000000000 RSI: ffffffff820d7e01 RDI: 00000000ffffffff Jan 7 13:58:20 Dougie-unRAID kernel: RBP: 0000000000000009 R08: 0000000000000000 R09: 0000000000aaaaaa Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000080 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: rewind_stack_and_make_dead+0x17/0x17 Jan 7 13:58:20 Dougie-unRAID kernel: RIP: 0033:0x150098beb739 Jan 7 13:58:20 Dougie-unRAID kernel: Code: c0 0f 85 24 00 00 00 49 89 fb 48 89 f0 48 89 d7 48 89 ce 4c 89 c2 4d 89 ca 4c 8b 44 24 08 4c 8b 4c 24 10 4c 89 5c 24 08 0f 05 <c3> e9 d5 cb ff ff 41 57 41 56 53 48 81 ec 90 00 00 00 49 89 f6 b8 Jan 7 13:58:20 Dougie-unRAID kernel: RSP: 002b:0000150090de7968 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca Jan 7 13:58:20 Dougie-unRAID kernel: RAX: ffffffffffffffda RBX: 00000000000000ca RCX: 0000150098beb739 Jan 7 13:58:20 Dougie-unRAID kernel: RDX: 0000000000000002 RSI: 0000000000000080 RDI: 0000150090de7a44 Jan 7 13:58:20 Dougie-unRAID kernel: RBP: 0000150090de7a44 R08: 0000000000000000 R09: 0000000000000000 Jan 7 13:58:20 Dougie-unRAID kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000150090de7b38 Jan 7 13:58:20 Dougie-unRAID kernel: R13: 0000000000000002 R14: 0000000000000000 R15: 0000150090de7b74 Jan 7 13:58:20 Dougie-unRAID kernel: </TASK> Full syslog attached. syslog syslog-previous
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.