InsaneNutter

Just to update on this, I believe the issue was the NVME drive. I replaced the drive mid week with a Samsung 990 PRO, moved over the data from the old drive and Unraid seems to have been perfectly happy for the last couple of days.

Unfortunately Unraid has crashed again just after midnight, seems to be with the NVME drive. 192.168.1.2 Jun 16 17:21:31 Server kern info kernel eth0: renamed from vethda87c4c 192.168.1.2 Jun 16 17:24:00 Server user info emhttpd read SMART /dev/sde 192.168.1.2 Jun 16 17:28:44 Server user info emhttpd spinning down /dev/sdd 192.168.1.2 Jun 16 17:46:26 Server kern info kernel eth0: renamed from vethbebac28 192.168.1.2 Jun 16 17:46:26 Server kern info kernel vethbebac28: renamed from eth0 192.168.1.2 Jun 16 17:48:21 Server kern info kernel docker0: port 1(vethd2f1ae1) entered blocking state 192.168.1.2 Jun 16 17:48:21 Server kern info kernel docker0: port 1(vethd2f1ae1) entered disabled state 192.168.1.2 Jun 16 17:48:21 Server kern info kernel device vethd2f1ae1 entered promiscuous mode 192.168.1.2 Jun 16 17:48:21 Server kern info kernel eth0: renamed from vethd699878 192.168.1.2 Jun 16 17:48:21 Server kern info kernel IPv6: ADDRCONF(NETDEV_CHANGE): vethd2f1ae1: link becomes ready 192.168.1.2 Jun 16 17:48:21 Server kern info kernel docker0: port 1(vethd2f1ae1) entered blocking state 192.168.1.2 Jun 16 17:48:21 Server kern info kernel docker0: port 1(vethd2f1ae1) entered forwarding state 192.168.1.2 Jun 16 17:48:54 Server kern info kernel vethd699878: renamed from eth0 192.168.1.2 Jun 16 17:48:54 Server kern info kernel docker0: port 1(vethd2f1ae1) entered disabled state 192.168.1.2 Jun 16 17:48:54 Server kern info kernel docker0: port 1(vethd2f1ae1) entered disabled state 192.168.1.2 Jun 16 17:48:54 Server kern info kernel device vethd2f1ae1 left promiscuous mode 192.168.1.2 Jun 16 17:48:54 Server kern info kernel docker0: port 1(vethd2f1ae1) entered disabled state 192.168.1.2 Jun 16 17:48:54 Server kern info kernel eth0: renamed from veth6f64f3c 192.168.1.2 Jun 16 17:55:43 Server user info emhttpd spinning down /dev/sde 192.168.1.2 Jun 16 18:02:32 Server kern info kernel vethd21dfcb: renamed from eth0 192.168.1.2 Jun 16 18:03:12 Server kern info kernel br-548a3d5ec277: port 2(vethe265799) entered blocking state 192.168.1.2 Jun 16 18:03:12 Server kern info kernel br-548a3d5ec277: port 2(vethe265799) entered disabled state 192.168.1.2 Jun 16 18:03:12 Server kern info kernel device vethe265799 entered promiscuous mode 192.168.1.2 Jun 16 18:03:12 Server kern info kernel br-548a3d5ec277: port 2(vethe265799) entered disabled state 192.168.1.2 Jun 16 18:03:12 Server kern info kernel device vethe265799 left promiscuous mode 192.168.1.2 Jun 16 18:03:12 Server kern info kernel br-548a3d5ec277: port 2(vethe265799) entered disabled state 192.168.1.2 Jun 16 18:03:31 Server kern info kernel eth0: renamed from veth73b23f5 192.168.1.2 Jun 16 18:04:17 Server kern info kernel veth5bc5fc6: renamed from eth0 192.168.1.2 Jun 16 18:04:17 Server kern info kernel br-548a3d5ec277: port 1(veth15f5eb2) entered disabled state 192.168.1.2 Jun 16 18:04:17 Server kern info kernel br-548a3d5ec277: port 1(veth15f5eb2) entered disabled state 192.168.1.2 Jun 16 18:04:17 Server kern info kernel device veth15f5eb2 left promiscuous mode 192.168.1.2 Jun 16 18:04:17 Server kern info kernel br-548a3d5ec277: port 1(veth15f5eb2) entered disabled state 192.168.1.2 Jun 16 18:04:17 Server kern info kernel eth0: renamed from veth6504ee7 192.168.1.2 Jun 16 18:30:06 Server user info emhttpd read SMART /dev/sde 192.168.1.2 Jun 16 18:30:40 Server user info emhttpd read SMART /dev/sdb 192.168.1.2 Jun 16 18:38:41 Server user info emhttpd read SMART /dev/sdd 192.168.1.2 Jun 16 19:02:43 Server user info emhttpd spinning down /dev/sdb 192.168.1.2 Jun 16 20:52:34 Server user info emhttpd spinning down /dev/sdd 192.168.1.2 Jun 16 20:53:22 Server user info emhttpd spinning down /dev/sde 192.168.1.2 Jun 16 22:10:13 Server user info emhttpd read SMART /dev/sde 192.168.1.2 Jun 16 22:56:54 Server user info emhttpd read SMART /dev/sdd 192.168.1.2 Jun 16 23:02:52 Server user info emhttpd spinning down /dev/sde 192.168.1.2 Jun 16 23:12:51 Server user info emhttpd read SMART /dev/sde 192.168.1.2 Jun 16 23:13:04 Server user info emhttpd spinning down /dev/sdc 192.168.1.2 Jun 17 00:00:49 Server kern warning kernel nvme nvme0: I/O 584 (I/O Cmd) QID 3 timeout, aborting 192.168.1.2 Jun 17 00:00:49 Server kern warning kernel nvme nvme0: I/O 665 (I/O Cmd) QID 6 timeout, aborting 192.168.1.2 Jun 17 00:00:50 Server kern warning kernel nvme nvme0: I/O 596 (I/O Cmd) QID 5 timeout, aborting 192.168.1.2 Jun 17 00:01:19 Server kern warning kernel nvme nvme0: I/O 584 QID 3 timeout, reset controller 192.168.1.2 Jun 17 00:01:38 Server kern warning kernel nvme nvme0: I/O 24 QID 0 timeout, reset controller 192.168.1.2 Jun 17 00:04:31 Server kern err kernel nvme nvme0: Device not ready; aborting reset, CSTS=0x1 192.168.1.2 Jun 17 00:04:37 Server kern warning kernel nvme nvme0: Abort status: 0x371 192.168.1.2 Jun 17 00:04:37 Server kern warning kernel nvme nvme0: Abort status: 0x371 192.168.1.2 Jun 17 00:04:37 Server kern warning kernel nvme nvme0: Abort status: 0x371 192.168.1.2 Jun 17 00:04:40 Server kern err kernel nvme 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible 192.168.1.2 Jun 17 00:05:51 Server kern err kernel rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: 192.168.1.2 Jun 17 00:05:51 Server kern err kernel rcu: 8-....: (52 ticks this GP) idle=905c/1/0x4000000000000000 softirq=6264128/6264128 fqs=25390 192.168.1.2 Jun 17 00:05:51 Server kern err kernel rcu: 9-....: (55 ticks this GP) idle=f4dc/1/0x4000000000000000 softirq=2796004/2796004 fqs=25390 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel (detected by 5, t=60002 jiffies, g=9828341, q=376561 ncpus=12) 192.168.1.2 Jun 17 00:05:51 Server kern info kernel Sending NMI from CPU 5 to CPUs 8: 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel NMI backtrace for cpu 8 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel CPU: 8 PID: 64 Comm: ksoftirqd/8 Tainted: P O 6.1.79-Unraid #1 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Hardware name: Default string Default string/Default string, BIOS 5.27 05/06/2024 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RIP: 0010:__ip_append_data+0x319/0xa24 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Code: 24 5e 88 44 24 5f 8b 84 24 c0 00 00 00 01 45 14 48 85 db 74 12 45 31 d2 44 89 54 24 14 44 89 54 24 3c e9 57 04 00 00 45 31 c9 <44> 89 4c 24 3c 44 89 4c 24 14 44 89 4c 24 18 0f b7 44 24 68 0f b7 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RSP: 0000:ffffc900003834f8 EFLAGS: 00000246 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RAX: 0000000000000014 RBX: 0000000000000000 RCX: 0000000000000014 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RDX: 00000000000005c8 RSI: ffffc90000383688 RDI: 0000000000000014 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RBP: ffff888101aaa340 R08: ffff888101aaa1d8 R09: 0000000000000000 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel R10: 0000000000000000 R11: 0000000000000000 R12: ffff888186f03000 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel R13: 0000000000000000 R14: 0000000000000000 R15: ffff888101aaa000 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel FS: 0000000000000000(0000) GS:ffff88885fa00000(0000) knlGS:0000000000000000 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel CR2: 0000153948601003 CR3: 00000002bcd76000 CR4: 0000000000752ee0 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel PKRU: 55555554 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Call Trace: 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel <NMI> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nmi_cpu_backtrace+0xd3/0x104 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nmi_cpu_backtrace_handler+0xd/0x15 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nmi_handle+0x54/0x131 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? __ip_append_data+0x319/0xa24 192.168.1.2 Jun 17 00:05:51 Server kern info kernel Sending NMI from CPU 5 to CPUs 9: 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? default_do_nmi+0x66/0x15b 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? exc_nmi+0xbf/0x130 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? end_repeat_nmi+0x16/0x67 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? __ip_append_data+0x319/0xa24 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? __ip_append_data+0x319/0xa24 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? __ip_append_data+0x319/0xa24 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel </NMI> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel <TASK> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? pskb_trim_unique+0x25/0x25 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? pskb_trim_unique+0x25/0x25 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ip_append_data.part.0+0xa7/0xbe 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ip_send_unicast_reply+0x2b1/0x3b7 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? fib_table_lookup+0x358/0x3ba 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? tcp_v4_send_reset+0x538/0x5e9 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel tcp_v4_send_reset+0x538/0x5e9 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? tcp_v4_rcv+0x74f/0x92c 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel tcp_v4_rcv+0x74f/0x92c 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ip_protocol_deliver_rcu+0x9c/0x164 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ip_local_deliver_finish+0x7e/0x8c 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel __netif_receive_skb_one_core+0x77/0x9c 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel netif_receive_skb+0xbf/0x127 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel br_handle_frame_finish+0x482/0x4bc [bridge] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nf_conntrack_in+0x30d/0x4bb [nf_conntrack] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? br_pass_frame_up+0xdd/0xdd [bridge] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel br_nf_hook_thresh+0xe2/0x109 [br_netfilter] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? br_pass_frame_up+0xdd/0xdd [bridge] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel br_nf_pre_routing_finish+0x2cd/0x2f8 [br_netfilter] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? br_pass_frame_up+0xdd/0xdd [bridge] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel br_nf_pre_routing+0x236/0x24a [br_netfilter] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? br_nf_hook_thresh+0x109/0x109 [br_netfilter] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel br_handle_frame+0x277/0x2e0 [bridge] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? br_pass_frame_up+0xdd/0xdd [bridge] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel __netif_receive_skb_core.constprop.0+0x4fa/0x6e9 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? asm_sysvec_apic_timer_interrupt+0x16/0x20 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel __netif_receive_skb_list_core+0x8a/0x11e 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel netif_receive_skb_list_internal+0x1d2/0x20b 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel gro_normal_one+0x46/0x63 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel napi_gro_receive+0x97/0x10a 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel igc_poll+0xc8c/0xf4e [igc] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel __napi_poll.constprop.0+0x28/0x124 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel net_rx_action+0x159/0x24f 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? raw_spin_rq_unlock_irq+0x5/0x10 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel __do_softirq+0x126/0x288 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel run_ksoftirqd+0x29/0x38 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel smpboot_thread_fn+0x1b6/0x1d3 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? find_next_bit+0x5/0x5 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel kthread+0xe4/0xef 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? kthread_complete_and_exit+0x1b/0x1b 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ret_from_fork+0x1f/0x30 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel </TASK> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel NMI backtrace for cpu 9 192.168.1.2 Jun 17 00:05:51 Server kern info kernel INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 9567.688 msecs 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel CPU: 9 PID: 541 Comm: kworker/u24:2 Tainted: P O 6.1.79-Unraid #1 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Hardware name: Default string Default string/Default string, BIOS 5.27 05/06/2024 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Workqueue: nvme-reset-wq nvme_reset_work [nvme] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RIP: 0010:pci_mmcfg_read+0xa4/0xbc 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Code: 83 fe 02 74 15 41 83 fe 04 74 1a 41 ff ce 75 1d 48 01 d8 8a 00 0f b6 c0 eb 10 48 01 d8 66 8b 00 0f b7 c0 eb 05 48 01 d8 8b 00 <89> 45 00 e8 26 65 89 ff 31 c0 5a 5b 5d 41 5c 41 5d 41 5e c3 cc cc 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RSP: 0018:ffffc90044cd3d18 EFLAGS: 00000286 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RAX: 00000000ffffffff RBX: 0000000000000ffc RCX: 0000000000000ffc 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RDX: 00000000000000ff RSI: 0000000000000001 RDI: 0000000000000000 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RBP: ffffc90044cd3d54 R08: 0000000000000004 R09: ffffc90044cd3d54 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel R10: 0000000000000001 R11: ffff8881004002f0 R12: 0000000000100000 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel R13: 0000000000000000 R14: 0000000000000004 R15: ffff88810192e0d0 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel FS: 0000000000000000(0000) GS:ffff88885fa40000(0000) knlGS:0000000000000000 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel CR2: 000014fd2fc01aa8 CR3: 0000000282f10000 CR4: 0000000000752ee0 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel PKRU: 55555554 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Call Trace: 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel <NMI> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nmi_cpu_backtrace+0xd3/0x104 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nmi_cpu_backtrace_handler+0xd/0x15 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nmi_handle+0x54/0x131 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? pci_mmcfg_read+0xa4/0xbc 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? default_do_nmi+0x66/0x15b 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? exc_nmi+0xbf/0x130 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? end_repeat_nmi+0x16/0x67 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? pci_mmcfg_read+0xa4/0xbc 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? pci_mmcfg_read+0xa4/0xbc 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? pci_mmcfg_read+0xa4/0xbc 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel </NMI> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel <TASK> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel pci_bus_read_config_dword+0x46/0x7a 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel pci_find_next_ext_capability+0x9d/0xcb 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel pci_save_state+0x15a/0x1ad 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel nvme_reset_work+0x376/0x1044 [nvme] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? _raw_spin_unlock+0x14/0x29 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? raw_spin_rq_unlock_irq+0x5/0x10 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? finish_task_switch.isra.0+0x140/0x218 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel process_one_work+0x1a8/0x295 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel worker_thread+0x18b/0x244 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? rescuer_thread+0x281/0x281 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel kthread+0xe4/0xef 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? kthread_complete_and_exit+0x1b/0x1b 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ret_from_fork+0x1f/0x30 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel </TASK> 192.168.1.2 Jun 17 00:05:51 Server kern err kernel rcu: rcu_preempt kthread starved for 10528 jiffies! g9828341 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=0 192.168.1.2 Jun 17 00:05:51 Server kern err kernel rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. 192.168.1.2 Jun 17 00:05:51 Server kern err kernel rcu: RCU grace-period kthread stack dump: 192.168.1.2 Jun 17 00:05:51 Server kern info kernel task:rcu_preempt state:R running task stack:0 pid:15 ppid:2 flags:0x00004808 192.168.1.2 Jun 17 00:05:51 Server kern info kernel Call Trace: 192.168.1.2 Jun 17 00:05:51 Server kern info kernel <TASK> 192.168.1.2 Jun 17 00:05:51 Server kern info kernel ? rcu_gp_kthread+0x16d/0x16d 192.168.1.2 Jun 17 00:05:51 Server kern info kernel ? _raw_spin_unlock_irqrestore+0x1a/0x3a 192.168.1.2 Jun 17 00:05:51 Server kern info kernel ? force_qs_rnp+0x1b1/0x1d0 192.168.1.2 Jun 17 00:05:51 Server kern info kernel ? rcu_gp_init+0x494/0x494 192.168.1.2 Jun 17 00:05:51 Server kern info kernel ? rcu_gp_fqs_loop+0x231/0x475 192.168.1.2 Jun 17 00:05:51 Server kern info kernel ? rcu_gp_kthread+0x151/0x16d 192.168.1.2 Jun 17 00:05:51 Server kern info kernel ? kthread+0xe4/0xef 192.168.1.2 Jun 17 00:05:51 Server kern info kernel ? kthread_complete_and_exit+0x1b/0x1b 192.168.1.2 Jun 17 00:05:51 Server kern info kernel ? ret_from_fork+0x1f/0x30 192.168.1.2 Jun 17 00:05:51 Server kern info kernel </TASK> 192.168.1.2 Jun 17 00:05:51 Server kern err kernel rcu: Stack dump where RCU GP kthread last ran: 192.168.1.2 Jun 17 00:05:51 Server kern info kernel Sending NMI from CPU 5 to CPUs 0: 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel NMI backtrace for cpu 0 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel CPU: 0 PID: 15 Comm: rcu_preempt Tainted: P O 6.1.79-Unraid #1 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Hardware name: Default string Default string/Default string, BIOS 5.27 05/06/2024 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RIP: 0010:preempt_count_sub+0x8e/0x93 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Code: ab a5 31 01 00 75 23 48 c7 c6 53 50 0c 82 48 c7 c7 88 30 0b 82 e8 c6 78 fd ff 0f 0b c3 cc cc cc cc f7 df 65 01 3d 5e 6b f8 7e <c3> cc cc cc cc 0f 1f 44 00 00 85 ff 74 05 e9 1a 40 ff ff 65 48 8b 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RSP: 0018:ffffc90000003f40 EFLAGS: 00000013 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RAX: 0000000000010002 RBX: 0000000000000002 RCX: ffff888106b36e70 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RDX: 0000000000010002 RSI: 0000000000000002 RDI: 00000000ffffffff 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RBP: ffff888106b36e70 R08: ffff88885f8200e0 R09: ffff88885f8200e0 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel R10: ffff888192627090 R11: ffffc90000003ff8 R12: ffff88885f820080 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel R13: 0000301ec8482ee3 R14: 0000000000000002 R15: ffffffffa067ae75 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel FS: 0000000000000000(0000) GS:ffff88885f800000(0000) knlGS:0000000000000000 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel CR2: 00002a3c00175000 CR3: 00000002bcd76000 CR4: 0000000000752ef0 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel PKRU: 55555554 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Call Trace: 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel <NMI> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nmi_cpu_backtrace+0xd3/0x104 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nmi_cpu_backtrace_handler+0xd/0x15 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? nmi_handle+0x54/0x131 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? preempt_count_sub+0x8e/0x93 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? default_do_nmi+0x66/0x15b 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? exc_nmi+0xbf/0x130 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? end_repeat_nmi+0x16/0x67 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? pit_latch_count+0x2b/0x2b [kvm] 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? preempt_count_sub+0x8e/0x93 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? preempt_count_sub+0x8e/0x93 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? preempt_count_sub+0x8e/0x93 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel </NMI> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel <IRQ> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel _raw_spin_unlock_irqrestore+0x24/0x3a 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel __hrtimer_run_queues+0x9c/0x190 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel hrtimer_interrupt+0x9c/0x16e 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel __sysvec_apic_timer_interrupt+0xc2/0x12f 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel sysvec_apic_timer_interrupt+0x80/0xa6 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel </IRQ> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel <TASK> 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel asm_sysvec_apic_timer_interrupt+0x16/0x20 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RIP: 0010:_raw_spin_unlock_irqrestore+0x1a/0x3a 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel Code: 79 7e 85 c0 75 05 0f 1f 44 00 00 c3 cc cc cc cc 0f 1f 44 00 00 53 48 89 f3 e8 32 83 83 ff 0f ba e3 09 73 06 fb 0f 1f 44 00 00 <bf> 01 00 00 00 e8 5f 43 81 ff 65 8b 05 44 af 79 7e 85 c0 75 05 0f 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RSP: 0018:ffffc900000e7e50 EFLAGS: 00000247 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RAX: 0000000000000040 RBX: 0000000000000286 RCX: 000000000000000a 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RDX: 0000000000000040 RSI: 0000000000000286 RDI: ffffffff822bae80 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000000 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel R10: 0000000000000020 R11: 000000000000009c R12: ffffffff810d9de9 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel R13: ffff8881002de000 R14: 0000000000000000 R15: 000000000002ee00 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? rcu_gp_kthread+0x16d/0x16d 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel force_qs_rnp+0x1b1/0x1d0 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? rcu_gp_init+0x494/0x494 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel rcu_gp_fqs_loop+0x231/0x475 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel rcu_gp_kthread+0x151/0x16d 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel kthread+0xe4/0xef 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ? kthread_complete_and_exit+0x1b/0x1b 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel ret_from_fork+0x1f/0x30 192.168.1.2 Jun 17 00:05:51 Server kern warning kernel </TASK> I wonder if the drive is starting to fail? although its only about an 8 month old Patriot P300.

The full log from before the crash is quite short as it happened about an hour after i'd setup remote logging, however I have pasted it below: 192.168.1.2 Jun 16 07:54:33 Server syslog info rsyslogd [origin software="rsyslogd" swVersion="8.2102.0" x-pid="7178" x-info="https://www.rsyslog.com"] start 192.168.1.2 Jun 16 08:11:43 Server user info emhttpd spinning down /dev/sdd 192.168.1.2 Jun 16 08:11:43 Server user info emhttpd spinning down /dev/sde 192.168.1.2 Jun 16 08:11:43 Server user info emhttpd spinning down /dev/sdb 192.168.1.2 Jun 16 08:11:49 Server user info emhttpd spinning down /dev/sdc 192.168.1.2 Jun 16 08:17:48 Server kern info kernel vethf705692: renamed from eth0 192.168.1.2 Jun 16 08:17:48 Server kern info kernel docker0: port 3(veth506f2af) entered disabled state 192.168.1.2 Jun 16 08:17:48 Server kern info kernel docker0: port 3(veth506f2af) entered disabled state 192.168.1.2 Jun 16 08:17:48 Server kern info kernel device veth506f2af left promiscuous mode 192.168.1.2 Jun 16 08:17:48 Server kern info kernel docker0: port 3(veth506f2af) entered disabled state 192.168.1.2 Jun 16 08:18:07 Server kern info kernel docker0: port 3(vethd4d190f) entered blocking state 192.168.1.2 Jun 16 08:18:07 Server kern info kernel docker0: port 3(vethd4d190f) entered disabled state 192.168.1.2 Jun 16 08:18:07 Server kern info kernel device vethd4d190f entered promiscuous mode 192.168.1.2 Jun 16 08:18:07 Server kern info kernel eth0: renamed from vethd9163fc 192.168.1.2 Jun 16 08:18:07 Server kern info kernel IPv6: ADDRCONF(NETDEV_CHANGE): vethd4d190f: link becomes ready 192.168.1.2 Jun 16 08:18:07 Server kern info kernel docker0: port 3(vethd4d190f) entered blocking state 192.168.1.2 Jun 16 08:18:07 Server kern info kernel docker0: port 3(vethd4d190f) entered forwarding state 192.168.1.2 Jun 16 08:23:57 Server kern info kernel br0: port 3(vnet1) entered blocking state 192.168.1.2 Jun 16 08:23:57 Server kern info kernel br0: port 3(vnet1) entered disabled state 192.168.1.2 Jun 16 08:23:57 Server kern info kernel device vnet1 entered promiscuous mode 192.168.1.2 Jun 16 08:23:57 Server kern info kernel br0: port 3(vnet1) entered blocking state 192.168.1.2 Jun 16 08:23:57 Server kern info kernel br0: port 3(vnet1) entered forwarding state 192.168.1.2 Jun 16 08:28:20 Server kern warning kernel x86/split lock detection: #AC: CPU 0/KVM/24166 took a split_lock trap at address: 0xfffff80545e4b9dd 192.168.1.2 Jun 16 08:32:11 Server user info emhttpd read SMART /dev/sde 192.168.1.2 Jun 16 08:35:28 Server kern warning kernel x86/split lock detection: #AC: CPU 2/KVM/24168 took a split_lock trap at address: 0x27403401dde 192.168.1.2 Jun 16 08:47:12 Server user info emhttpd read SMART /dev/sdd 192.168.1.2 Jun 16 08:47:12 Server user info emhttpd read SMART /dev/sdc 192.168.1.2 Jun 16 08:51:06 Server user info emhttpd read SMART /dev/sdb 192.168.1.2 Jun 16 09:06:55 Server kern warning kernel nvme nvme0: I/O 698 (I/O Cmd) QID 1 timeout, aborting 192.168.1.2 Jun 16 09:06:55 Server kern warning kernel nvme nvme0: I/O 450 (I/O Cmd) QID 2 timeout, aborting 192.168.1.2 Jun 16 09:06:56 Server kern warning kernel nvme nvme0: I/O 697 (I/O Cmd) QID 1 timeout, aborting 192.168.1.2 Jun 16 09:07:25 Server kern warning kernel nvme nvme0: I/O 698 QID 1 timeout, reset controller 192.168.1.2 Jun 16 09:07:31 Server kern warning kernel nvme nvme0: I/O 8 QID 0 timeout, reset controller 192.168.1.2 Jun 16 09:10:37 Server kern err kernel nvme nvme0: Device not ready; aborting reset, CSTS=0x1 I found a post by yourself as it happens on a simlar issue: I've appended: initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off as advised there to see if that helps. I suspect not ideal from a power use point of view as my drive will never enter a low power mode, curious as to if it helps however.

Currently I am on Unraid 6.12.10, over the last few months Unraid has been become increasingly unstable. The server started crashing ever few days, now its got to the point where its every 10-12 hours, sometimes less. Crash = no output to a connected monitor, server totally inaccessible via the web ui, network shares, Docker containers down. Yesterday I switched to a different motherboard, ram and processor, using my existing configuration and exactly the same has just happened after around 10 hours, so I feel it must be related to my configuration in some way and not the hardware its self. I've setup a syslog server on my PC, so I guess its a waiting game to see if anything is logged when it crashes again. Any advice would be appreciated. Edit - from looking at what was logged so my syslog server before it crashed it would appear to be an nvme related issue: 192.168.1.2 Jun 16 08:47:12 Server user info emhttpd read SMART /dev/sdd 192.168.1.2 Jun 16 08:47:12 Server user info emhttpd read SMART /dev/sdc 192.168.1.2 Jun 16 08:51:06 Server user info emhttpd read SMART /dev/sdb 192.168.1.2 Jun 16 09:06:55 Server kern warning kernel nvme nvme0: I/O 698 (I/O Cmd) QID 1 timeout, aborting 192.168.1.2 Jun 16 09:06:55 Server kern warning kernel nvme nvme0: I/O 450 (I/O Cmd) QID 2 timeout, aborting 192.168.1.2 Jun 16 09:06:56 Server kern warning kernel nvme nvme0: I/O 697 (I/O Cmd) QID 1 timeout, aborting 192.168.1.2 Jun 16 09:07:25 Server kern warning kernel nvme nvme0: I/O 698 QID 1 timeout, reset controller 192.168.1.2 Jun 16 09:07:31 Server kern warning kernel nvme nvme0: I/O 8 QID 0 timeout, reset controller 192.168.1.2 Jun 16 09:10:37 Server kern err kernel nvme nvme0: Device not ready; aborting reset, CSTS=0x1

Perfect, setting the UMASK to 000 and running New Permissions on that share resolved the issue as desired. Thanks a lot for the advice!

Hi all. I've recently built a new home server and decided to switch over to Unraid. After setting up various shares I've created a user with read / write permission for my SMB shares. Using the user with read / write permission I've copied all the data across from my old server. I then set up other users who just have read access to this data. All is good and the permissions work as expected for the shares. I've now set a few Dockers up, these could read the data as expected. My problem now is I've allowed Sonarr to rename some of my library, the SMB user with read / write access can now only read any of the folder / files Sonarr has touched. Windows states "You require permission from UNRAID\digiex to make changes to this file" yet I am accessing that share from windows as UNRAID\digiex. TV series Sonarr has renamed and made the SMB user read only: root@Unraid:/mnt/user/TV/NotWorkingSeries/Season 1# ls -al total 83224578 drwxr-xr-x 1 nobody users 26 Nov 22 10:06 ./ drwxrwxrwx 1 digiex users 13 Nov 22 10:06 ../ -rwxrw-rw- 1 digiex users 4261849792 Sep 30 23:56 NotWorkingSeries\ -\ S01E01\ -\ Pilot\ -\ Bluray-1080p.mkv* TV series only the SMB user has accessed and can still read / write to: root@Unraid:/mnt/user/TV/WorkingSeries/Season 01# ls -al total 39544848 drwxrwxrwx 1 digiex users 12 Nov 9 17:55 ./ drwxrwxrwx 1 digiex users 3 Nov 9 17:45 ../ -rwxrw-rw- 1 digiex users 4879437296 May 29 12:54 WorkingSeries\ -\ S01E01\ -\ Pilot\ -\ Bluray-1080p.mkv* The Sonarr Docker: PUID: 99 PGID: 100 UMASK: 022 Essentially what I'm asking is how can I allow the SMB permissions to work as set in the Unraid UI, yet still allow Dockers to manipulate the data without breaking the SMB permissions? Thanks for any advice.

Thanks for the info, I initially had problems with this as /etc/modprobe.d/i915.conf will not persist when you reboot. So you want to create an i915.conf in /boot/config/modprobe.d/ echo "options i915 enable_fbc=1 enable_guc=2" > /boot/config/modprobe.d/i915.conf or nano /boot/config/modprobe.d/i915.conf then type the following options i915 enable_fbc=1 enable_guc=2 That will be copied to /etc/modprobe.d/i915.conf when Unraid boots.