Jupiler

I obviously don't recommend doing this, but if you really need access before a fix is available, here is how I fixed it for now I fixed it by changing the following on line 311~312 in the /usr/local/emhttp/plugins/community.applications/include/exec.php file. from this: if ( $o['RequiresFile'] ?? null) $o['RequiresFile'] = trim($o['RequiresFile']); if ( $o['Requires'] ?? null) $o['Requires'] = trim($o['Requires']); to this: if ( $o['RequiresFile'] ?? null) { if ( is_array($o['RequiresFile']) ) { $tmp = array_map('strval',$o['RequiresFile']); $tmp = array_filter($tmp,function($s){ return trim($s) !== ''; }); $o['RequiresFile'] = trim($tmp[0] ?? ""); } else { $o['RequiresFile'] = trim((string)$o['RequiresFile']); } } if ( $o['Requires'] ?? null) { if ( is_array($o['Requires']) ) { $req = array_map('strval',$o['Requires']); $req = array_filter($req,function($s){ return trim($s) !== ''; }); $o['Requires'] = trim(implode("\n",$req)); } else { $o['Requires'] = trim((string)$o['Requires']); } } Looks like it's caused by one of the app info that is being loaded. Again this is a TEMPORARY fix, I don't recommend doing this

Sounds good, just added it to the original post, as well as the persistent syslog when one of the crash happened

I've been battling with an odd issue for the past few days, every so often Unraid would become totally unresponsive, usually the exact moment I'm doing something on the GUI, either loading the docker page of Unraid, or accessing a docker container webui (none in particular). Sometimes I would still be able to ping the Unraid box on the network, but not able to SSH into it (SSH would just hang). This time around the Unraid box became unresponsive but I'm also not able to ping it. I plugged a monitor and keyboard on the box itself and I'm still able to login directly. I've tried to troubleshoot from the console while I'm able to access it, but I'm not too sure what I should be looking for. Thinking of it being a network issue possibly I started checking some of the network interfaces, however when I typed `ifconfig bond0` it became unresponsive as well. Then only way out was to do a CTRL ALT DEL, which triggered a shutdown, however the shutdown process has been stuck at "Starting diagnostic collection" for the past couple of hours. (see image) Server's uptime before the issues started was 7 months without problems, no hardware changes were made recently. Any idea on what I should do next to troubleshoot this issue? I've also been running a syslog server to get a more permanent record of the syslog, see below (server became unresponsive at 17:00:00 or so): Aug 14 14:30:10 BorgCube emhttpd: spinning down /dev/sdp Aug 14 14:46:09 BorgCube emhttpd: spinning down /dev/sdf Aug 14 14:47:16 BorgCube emhttpd: spinning down /dev/sdo Aug 14 14:47:16 BorgCube emhttpd: spinning down /dev/sdi Aug 14 14:54:02 BorgCube emhttpd: read SMART /dev/sdi Aug 14 14:54:13 BorgCube emhttpd: read SMART /dev/sdp Aug 14 14:54:44 BorgCube emhttpd: read SMART /dev/sdo Aug 14 15:09:55 BorgCube emhttpd: read SMART /dev/sda Aug 14 15:26:55 BorgCube emhttpd: spinning down /dev/sdo Aug 14 15:33:16 BorgCube emhttpd: read SMART /dev/sdo Aug 14 15:45:15 BorgCube emhttpd: spinning down /dev/sda Aug 14 15:47:38 BorgCube emhttpd: spinning down /dev/sdp Aug 14 15:52:59 BorgCube emhttpd: read SMART /dev/sdf Aug 14 16:00:06 BorgCube emhttpd: read SMART /dev/sdp Aug 14 16:00:44 BorgCube Parity Check Tuning: Send notification: mover running: (type=normal link=/Settings/Scheduler) Aug 14 16:18:19 BorgCube Parity Check Tuning: Send notification: mover no longer running: (type=normal link=/Settings/Scheduler) Aug 14 16:43:36 BorgCube emhttpd: spinning down /dev/sdp Aug 14 16:45:58 BorgCube emhttpd: spinning down /dev/sdf Aug 14 16:47:10 BorgCube emhttpd: spinning down /dev/sdo Aug 14 16:47:10 BorgCube emhttpd: spinning down /dev/sdi Aug 14 16:59:07 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:14 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:19 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:23 BorgCube kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 3-.... } 21145 jiffies s: 74473 root: 0x8/. Aug 14 16:59:23 BorgCube kernel: rcu: blocking rcu_node structures (internal RCU debug): Aug 14 16:59:23 BorgCube kernel: Sending NMI from CPU 7 to CPUs 3: Aug 14 16:59:23 BorgCube kernel: NMI backtrace for cpu 3 Aug 14 16:59:23 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 16:59:23 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 16:59:23 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 16:59:23 BorgCube kernel: RIP: 0010:preempt_count_sub+0x85/0x93 Aug 14 16:59:23 BorgCube kernel: Code: d5 39 00 85 c0 74 2c 83 3d f3 73 31 01 00 75 23 48 c7 c6 f2 5b 0c 82 48 c7 c7 f8 39 0b 82 e8 2e 76 fd ff 0f 0b c3 cc cc cc cc <f7> df 65 01 3d e6 38 f8 7e c3 cc cc cc cc 0f 1f 44 00 00 85 ff 74 Aug 14 16:59:23 BorgCube kernel: RSP: 0018:ffffc900001d8ea8 EFLAGS: 00000202 Aug 14 16:59:23 BorgCube kernel: RAX: 0000000000000102 RBX: 0000000000000246 RCX: ffff888103c98c40 Aug 14 16:59:23 BorgCube kernel: RDX: 0000000000000102 RSI: 0000000000000246 RDI: 0000000000000001 Aug 14 16:59:23 BorgCube kernel: RBP: ffffc900001d8f70 R08: 0000000000000000 R09: 0000000000000082 Aug 14 16:59:23 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103c98da0 Aug 14 16:59:23 BorgCube kernel: R13: ffff888101143c00 R14: ffff888103c98000 R15: 0000000000001296 Aug 14 16:59:23 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 16:59:23 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 16:59:23 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 16:59:23 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 16:59:23 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 16:59:23 BorgCube kernel: Call Trace: Aug 14 16:59:23 BorgCube kernel: <NMI> Aug 14 16:59:23 BorgCube kernel: ? nmi_cpu_backtrace+0xd3/0x104 Aug 14 16:59:23 BorgCube kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Aug 14 16:59:23 BorgCube kernel: ? nmi_handle+0x54/0x131 Aug 14 16:59:23 BorgCube kernel: ? preempt_count_sub+0x85/0x93 Aug 14 16:59:23 BorgCube kernel: ? default_do_nmi+0x66/0x15b Aug 14 16:59:23 BorgCube kernel: ? exc_nmi+0xbf/0x11c Aug 14 16:59:23 BorgCube kernel: ? end_repeat_nmi+0x16/0x67 Aug 14 16:59:23 BorgCube kernel: ? preempt_count_sub+0x85/0x93 Aug 14 16:59:23 BorgCube kernel: ? preempt_count_sub+0x85/0x93 Aug 14 16:59:23 BorgCube kernel: ? preempt_count_sub+0x85/0x93 Aug 14 16:59:23 BorgCube kernel: </NMI> Aug 14 16:59:23 BorgCube kernel: <IRQ> Aug 14 16:59:23 BorgCube kernel: _raw_spin_unlock_irqrestore+0x24/0x3a Aug 14 16:59:23 BorgCube kernel: bottom_half+0x473/0x8ef [r8152] Aug 14 16:59:23 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 16:59:23 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 16:59:23 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 16:59:23 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 16:59:23 BorgCube kernel: </IRQ> Aug 14 16:59:23 BorgCube kernel: <TASK> Aug 14 16:59:23 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 16:59:23 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 16:59:23 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 16:59:23 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 16:59:23 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 16:59:23 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 16:59:23 BorgCube kernel: kthread+0xe4/0xef Aug 14 16:59:23 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 16:59:23 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 16:59:23 BorgCube kernel: </TASK> Aug 14 16:59:24 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:29 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:36 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:41 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:46 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:51 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:58 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 16:59:58 BorgCube kernel: rcu: INFO: rcu_preempt self-detected stall on CPU Aug 14 16:59:58 BorgCube kernel: rcu: #0113-....: (1 GPs behind) idle=ae14/1/0x4000000000000000 softirq=20606672/20606673 fqs=26324 Aug 14 16:59:58 BorgCube kernel: #011(t=60000 jiffies g=37415333 q=1118808 ncpus=8) Aug 14 16:59:58 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 16:59:58 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 16:59:58 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 16:59:58 BorgCube kernel: RIP: 0010:sg_init_table+0xb/0x2a Aug 14 16:59:58 BorgCube kernel: Code: 04 25 28 00 00 00 74 05 e8 0b 38 44 00 48 83 c4 50 48 89 d8 5b 5d 41 5c 41 5d c3 cc cc cc cc 89 f1 31 c0 48 89 fa 48 c1 e1 05 <f3> aa 8d 46 ff 48 c1 e0 05 48 01 c2 48 8b 02 48 83 e0 fe 48 83 c8 Aug 14 16:59:58 BorgCube kernel: RSP: 0018:ffffc900001d8eb8 EFLAGS: 00010202 Aug 14 16:59:58 BorgCube kernel: RAX: 0000000000000000 RBX: ffff888103c98ab0 RCX: 0000000000000160 Aug 14 16:59:58 BorgCube kernel: RDX: ffff888104640000 RSI: 0000000000000011 RDI: ffff8881046400c0 Aug 14 16:59:58 BorgCube kernel: RBP: ffffc900001d8f70 R08: 0000000000000000 R09: 0000000000000082 Aug 14 16:59:58 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103c98da0 Aug 14 16:59:58 BorgCube kernel: R13: ffff888101143c00 R14: ffff888103c98000 R15: 0000000000001296 Aug 14 16:59:58 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 16:59:58 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 16:59:58 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 16:59:58 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 16:59:58 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 16:59:58 BorgCube kernel: Call Trace: Aug 14 16:59:58 BorgCube kernel: <IRQ> Aug 14 16:59:58 BorgCube kernel: ? rcu_dump_cpu_stacks+0x95/0xb9 Aug 14 16:59:58 BorgCube kernel: ? rcu_sched_clock_irq+0x345/0xa48 Aug 14 16:59:58 BorgCube kernel: ? tick_init_jiffy_update+0x7c/0x7c Aug 14 16:59:58 BorgCube kernel: ? update_process_times+0x62/0x81 Aug 14 16:59:58 BorgCube kernel: ? tick_sched_timer+0x43/0x71 Aug 14 16:59:58 BorgCube kernel: ? __hrtimer_run_queues+0xe8/0x190 Aug 14 16:59:58 BorgCube kernel: ? hrtimer_interrupt+0x9c/0x16e Aug 14 16:59:58 BorgCube kernel: ? __sysvec_apic_timer_interrupt+0xc2/0x12f Aug 14 16:59:58 BorgCube kernel: ? sysvec_apic_timer_interrupt+0x27/0xa6 Aug 14 16:59:58 BorgCube kernel: ? asm_sysvec_apic_timer_interrupt+0x16/0x20 Aug 14 16:59:58 BorgCube kernel: ? sg_init_table+0xb/0x2a Aug 14 16:59:58 BorgCube kernel: ? _raw_spin_unlock+0x14/0x29 Aug 14 16:59:58 BorgCube kernel: bottom_half+0x18e/0x8ef [r8152] Aug 14 16:59:58 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 16:59:58 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 16:59:58 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 16:59:58 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 16:59:58 BorgCube kernel: </IRQ> Aug 14 16:59:58 BorgCube kernel: <TASK> Aug 14 16:59:58 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 16:59:58 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 16:59:58 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 16:59:58 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 16:59:58 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 16:59:58 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 16:59:58 BorgCube kernel: kthread+0xe4/0xef Aug 14 16:59:58 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 16:59:58 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 16:59:58 BorgCube kernel: </TASK> Aug 14 17:00:03 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:08 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:13 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:20 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:25 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:29 BorgCube kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 3-.... } 87705 jiffies s: 74473 root: 0x8/. Aug 14 17:00:29 BorgCube kernel: rcu: blocking rcu_node structures (internal RCU debug): Aug 14 17:00:29 BorgCube kernel: Sending NMI from CPU 7 to CPUs 3: Aug 14 17:00:29 BorgCube kernel: NMI backtrace for cpu 3 Aug 14 17:00:29 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 17:00:29 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 17:00:29 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 17:00:29 BorgCube kernel: RIP: 0010:bottom_half+0xc1/0x8ef [r8152] Aug 14 17:00:29 BorgCube kernel: Code: b0 fe ff ff 0f 84 26 08 00 00 49 8d 84 24 a0 fe ff ff 48 89 44 24 10 49 8b 84 24 a0 fe ff ff 48 39 44 24 10 0f 84 06 08 00 00 <49> 8d 84 24 e4 fe ff ff 31 db 48 89 c7 48 89 44 24 18 e8 d0 18 4e Aug 14 17:00:29 BorgCube kernel: RSP: 0018:ffffc900001d8ec0 EFLAGS: 00000212 Aug 14 17:00:29 BorgCube kernel: RAX: ffff888103c98b08 RBX: ffff888103c98ab0 RCX: ffff888103c98c40 Aug 14 17:00:29 BorgCube kernel: RDX: 0000000000000102 RSI: 0000000000000246 RDI: 00000000ffffffff Aug 14 17:00:29 BorgCube kernel: RBP: ffffc900001d8f70 R08: 0000000000000000 R09: 0000000000000082 Aug 14 17:00:29 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103c98da0 Aug 14 17:00:29 BorgCube kernel: R13: ffff888101143c00 R14: ffff888103c98000 R15: 0000000000001296 Aug 14 17:00:29 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 17:00:29 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 17:00:29 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 17:00:29 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 17:00:29 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 17:00:29 BorgCube kernel: Call Trace: Aug 14 17:00:29 BorgCube kernel: <NMI> Aug 14 17:00:29 BorgCube kernel: ? nmi_cpu_backtrace+0xd3/0x104 Aug 14 17:00:29 BorgCube kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Aug 14 17:00:29 BorgCube kernel: ? nmi_handle+0x54/0x131 Aug 14 17:00:29 BorgCube kernel: ? bottom_half+0xc1/0x8ef [r8152] Aug 14 17:00:29 BorgCube kernel: ? default_do_nmi+0x66/0x15b Aug 14 17:00:29 BorgCube kernel: ? exc_nmi+0xbf/0x11c Aug 14 17:00:29 BorgCube kernel: ? end_repeat_nmi+0x16/0x67 Aug 14 17:00:29 BorgCube kernel: ? bottom_half+0xc1/0x8ef [r8152] Aug 14 17:00:29 BorgCube kernel: ? bottom_half+0xc1/0x8ef [r8152] Aug 14 17:00:29 BorgCube kernel: ? bottom_half+0xc1/0x8ef [r8152] Aug 14 17:00:29 BorgCube kernel: </NMI> Aug 14 17:00:29 BorgCube kernel: <IRQ> Aug 14 17:00:29 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 17:00:29 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 17:00:29 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 17:00:29 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 17:00:29 BorgCube kernel: </IRQ> Aug 14 17:00:29 BorgCube kernel: <TASK> Aug 14 17:00:29 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 17:00:29 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 17:00:29 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 17:00:29 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 17:00:29 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 17:00:29 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 17:00:29 BorgCube kernel: kthread+0xe4/0xef Aug 14 17:00:29 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 17:00:29 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 17:00:29 BorgCube kernel: </TASK> Aug 14 17:00:30 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:35 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:42 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:47 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:52 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:00:57 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:04 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:09 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:14 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:20 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:26 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:31 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:35 BorgCube kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 3-.... } 153241 jiffies s: 74473 root: 0x8/. Aug 14 17:01:35 BorgCube kernel: rcu: blocking rcu_node structures (internal RCU debug): Aug 14 17:01:35 BorgCube kernel: Sending NMI from CPU 7 to CPUs 3: Aug 14 17:01:35 BorgCube kernel: NMI backtrace for cpu 3 Aug 14 17:01:35 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 17:01:35 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 17:01:35 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 17:01:35 BorgCube kernel: RIP: 0010:netif_unfreeze_queues+0x24/0x3c Aug 14 17:01:35 BorgCube kernel: Code: 01 c3 cc cc cc cc 0f 1f 44 00 00 55 31 ed 53 48 89 fb 3b ab 88 03 00 00 73 21 89 ef 48 69 ff 40 01 00 00 48 03 bb 80 03 00 00 <f0> 80 a7 90 00 00 00 fb e8 8e 75 fb ff ff c5 eb d7 5b 5d c3 cc cc Aug 14 17:01:35 BorgCube kernel: RSP: 0018:ffffc900001d8e98 EFLAGS: 00000286 Aug 14 17:01:35 BorgCube kernel: RAX: 0000000000000000 RBX: ffff888103c98000 RCX: ffff888146d564e0 Aug 14 17:01:35 BorgCube kernel: RDX: 0000000000000103 RSI: ffffffff820d96e1 RDI: ffff888101143c00 Aug 14 17:01:35 BorgCube kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000082 Aug 14 17:01:35 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103c98da0 Aug 14 17:01:35 BorgCube kernel: R13: ffff888101143c00 R14: ffff888103c98000 R15: 0000000000001296 Aug 14 17:01:35 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 17:01:35 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 17:01:35 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 17:01:35 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 17:01:35 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 17:01:35 BorgCube kernel: Call Trace: Aug 14 17:01:35 BorgCube kernel: <NMI> Aug 14 17:01:35 BorgCube kernel: ? nmi_cpu_backtrace+0xd3/0x104 Aug 14 17:01:35 BorgCube kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Aug 14 17:01:35 BorgCube kernel: ? nmi_handle+0x54/0x131 Aug 14 17:01:35 BorgCube kernel: ? netif_unfreeze_queues+0x24/0x3c Aug 14 17:01:35 BorgCube kernel: ? default_do_nmi+0x66/0x15b Aug 14 17:01:35 BorgCube kernel: ? exc_nmi+0xbf/0x11c Aug 14 17:01:35 BorgCube kernel: ? end_repeat_nmi+0x16/0x67 Aug 14 17:01:35 BorgCube kernel: ? netif_unfreeze_queues+0x24/0x3c Aug 14 17:01:35 BorgCube kernel: ? netif_unfreeze_queues+0x24/0x3c Aug 14 17:01:35 BorgCube kernel: ? netif_unfreeze_queues+0x24/0x3c Aug 14 17:01:35 BorgCube kernel: </NMI> Aug 14 17:01:35 BorgCube kernel: <IRQ> Aug 14 17:01:35 BorgCube kernel: netif_tx_unlock+0xe/0x1b Aug 14 17:01:35 BorgCube kernel: bottom_half+0x436/0x8ef [r8152] Aug 14 17:01:35 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 17:01:35 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 17:01:35 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 17:01:35 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 17:01:35 BorgCube kernel: </IRQ> Aug 14 17:01:35 BorgCube kernel: <TASK> Aug 14 17:01:35 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 17:01:35 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 17:01:35 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 17:01:35 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 17:01:35 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 17:01:35 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 17:01:35 BorgCube kernel: kthread+0xe4/0xef Aug 14 17:01:35 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 17:01:35 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 17:01:35 BorgCube kernel: </TASK> Aug 14 17:01:36 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:42 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:48 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:53 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:01:58 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:05 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:10 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:15 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:20 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:27 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:32 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:37 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:40 BorgCube kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 3-.... } 218777 jiffies s: 74473 root: 0x8/. Aug 14 17:02:40 BorgCube kernel: rcu: blocking rcu_node structures (internal RCU debug): Aug 14 17:02:40 BorgCube kernel: Sending NMI from CPU 7 to CPUs 3: Aug 14 17:02:40 BorgCube kernel: NMI backtrace for cpu 3 Aug 14 17:02:40 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 17:02:40 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 17:02:40 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 17:02:40 BorgCube kernel: RIP: 0010:netif_freeze_queues+0x53/0x77 Aug 14 17:02:40 BorgCube kernel: Code: db 40 01 00 00 48 03 9d 80 03 00 00 4c 8d b3 80 00 00 00 4c 89 f7 e8 78 a9 14 00 44 89 ab 84 00 00 00 f0 80 8b 90 00 00 00 04 <c7> 83 84 00 00 00 ff ff ff ff 4c 89 f7 41 ff c4 e8 2d aa 14 00 eb Aug 14 17:02:40 BorgCube kernel: RSP: 0018:ffffc900001d8e90 EFLAGS: 00000206 Aug 14 17:02:40 BorgCube kernel: RAX: 0000000000000000 RBX: ffff888101143c00 RCX: ffff888146d564e0 Aug 14 17:02:40 BorgCube kernel: RDX: 0000000000000001 RSI: ffffffff820d96e1 RDI: ffff888101143c80 Aug 14 17:02:40 BorgCube kernel: RBP: ffff888103c98000 R08: 0000000000000000 R09: 0000000000000082 Aug 14 17:02:40 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 Aug 14 17:02:40 BorgCube kernel: R13: 0000000000000003 R14: ffff888101143c80 R15: 0000000000001296 Aug 14 17:02:40 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 17:02:40 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 17:02:40 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 17:02:40 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 17:02:40 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 17:02:40 BorgCube kernel: Call Trace: Aug 14 17:02:40 BorgCube kernel: <NMI> Aug 14 17:02:40 BorgCube kernel: ? nmi_cpu_backtrace+0xd3/0x104 Aug 14 17:02:40 BorgCube kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Aug 14 17:02:40 BorgCube kernel: ? nmi_handle+0x54/0x131 Aug 14 17:02:40 BorgCube kernel: ? netif_freeze_queues+0x53/0x77 Aug 14 17:02:40 BorgCube kernel: ? default_do_nmi+0x66/0x15b Aug 14 17:02:40 BorgCube kernel: ? exc_nmi+0xbf/0x11c Aug 14 17:02:40 BorgCube kernel: ? end_repeat_nmi+0x16/0x67 Aug 14 17:02:40 BorgCube kernel: ? netif_freeze_queues+0x53/0x77 Aug 14 17:02:40 BorgCube kernel: ? netif_freeze_queues+0x53/0x77 Aug 14 17:02:40 BorgCube kernel: ? netif_freeze_queues+0x53/0x77 Aug 14 17:02:40 BorgCube kernel: </NMI> Aug 14 17:02:40 BorgCube kernel: <IRQ> Aug 14 17:02:40 BorgCube kernel: bottom_half+0x401/0x8ef [r8152] Aug 14 17:02:40 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 17:02:40 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 17:02:40 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 17:02:40 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 17:02:40 BorgCube kernel: </IRQ> Aug 14 17:02:40 BorgCube kernel: <TASK> Aug 14 17:02:40 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 17:02:40 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 17:02:40 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 17:02:40 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 17:02:40 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 17:02:40 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 17:02:40 BorgCube kernel: kthread+0xe4/0xef Aug 14 17:02:40 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 17:02:40 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 17:02:40 BorgCube kernel: </TASK> Aug 14 17:02:42 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:49 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:54 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:02:58 BorgCube kernel: rcu: INFO: rcu_preempt self-detected stall on CPU Aug 14 17:02:58 BorgCube kernel: rcu: #0113-....: (1 GPs behind) idle=ae14/1/0x4000000000000000 softirq=20606672/20606673 fqs=107458 Aug 14 17:02:58 BorgCube kernel: #011(t=240005 jiffies g=37415333 q=4196028 ncpus=8) Aug 14 17:02:58 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 17:02:58 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 17:02:58 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 17:02:58 BorgCube kernel: RIP: 0010:_raw_spin_unlock_irqrestore+0x1a/0x3a Aug 14 17:02:58 BorgCube kernel: Code: 79 7e 85 c0 75 05 0f 1f 44 00 00 c3 cc cc cc cc 0f 1f 44 00 00 53 48 89 f3 e8 7a 54 83 ff 0f ba e3 09 73 06 fb 0f 1f 44 00 00 <bf> 01 00 00 00 e8 e7 15 81 ff 65 8b 05 54 4f 79 7e 85 c0 75 05 0f Aug 14 17:02:58 BorgCube kernel: RSP: 0018:ffffc900001d8eb0 EFLAGS: 00000247 Aug 14 17:02:58 BorgCube kernel: RAX: ffff888103c98bb8 RBX: 0000000000000246 RCX: ffff888103c98c40 Aug 14 17:02:58 BorgCube kernel: RDX: 0000000000000001 RSI: 0000000000000246 RDI: ffff888103c98c84 Aug 14 17:02:58 BorgCube kernel: RBP: ffffc900001d8f70 R08: 0000000000000000 R09: 0000000000000082 Aug 14 17:02:58 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103c98da0 Aug 14 17:02:58 BorgCube kernel: R13: ffff888101143c00 R14: ffff888103c98000 R15: 0000000000001296 Aug 14 17:02:58 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 17:02:58 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 17:02:58 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 17:02:58 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 17:02:58 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 17:02:58 BorgCube kernel: Call Trace: Aug 14 17:02:58 BorgCube kernel: <IRQ> Aug 14 17:02:58 BorgCube kernel: ? rcu_dump_cpu_stacks+0x95/0xb9 Aug 14 17:02:58 BorgCube kernel: ? rcu_sched_clock_irq+0x345/0xa48 Aug 14 17:02:58 BorgCube kernel: ? sched_slice+0xaf/0x101 Aug 14 17:02:58 BorgCube kernel: ? tick_init_jiffy_update+0x7c/0x7c Aug 14 17:02:58 BorgCube kernel: ? update_process_times+0x62/0x81 Aug 14 17:02:58 BorgCube kernel: ? tick_sched_timer+0x43/0x71 Aug 14 17:02:58 BorgCube kernel: ? __hrtimer_run_queues+0xe8/0x190 Aug 14 17:02:58 BorgCube kernel: ? hrtimer_interrupt+0x9c/0x16e Aug 14 17:02:58 BorgCube kernel: ? __sysvec_apic_timer_interrupt+0xc2/0x12f Aug 14 17:02:58 BorgCube kernel: ? sysvec_apic_timer_interrupt+0x27/0xa6 Aug 14 17:02:58 BorgCube kernel: ? asm_sysvec_apic_timer_interrupt+0x16/0x20 Aug 14 17:02:58 BorgCube kernel: ? _raw_spin_unlock_irqrestore+0x1a/0x3a Aug 14 17:02:58 BorgCube kernel: bottom_half+0x473/0x8ef [r8152] Aug 14 17:02:58 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 17:02:58 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 17:02:58 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 17:02:58 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 17:02:58 BorgCube kernel: </IRQ> Aug 14 17:02:58 BorgCube kernel: <TASK> Aug 14 17:02:58 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 17:02:58 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 17:02:58 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 17:02:58 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 17:02:58 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 17:02:58 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 17:02:58 BorgCube kernel: kthread+0xe4/0xef Aug 14 17:02:58 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 17:02:58 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 17:02:58 BorgCube kernel: </TASK> Aug 14 17:02:59 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:04 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:11 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:16 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:21 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:26 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:33 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:38 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:43 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:46 BorgCube kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 3-.... } 284313 jiffies s: 74473 root: 0x8/. Aug 14 17:03:46 BorgCube kernel: rcu: blocking rcu_node structures (internal RCU debug): Aug 14 17:03:46 BorgCube kernel: Sending NMI from CPU 7 to CPUs 3: Aug 14 17:03:46 BorgCube kernel: NMI backtrace for cpu 3 Aug 14 17:03:46 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 17:03:46 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 17:03:46 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 17:03:46 BorgCube kernel: RIP: 0010:netif_freeze_queues+0x53/0x77 Aug 14 17:03:46 BorgCube kernel: Code: db 40 01 00 00 48 03 9d 80 03 00 00 4c 8d b3 80 00 00 00 4c 89 f7 e8 78 a9 14 00 44 89 ab 84 00 00 00 f0 80 8b 90 00 00 00 04 <c7> 83 84 00 00 00 ff ff ff ff 4c 89 f7 41 ff c4 e8 2d aa 14 00 eb Aug 14 17:03:46 BorgCube kernel: RSP: 0018:ffffc900001d8e90 EFLAGS: 00000206 Aug 14 17:03:46 BorgCube kernel: RAX: 0000000000000000 RBX: ffff888101143c00 RCX: ffff888146d564e0 Aug 14 17:03:46 BorgCube kernel: RDX: 0000000000000001 RSI: ffffffff820d96e1 RDI: ffff888101143c80 Aug 14 17:03:46 BorgCube kernel: RBP: ffff888103c98000 R08: 0000000000000000 R09: 0000000000000082 Aug 14 17:03:46 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 Aug 14 17:03:46 BorgCube kernel: R13: 0000000000000003 R14: ffff888101143c80 R15: 0000000000001296 Aug 14 17:03:46 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 17:03:46 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 17:03:46 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 17:03:46 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 17:03:46 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 17:03:46 BorgCube kernel: Call Trace: Aug 14 17:03:46 BorgCube kernel: <NMI> Aug 14 17:03:46 BorgCube kernel: ? nmi_cpu_backtrace+0xd3/0x104 Aug 14 17:03:46 BorgCube kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Aug 14 17:03:46 BorgCube kernel: ? nmi_handle+0x54/0x131 Aug 14 17:03:46 BorgCube kernel: ? netif_freeze_queues+0x53/0x77 Aug 14 17:03:46 BorgCube kernel: ? default_do_nmi+0x66/0x15b Aug 14 17:03:46 BorgCube kernel: ? exc_nmi+0xbf/0x11c Aug 14 17:03:46 BorgCube kernel: ? end_repeat_nmi+0x16/0x67 Aug 14 17:03:46 BorgCube kernel: ? netif_freeze_queues+0x53/0x77 Aug 14 17:03:46 BorgCube kernel: ? netif_freeze_queues+0x53/0x77 Aug 14 17:03:46 BorgCube kernel: ? netif_freeze_queues+0x53/0x77 Aug 14 17:03:46 BorgCube kernel: </NMI> Aug 14 17:03:46 BorgCube kernel: <IRQ> Aug 14 17:03:46 BorgCube kernel: bottom_half+0x401/0x8ef [r8152] Aug 14 17:03:46 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 17:03:46 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 17:03:46 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 17:03:46 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 17:03:46 BorgCube kernel: </IRQ> Aug 14 17:03:46 BorgCube kernel: <TASK> Aug 14 17:03:46 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 17:03:46 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 17:03:46 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 17:03:46 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 17:03:46 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 17:03:46 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 17:03:46 BorgCube kernel: kthread+0xe4/0xef Aug 14 17:03:46 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 17:03:46 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 17:03:46 BorgCube kernel: </TASK> Aug 14 17:03:48 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:03:55 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:00 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:05 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:11 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:17 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:22 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:27 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:34 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:39 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:44 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:49 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:04:51 BorgCube kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 3-.... } 349849 jiffies s: 74473 root: 0x8/. Aug 14 17:04:51 BorgCube kernel: rcu: blocking rcu_node structures (internal RCU debug): Aug 14 17:04:51 BorgCube kernel: Sending NMI from CPU 7 to CPUs 3: Aug 14 17:04:51 BorgCube kernel: NMI backtrace for cpu 3 Aug 14 17:04:51 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 17:04:51 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 17:04:51 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 17:04:51 BorgCube kernel: RIP: 0010:_raw_spin_unlock_irqrestore+0x1a/0x3a Aug 14 17:04:51 BorgCube kernel: Code: 79 7e 85 c0 75 05 0f 1f 44 00 00 c3 cc cc cc cc 0f 1f 44 00 00 53 48 89 f3 e8 7a 54 83 ff 0f ba e3 09 73 06 fb 0f 1f 44 00 00 <bf> 01 00 00 00 e8 e7 15 81 ff 65 8b 05 54 4f 79 7e 85 c0 75 05 0f Aug 14 17:04:51 BorgCube kernel: RSP: 0018:ffffc900001d8eb0 EFLAGS: 00000247 Aug 14 17:04:51 BorgCube kernel: RAX: ffff888103c98b08 RBX: 0000000000000246 RCX: ffff888103c98c40 Aug 14 17:04:51 BorgCube kernel: RDX: 0000000000000001 RSI: 0000000000000246 RDI: ffff888103c98c84 Aug 14 17:04:51 BorgCube kernel: RBP: ffffc900001d8f70 R08: 0000000000000000 R09: 0000000000000082 Aug 14 17:04:51 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103c98da0 Aug 14 17:04:51 BorgCube kernel: R13: ffff888101143c00 R14: ffff888103c98000 R15: 0000000000001296 Aug 14 17:04:51 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 17:04:51 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 17:04:51 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 17:04:51 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 17:04:51 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 17:04:51 BorgCube kernel: Call Trace: Aug 14 17:04:51 BorgCube kernel: <NMI> Aug 14 17:04:51 BorgCube kernel: ? nmi_cpu_backtrace+0xd3/0x104 Aug 14 17:04:51 BorgCube kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Aug 14 17:04:51 BorgCube kernel: ? nmi_handle+0x54/0x131 Aug 14 17:04:51 BorgCube kernel: ? _raw_spin_unlock_irqrestore+0x1a/0x3a Aug 14 17:04:51 BorgCube kernel: ? default_do_nmi+0x66/0x15b Aug 14 17:04:51 BorgCube kernel: ? exc_nmi+0xbf/0x11c Aug 14 17:04:51 BorgCube kernel: ? end_repeat_nmi+0x16/0x67 Aug 14 17:04:51 BorgCube kernel: ? _raw_spin_unlock_irqrestore+0x1a/0x3a Aug 14 17:04:51 BorgCube kernel: ? _raw_spin_unlock_irqrestore+0x1a/0x3a Aug 14 17:04:51 BorgCube kernel: ? _raw_spin_unlock_irqrestore+0x1a/0x3a Aug 14 17:04:51 BorgCube kernel: </NMI> Aug 14 17:04:51 BorgCube kernel: <IRQ> Aug 14 17:04:51 BorgCube kernel: bottom_half+0x473/0x8ef [r8152] Aug 14 17:04:51 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 17:04:51 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 17:04:51 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 17:04:51 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 17:04:51 BorgCube kernel: </IRQ> Aug 14 17:04:51 BorgCube kernel: <TASK> Aug 14 17:04:51 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 17:04:51 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 17:04:51 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 17:04:51 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 17:04:51 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 17:04:51 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 17:04:51 BorgCube kernel: kthread+0xe4/0xef Aug 14 17:04:51 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 17:04:51 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 17:04:51 BorgCube kernel: </TASK> Aug 14 17:04:56 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:01 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:06 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:11 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:18 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:23 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:28 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:33 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:40 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:45 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:50 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:55 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:05:57 BorgCube kernel: rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { 3-.... } 415385 jiffies s: 74473 root: 0x8/. Aug 14 17:05:57 BorgCube kernel: rcu: blocking rcu_node structures (internal RCU debug): Aug 14 17:05:57 BorgCube kernel: Sending NMI from CPU 7 to CPUs 3: Aug 14 17:05:57 BorgCube kernel: NMI backtrace for cpu 3 Aug 14 17:05:57 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 17:05:57 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 17:05:57 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 17:05:57 BorgCube kernel: RIP: 0010:netif_freeze_queues+0x13/0x77 Aug 14 17:05:57 BorgCube kernel: Code: c3 cc cc cc cc 48 89 f2 48 89 fe 48 c7 c7 20 d1 3a 82 e9 c1 f9 cf ff 0f 1f 44 00 00 41 56 41 55 41 54 45 31 e4 55 48 89 fd 53 <e8> 17 f0 13 00 41 89 c5 44 3b a5 88 03 00 00 73 46 44 89 e3 48 69 Aug 14 17:05:57 BorgCube kernel: RSP: 0018:ffffc900001d8e90 EFLAGS: 00000246 Aug 14 17:05:57 BorgCube kernel: RAX: 0000000000000000 RBX: ffff888103c98ab0 RCX: ffff888146d564e0 Aug 14 17:05:57 BorgCube kernel: RDX: 0000000000000001 RSI: ffff888103c98c50 RDI: ffff888103c98000 Aug 14 17:05:57 BorgCube kernel: RBP: ffff888103c98000 R08: 0000000000000000 R09: 0000000000000082 Aug 14 17:05:57 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 Aug 14 17:05:57 BorgCube kernel: R13: ffffc900001d8f20 R14: ffff888103c98000 R15: 0000000000001296 Aug 14 17:05:57 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 17:05:57 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 17:05:57 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 17:05:57 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 17:05:57 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 17:05:57 BorgCube kernel: Call Trace: Aug 14 17:05:57 BorgCube kernel: <NMI> Aug 14 17:05:57 BorgCube kernel: ? nmi_cpu_backtrace+0xd3/0x104 Aug 14 17:05:57 BorgCube kernel: ? nmi_cpu_backtrace_handler+0xd/0x15 Aug 14 17:05:57 BorgCube kernel: ? nmi_handle+0x54/0x131 Aug 14 17:05:57 BorgCube kernel: ? netif_freeze_queues+0x13/0x77 Aug 14 17:05:57 BorgCube kernel: ? default_do_nmi+0x66/0x15b Aug 14 17:05:57 BorgCube kernel: ? exc_nmi+0xbf/0x11c Aug 14 17:05:57 BorgCube kernel: ? end_repeat_nmi+0x16/0x67 Aug 14 17:05:57 BorgCube kernel: ? netif_freeze_queues+0x13/0x77 Aug 14 17:05:57 BorgCube kernel: ? netif_freeze_queues+0x13/0x77 Aug 14 17:05:57 BorgCube kernel: ? netif_freeze_queues+0x13/0x77 Aug 14 17:05:57 BorgCube kernel: </NMI> Aug 14 17:05:57 BorgCube kernel: <IRQ> Aug 14 17:05:57 BorgCube kernel: bottom_half+0x401/0x8ef [r8152] Aug 14 17:05:57 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 17:05:57 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 17:05:57 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 17:05:57 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 17:05:57 BorgCube kernel: </IRQ> Aug 14 17:05:57 BorgCube kernel: <TASK> Aug 14 17:05:57 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 17:05:57 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 17:05:57 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 17:05:57 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 17:05:57 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 17:05:57 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 17:05:57 BorgCube kernel: kthread+0xe4/0xef Aug 14 17:05:57 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 17:05:57 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 17:05:57 BorgCube kernel: </TASK> Aug 14 17:05:58 BorgCube kernel: rcu: INFO: rcu_preempt self-detected stall on CPU Aug 14 17:05:58 BorgCube kernel: rcu: #0113-....: (1 GPs behind) idle=ae14/1/0x4000000000000000 softirq=20606672/20606673 fqs=186790 Aug 14 17:05:58 BorgCube kernel: #011(t=420008 jiffies g=37415333 q=5508715 ncpus=8) Aug 14 17:05:58 BorgCube kernel: CPU: 3 PID: 17455 Comm: kworker/3:2 Tainted: P W O 6.1.99-Unraid #1 Aug 14 17:05:58 BorgCube kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z390M Pro4, BIOS P4.30 12/03/2019 Aug 14 17:05:58 BorgCube kernel: Workqueue: wg-crypt-wg0 wg_packet_tx_worker [wireguard] Aug 14 17:05:58 BorgCube kernel: RIP: 0010:preempt_count_sub+0x15/0x93 Aug 14 17:05:58 BorgCube kernel: Code: 77 0e 11 00 0f 1f 44 00 00 48 c7 c7 30 09 95 82 e9 96 0d 11 00 0f 1f 44 00 00 83 3d 80 9d 8b 01 00 75 77 65 8b 05 5f 39 f8 7e <89> c2 81 e2 ff ff ff 7f 39 fa 7d 2c e8 c0 d5 39 00 85 c0 74 64 83 Aug 14 17:05:58 BorgCube kernel: RSP: 0018:ffffc900001d8eb0 EFLAGS: 00000246 Aug 14 17:05:58 BorgCube kernel: RAX: 0000000000000102 RBX: ffff888103c98bb8 RCX: ffff888146d564e0 Aug 14 17:05:58 BorgCube kernel: RDX: ffff8881028a8000 RSI: ffffffff820d96e1 RDI: 0000000000000001 Aug 14 17:05:58 BorgCube kernel: RBP: ffffc900001d8f70 R08: 0000000000000000 R09: 0000000000000082 Aug 14 17:05:58 BorgCube kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103c98da0 Aug 14 17:05:58 BorgCube kernel: R13: ffff888101143c00 R14: ffff888103c98000 R15: 0000000000001296 Aug 14 17:05:58 BorgCube kernel: FS: 0000000000000000(0000) GS:ffff88885dcc0000(0000) knlGS:0000000000000000 Aug 14 17:05:58 BorgCube kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 14 17:05:58 BorgCube kernel: CR2: 0000153e416f7000 CR3: 000000000220a006 CR4: 00000000003706e0 Aug 14 17:05:58 BorgCube kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Aug 14 17:05:58 BorgCube kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Aug 14 17:05:58 BorgCube kernel: Call Trace: Aug 14 17:05:58 BorgCube kernel: <IRQ> Aug 14 17:05:58 BorgCube kernel: ? rcu_dump_cpu_stacks+0x95/0xb9 Aug 14 17:05:58 BorgCube kernel: ? rcu_sched_clock_irq+0x345/0xa48 Aug 14 17:05:58 BorgCube kernel: ? nohz_balance_exit_idle+0x16/0xab Aug 14 17:05:58 BorgCube kernel: ? trigger_load_balance+0x72/0x307 Aug 14 17:05:58 BorgCube kernel: ? tick_init_jiffy_update+0x7c/0x7c Aug 14 17:05:58 BorgCube kernel: ? update_process_times+0x62/0x81 Aug 14 17:05:58 BorgCube kernel: ? tick_sched_timer+0x43/0x71 Aug 14 17:05:58 BorgCube kernel: ? __hrtimer_run_queues+0xe8/0x190 Aug 14 17:05:58 BorgCube kernel: ? hrtimer_interrupt+0x9c/0x16e Aug 14 17:05:58 BorgCube kernel: ? __sysvec_apic_timer_interrupt+0xc2/0x12f Aug 14 17:05:58 BorgCube kernel: ? sysvec_apic_timer_interrupt+0x27/0xa6 Aug 14 17:05:58 BorgCube kernel: ? asm_sysvec_apic_timer_interrupt+0x16/0x20 Aug 14 17:05:58 BorgCube kernel: ? preempt_count_sub+0x15/0x93 Aug 14 17:05:58 BorgCube kernel: _raw_spin_unlock+0x14/0x29 Aug 14 17:05:58 BorgCube kernel: bottom_half+0x436/0x8ef [r8152] Aug 14 17:05:58 BorgCube kernel: ? _raw_spin_unlock_irq+0x1a/0x2f Aug 14 17:05:58 BorgCube kernel: tasklet_action_common.constprop.0+0x60/0xa6 Aug 14 17:05:58 BorgCube kernel: handle_softirqs+0x129/0x271 Aug 14 17:05:58 BorgCube kernel: do_softirq+0x7f/0xab Aug 14 17:05:58 BorgCube kernel: </IRQ> Aug 14 17:05:58 BorgCube kernel: <TASK> Aug 14 17:05:58 BorgCube kernel: __local_bh_enable_ip+0x4c/0x6b Aug 14 17:05:58 BorgCube kernel: wg_socket_send_skb_to_peer+0x9c/0xab [wireguard] Aug 14 17:05:58 BorgCube kernel: wg_packet_tx_worker+0xa6/0x18a [wireguard] Aug 14 17:05:58 BorgCube kernel: process_one_work+0x1a8/0x295 Aug 14 17:05:58 BorgCube kernel: worker_thread+0x18b/0x244 Aug 14 17:05:58 BorgCube kernel: ? rescuer_thread+0x281/0x281 Aug 14 17:05:58 BorgCube kernel: kthread+0xe4/0xef Aug 14 17:05:58 BorgCube kernel: ? kthread_complete_and_exit+0x1b/0x1b Aug 14 17:05:58 BorgCube kernel: ret_from_fork+0x1f/0x30 Aug 14 17:05:58 BorgCube kernel: </TASK> Aug 14 17:06:02 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:07 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:12 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:17 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:24 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:29 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:34 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:40 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:46 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:51 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:06:56 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout Aug 14 17:07:02 BorgCube kernel: r8152 2-4:1.0 eth0: Tx timeout borgcube-diagnostics-20240814-1801.zip

I've tried your suggestion @yayitazale but the new tensorRT model generation isn't very clear to me... I'm running into the following error when running the frigate:0.13.0-beta1-tensorrt docker I don't see any reference to having to build the new models myself in the new doc: https://deploy-preview-6262--frigate-docs.netlify.app/configuration/object_detectors/#generate-models D̶o̶ ̶w̶e̶ ̶s̶t̶i̶l̶l̶ ̶n̶e̶e̶d̶ ̶t̶o̶ ̶u̶s̶e̶ ̶t̶h̶e̶ ̶"̶t̶e̶n̶s̶o̶r̶r̶t̶-̶m̶o̶d̶e̶l̶s̶"̶ ̶d̶o̶c̶k̶e̶r̶ ̶w̶i̶t̶h̶ ̶t̶h̶e̶ ̶p̶r̶o̶c̶e̶s̶s̶ ̶o̶u̶t̶l̶i̶n̶e̶d̶ ̶i̶n̶ ̶t̶h̶e̶ ̶p̶r̶e̶v̶i̶o̶u̶s̶ ̶d̶o̶c̶ ̶f̶o̶r̶ ̶v̶1̶2̶ ̶ (like here) ̶t̶o̶ ̶g̶e̶n̶e̶r̶a̶t̶e̶ ̶t̶h̶e̶ ̶m̶o̶d̶e̶l̶s̶ ̶o̶r̶ ̶i̶s̶ ̶t̶h̶e̶ ̶n̶e̶w̶ ̶B̶e̶t̶a̶ ̶1̶3̶ ̶t̶a̶k̶i̶n̶g̶ ̶c̶a̶r̶e̶ ̶o̶f̶ ̶i̶t̶ ̶a̶l̶l̶ ̶w̶i̶t̶h̶i̶n̶ ̶t̶h̶e̶ ̶n̶e̶w̶ ̶c̶o̶n̶t̶a̶i̶n̶e̶r̶?̶ EDIT: nevermind, found the issue, I had to re-add --runtime=nvidia to the extra parameter of the docker container

Updated to 6.10.3, running an intel CPU (Q9400), appdata is set up as "Prefer: Cache" (should I set it up as cache only?). If we're talking about ServerFiles path it's `/mnt/user/appdata/vrising`. Still seeing the same errors after update: tail: cannot open '/serverdata/serverfiles/logs/VRisingServer.log' for reading: No such file or directory tail: no files remaining tail: invalid PID: ‘’ /opt/scripts/start-watchdog.sh: line 6: kill: `': not a pid or valid job spec tail: cannot open '/serverdata/serverfiles/logs/VRisingServer.log' for reading: No such file or directory tail: no files remaining tail: invalid PID: ‘’ /opt/scripts/start-watchdog.sh: line 6: kill: `': not a pid or valid job spec Session terminated, killing shell...kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

Hello, running into some issues when running the V Rising version of the container, I left all the parameters as default, didn't change any configuration, and on first boot, here is the log it produces, it keeps looping after a certain point ---Ensuring UID: 99 matches user--- ---Ensuring GID: 100 matches user--- ---Setting umask to 000--- ---Checking for optional scripts--- ---No optional script found, continuing--- ---Taking ownership of data...--- ---Starting...--- ---Update SteamCMD--- Redirecting stderr to '/serverdata/Steam/logs/stderr.txt' [ 0%] Checking for available updates... [----] Verifying installation... Steam Console Client (c) Valve Corporation - version 1654574676 -- type 'quit' to exit -- Loading Steam API...OK Connecting anonymously to Steam Public...OK Waiting for client config...OK /serverdata/steamcmd/steamcmd.sh: line 39: 32 Killed $DEBUGGER "$STEAMEXE" "$@" Waiting for user info...---Update Server--- Redirecting stderr to '/serverdata/Steam/logs/stderr.txt' Looks like steam didn't shutdown cleanly, scheduling immediate update check [ 0%] Checking for available updates... [----] Verifying installation... Steam Console Client (c) Valve Corporation - version 1654574676 -- type 'quit' to exit -- Loading Steam API...OK "@sSteamCmdForcePlatformType" = "windows" �[0m Connecting anonymously to Steam Public...OK Waiting for client config...OK /serverdata/steamcmd/steamcmd.sh: line 39: 54 Killed $DEBUGGER "$STEAMEXE" "$@" Waiting for user info...---BepInEx for V Rising disabled!--- ---Checking if WINE workdirectory is present--- ---WINE workdirectory found--- ---Checking if WINE is properly installed--- ---WINE properly set up--- ---Checking for old display lock files--- ---Server ready--- ---Start Server--- tail: cannot open '/serverdata/serverfiles/logs/VRisingServer.log' for reading: No such file or directory tail: no files remaining tail: invalid PID: ‘’ /opt/scripts/start-watchdog.sh: line 6: kill: `': not a pid or valid job spec ---Ensuring UID: 99 matches user--- usermod: no changes ---Ensuring GID: 100 matches user--- usermod: no changes ---Setting umask to 000--- ---Checking for optional scripts--- ---No optional script found, continuing--- ---Taking ownership of data...--- ---Starting...--- ---Update SteamCMD--- ---Starting...--- ---Update SteamCMD--- Redirecting stderr to '/serverdata/Steam/logs/stderr.txt' Looks like steam didn't shutdown cleanly, scheduling immediate update check [ 0%] Checking for available updates... [----] Verifying installation... Steam Console Client (c) Valve Corporation - version 1654574676 -- type 'quit' to exit -- Loading Steam API...OK Connecting anonymously to Steam Public...OK Waiting for client config...OK /serverdata/steamcmd/steamcmd.sh: line 39: 20 Killed $DEBUGGER "$STEAMEXE" "$@" Waiting for user info...---Update Server--- Redirecting stderr to '/serverdata/Steam/logs/stderr.txt' Looks like steam didn't shutdown cleanly, scheduling immediate update check [ 0%] Checking for available updates... [----] Verifying installation... Steam Console Client (c) Valve Corporation - version 1654574676 -- type 'quit' to exit -- Loading Steam API...OK "@sSteamCmdForcePlatformType" = "windows" �[0m Redirecting stderr to '/serverdata/Steam/logs/stderr.txt' Looks like steam didn't shutdown cleanly, scheduling immediate update check [ 0%] Checking for available updates... [----] Verifying installation... Steam Console Client (c) Valve Corporation - version 1654574676 -- type 'quit' to exit -- Loading Steam API...OK "@sSteamCmdForcePlatformType" = "windows" �[0m Connecting anonymously to Steam Public...OK Waiting for client config...OK /serverdata/steamcmd/steamcmd.sh: line 39: 40 Killed $DEBUGGER "$STEAMEXE" "$@" Waiting for user info...---BepInEx for V Rising disabled!--- ---Checking if WINE workdirectory is present--- ---WINE workdirectory found--- ---Checking if WINE is properly installed--- ---WINE properly set up--- ---Checking for old display lock files--- ---Server ready--- ---Start Server--- tail: cannot open '/serverdata/serverfiles/logs/VRisingServer.log' for reading: No such file or directory tail: no files remaining tail: invalid PID: ‘’ /opt/scripts/start-watchdog.sh: line 6: kill: `': not a pid or valid job spec ---Ensuring UID: 99 matches user--- usermod: no changes ---Ensuring GID: 100 matches user--- ---Ensuring UID: 99 matches user--- usermod: no changes ---Ensuring GID: 100 matches user--- usermod: no changes ---Setting umask to 000--- ---Checking for optional scripts--- ---No optional script found, continuing--- ---Taking ownership of data...--- ---Starting...--- ---Update SteamCMD--- Redirecting stderr to '/serverdata/Steam/logs/stderr.txt' Looks like steam didn't shutdown cleanly, scheduling immediate update check [ 0%] Checking for available updates... [----] Verifying installation... Steam Console Client (c) Valve Corporation - version 1654574676 -- type 'quit' to exit -- Loading Steam API...OK Redirecting stderr to '/serverdata/Steam/logs/stderr.txt' Looks like steam didn't shutdown cleanly, scheduling immediate update check [ 0%] Checking for available updates... [----] Verifying installation... Steam Console Client (c) Valve Corporation - version 1654574676 -- type 'quit' to exit -- Loading Steam API...OK Connecting anonymously to Steam Public...OK Waiting for client config...OK /serverdata/steamcmd/steamcmd.sh: line 39: 20 Killed $DEBUGGER "$STEAMEXE" "$@" Waiting for user info...---Update Server--- Maybe more specifically tied to this? Any help/insight appreciated! I have a Valheim version of the same container running flawlessly for months.

Apologies if this has been asked before, I looked but couldn't find an answer Regarding the Valheim (of course) dedicated server. I there a way to see the server logs themselves? I know how to access the docker logs (which might just be the exact server logs I'm talking about...), which works fine. But what I'm looking for is to be able to see which users connected to the server and at what time? Not sure if that can be surfaced in the docker logs or if that's even possible? Saw this image that outlined something similar, do I need to enable a different level of logging somewhere?