August 11, 20214 yr Hi guys, My Unraid server hangs every few days. I grabbed the below info from syslog showing the trace. I'm not a Linux guru at all so I have no clue what to look at that shows what happened. Any insights would be appreciated. Thanks! 2021-08-11 10:20:23 Kernel.Error 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: rcu: INFO: rcu_sched detected stalls on CPUs/tasks: 2021-08-11 10:20:23 Kernel.Error 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: rcu: 1-...0: (5 ticks this GP) idle=586/1/0x4000000000000000 softirq=41567442/41567443 fqs=14976 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: (detected by 0, t=60002 jiffies, g=65020929, q=2400) 2021-08-11 10:20:23 Kernel.Info 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: Sending NMI from CPU 0 to CPUs 1: 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: NMI backtrace for cpu 1 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: CPU: 1 PID: 17600 Comm: Tdarr_Node Tainted: P O 5.10.28-Unraid #1 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: Hardware name: HPE ProLiant MicroServer Gen10/ProLiant MicroServer Gen10, BIOS 5.12 06/26/2018 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x79/0x18a 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: Code: c1 e0 08 89 c2 8b 07 30 e4 09 d0 a9 00 01 ff ff 74 0c 0f ba e0 08 72 1a c6 47 01 00 eb 14 85 c0 74 0a 8b 07 84 c0 74 04 f3 90 <eb> f6 66 c7 07 01 00 c3 48 c7 c0 00 30 02 00 65 48 03 05 f0 8e f8 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RSP: 0018:ffffc9000015ce68 EFLAGS: 00000002 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RAX: 0000000000040101 RBX: ffffc9000015ce90 RCX: 0000000000000000 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8881000a9d40 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RBP: 8000015d00006c0d R08: ffffffff82013888 R09: ffffffff82013580 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: R10: 0001370722c9b2fd R11: 0000972caf555a8f R12: 00008044bfcf4231 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: R13: 0000000000000001 R14: 000000003b9aca00 R15: ffff8887ef49f040 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: FS: 000014f8eece3780(0000) GS:ffff8887ef480000(0000) knlGS:0000000000000000 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: CR2: 000017d29dc3b000 CR3: 0000000217226000 CR4: 00000000001506e0 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: Call Trace: 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: <IRQ> 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: queued_spin_lock_slowpath+0x7/0xa 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: nr_blockdev_pages+0x13/0x64 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: si_meminfo+0x3a/0x57 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: Sys_MemInfo+0x20/0x9b [corefreqk] 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? _raw_spin_unlock_irqrestore+0xd/0xe 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? try_to_wake_up+0x1b0/0x1e5 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? timekeeping_get_ns+0x19/0x2f 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? Sys_DumpTask+0xe9/0xf1 [corefreqk] 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: Cycle_AMD_Family_15h+0x265/0x3e6 [corefreqk] 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: __hrtimer_run_queues+0xb7/0x10b 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? Core_AMD_Family_15h_Temp+0x8b/0x8b [corefreqk] 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: hrtimer_interrupt+0x8d/0x15b 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: __sysvec_apic_timer_interrupt+0x5d/0x68 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: asm_call_irq_on_stack+0x12/0x20 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: </IRQ> 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: sysvec_apic_timer_interrupt+0x71/0x95 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: asm_sysvec_apic_timer_interrupt+0x12/0x20 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RIP: 0010:nr_blockdev_pages+0x4c/0x64 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: Code: 00 00 48 8d 97 48 05 00 00 48 2d 10 01 00 00 48 8d 88 10 01 00 00 48 39 d1 74 17 48 8b 48 30 48 8b 80 10 01 00 00 4c 03 41 58 <48> 2d 10 01 00 00 eb dd 48 81 c7 40 05 00 00 e8 fa de ff ff 4c 89 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RSP: 0018:ffffc90001adfcd8 EFLAGS: 00000202 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RAX: ffff888100444dc8 RBX: ffffc90001adfd20 RCX: ffff888100445128 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RDX: ffff8881000a9d48 RSI: 0000000000000001 RDI: ffff8881000a9800 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RBP: ffff888100a8e4b0 R08: 000000000000005b R09: ffff8883f6308400 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: R10: 0000000000001008 R11: 0000000000000001 R12: ffffc90001adfe80 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: R13: 0000000000000001 R14: ffff888100a8e4d8 R15: ffff888100a8e4b0 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? nr_blockdev_pages+0x13/0x64 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: si_meminfo+0x3a/0x57 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: meminfo_proc_show+0x33/0x540 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? __mod_memcg_state+0x5/0x9d 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? mod_objcg_state+0x2a/0x30 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? slab_post_alloc_hook+0x10c/0x14a 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ? __kmalloc_node+0x144/0x16d 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: seq_read_iter+0x15e/0x339 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: proc_reg_read_iter+0x4d/0x5f 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: new_sync_read+0x77/0xaa 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: vfs_read+0xbe/0xff 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: ksys_read+0x71/0xba 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: do_syscall_64+0x5d/0x6a 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RIP: 0033:0x14f8ee95636c 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: Code: ec 28 48 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 89 fc ff ff 48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 30 44 89 c7 48 89 44 24 08 e8 bf fc ff ff 48 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RSP: 002b:00007ffc2983d500 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RAX: ffffffffffffffda RBX: 0000000000000017 RCX: 000014f8ee95636c 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RDX: 0000000000000fff RSI: 00007ffc2983d540 RDI: 0000000000000017 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: RBP: 00007ffc2983e560 R08: 0000000000000000 R09: 0000339d65e80471 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 000017d29dc3b299 2021-08-11 10:20:23 Kernel.Warning 192.168.1.4 Aug 11 10:20:22 VLARPSERV kernel: R13: 0000000001ff3a48 R14: 000000000435ee88 R15: 00000000042d5658
August 11, 20214 yr Community Expert 8 hours ago, VlarpNL said: [corefreqk] You should always post the complete diagnostics, are you using the corefreq plugin? If yes try without it.
August 11, 20214 yr Author 43 minutes ago, JorgeB said: You should always post the complete diagnostics, are you using the corefreq plugin? If yes try without it. Yeah, I was just going through the logs and didn't think to post the full diagnostics, sorry about that. I had the corefreq plugin installed but I removed it. I'm not completely sure if I removed it before the hang or not. I attached the diagnostics but I just checked and it doesn't seem to contain the moment the hang/crash occured. The only thing that contains that is the log saved by my syslog server. I attached that log as well. vlarpserv-diagnostics-20210811-2054.zip Syslog.txt
August 12, 20214 yr Author 2 hours ago, JorgeB said: Removing the plugin should get rid of them. Thanks for the help. It's gone now, I'll monitor if my system is stable now.
September 12, 20214 yr Tested CPU are listed into wiki/cpu support. So far no AMD family 15h has contributed to the project.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.