I just transferred my old PC to an unraid server and tossed my old unraid equipment. The server is now slowing locking up over time until lit completely unresponsive it generally takes a few hours to get this point. I have did a memtest for 1 day, I have reinstalled unraid onto a new USB drive, I have downloaded a new unraid install and setup the server again with same results. 

 

I was thinking it was a hardware issue, so i grabbed a SSD i had sitting around and installed windows 11 on it and booted up the computer into windows and then did the prime95 torture test for multiple hours and no issues. Its driving me crazy at this point!!

 

I see this is system log over and over

 

 

<TASK>
Aug 25 17:40:01 Tower kernel: do_raw_spin_lock+0x14/0x1a
Aug 25 17:40:01 Tower kernel: release_stripe+0x20/0x37 [md_mod]
Aug 25 17:40:01 Tower kernel: unraidd+0x10ce/0x1140 [md_mod]
Aug 25 17:40:01 Tower kernel: md_thread+0xf7/0x122 [md_mod]
Aug 25 17:40:01 Tower kernel: ? _raw_spin_rq_lock_irqsave+0x20/0x20
Aug 25 17:40:01 Tower kernel: ? signal_pending+0x1d/0x1d [md_mod]
Aug 25 17:40:01 Tower kernel: kthread+0xe7/0xef
Aug 25 17:40:01 Tower kernel: ? kthread_complete_and_exit+0x1b/0x1b
Aug 25 17:40:01 Tower kernel: ret_from_fork+0x22/0x30
Aug 25 17:40:01 Tower kernel: </TASK>
Aug 25 17:42:00 Tower root: ACPI action volumeup is not defined
Aug 25 17:42:00 Tower root: ACPI action volumeup is not defined
Aug 25 17:42:04 Tower root: ACPI action up is not defined
Aug 25 17:42:04 Tower root: ACPI action down is not defined
Aug 25 17:43:01 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Aug 25 17:43:01 Tower kernel: rcu: 	6-...0: (14 ticks this GP) idle=5a04/1/0x4000000000000000 softirq=160118/160118 fqs=5402075
Aug 25 17:43:01 Tower kernel: 	(detected by 15, t=12660421 jiffies, g=646893, q=11596026 ncpus=24)
Aug 25 17:43:01 Tower kernel: Sending NMI from CPU 15 to CPUs 6:
Aug 25 17:43:01 Tower kernel: NMI backtrace for cpu 6
Aug 25 17:43:01 Tower kernel: CPU: 6 PID: 4777 Comm: unraidd0 Tainted: P           O       6.1.38-Unraid #2
Aug 25 17:43:01 Tower kernel: Hardware name: Micro-Star International Co., Ltd. MS-7C35/MEG X570 ACE (MS-7C35), BIOS 1.M0 06/28/2023
Aug 25 17:43:01 Tower kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x86/0x1cf
Aug 25 17:43:01 Tower kernel: Code: c2 0f b6 d2 c1 e2 08 30 e4 09 d0 3d ff 00 00 00 76 0c 0f ba e0 08 72 1e c6 43 01 00 eb 18 85 c0 74 0a 8b 03 84 c0 74 04 f3 90 <eb> f6 66 c7 03 01 00 e9 32 01 00 00 e8 4a 3f ff ff 49 c7 c4 80 e1
Aug 25 17:43:01 Tower kernel: RSP: 0018:ffffc90005bb3da0 EFLAGS: 00000002
Aug 25 17:43:01 Tower kernel: RAX: 0000000000000101 RBX: ffff888101542570 RCX: 0000000000000001
Aug 25 17:43:01 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff888101542570
Aug 25 17:43:01 Tower kernel: RBP: ffff888101542570 R08: 0000000000000000 R09: ffff88814e7de530
Aug 25 17:43:01 Tower kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff888101542000
Aug 25 17:43:01 Tower kernel: R13: ffff88814e7de5e0 R14: ffff88814e7de708 R15: ffff8881314d6718
Aug 25 17:43:01 Tower kernel: FS:  0000000000000000(0000) GS:ffff8887fe980000(0000) knlGS:0000000000000000
Aug 25 17:43:01 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug 25 17:43:01 Tower kernel: CR2: 000015127f9ee8c8 CR3: 000000000520a000 CR4: 0000000000350ee0
Aug 25 17:43:01 Tower kernel: Call Trace:
Aug 25 17:43:01 Tower kernel: <NMI>
Aug 25 17:43:01 Tower kernel: ? nmi_cpu_backtrace+0xd3/0x104
Aug 25 17:43:01 Tower kernel: ? nmi_cpu_backtrace_handler+0xd/0x15
Aug 25 17:43:01 Tower kernel: ? nmi_handle+0x57/0x131
Aug 25 17:43:01 Tower kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf
Aug 25 17:43:01 Tower kernel: ? default_do_nmi+0x66/0x15b
Aug 25 17:43:01 Tower kernel: ? exc_nmi+0xbf/0x130
Aug 25 17:43:01 Tower kernel: ? end_repeat_nmi+0x16/0x67
Aug 25 17:43:01 Tower kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf
Aug 25 17:43:01 Tower kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf
Aug 25 17:43:01 Tower kernel: ? native_queued_spin_lock_slowpath+0x86/0x1cf
Aug 25 17:43:01 Tower kernel: </NMI>
Aug 25 17:43:01 Tower kernel: <TASK>

 

tower-diagnostics-20230825-2107.zip

It does look more like a hardware issue, you can try a different Unraid release to rule out any kernel compatibility issues, e.g. v6.11.5, also look for a BIOS update.

I thought the same thing, but the same hardware i have been using for the last 4 years with no major issues as my main desktop. It slowly depreciates over time. It runs great at the startup then over a few hours it gets slower and slower till it no longer functions. I ran a memtest for well over 12hrs and it never locked up and I ran prim95 well over 2hrs on torture mode. 

 

Last night i turned off dockers, shares, and VMS and didn't start the array, it still locked up after couple hours. first the GUI depreciates and network stops working. The console still seems to work until you enter a command then it just grinds to a half and you hold alt ctrl delete it says it shutting down but never does and then i have to hard reset it.

 

It also as given me a kernel panic in the past se attached photo's. I'll attach a syslog and another diagnostic from last night.

 

 

syslog (2) tower-diagnostics-20230826-0740.zip

Edited August 26, 20232 yr by Ctug

At this point I took your advice and downgraded to 6.11.5 and its almost done with parity validation and has been online for nearly 9 hours with no issues.

 

I may have spoken too soon. I am now seeing this again and server is starting to slow down and degrade as its at 87% parity check but now saying its going to take 5 days. 

 

 

Aug 26 18:59:36 Tower kernel: CPU: 18 PID: 355 Comm: unraidd0 Tainted: G      D           5.19.17-Unraid #2
Aug 26 18:59:36 Tower kernel: Hardware name: Micro-Star International Co., Ltd. MS-7C35/MEG X570 ACE (MS-7C35), BIOS 1.M0 06/28/2023
Aug 26 18:59:36 Tower kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x87/0x1d0
Aug 26 18:59:36 Tower kernel: Code: c2 0f b6 d2 c1 e2 08 30 e4 09 d0 3d ff 00 00 00 76 0c 0f ba e0 08 72 1e c6 43 01 00 eb 18 85 c0 74 0a 8b 03 84 c0 74 04 f3 90 <eb> f6 66 c7 03 01 00 e9 32 01 00 00 e8 0f 9d 77 00 49 c7 c4 00 ce
Aug 26 18:59:36 Tower kernel: RSP: 0018:ffffc90003a47da0 EFLAGS: 00000002
Aug 26 18:59:36 Tower kernel: RAX: 0000000000000101 RBX: ffff8881023a5d70 RCX: 0000000000000000
Aug 26 18:59:36 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8881023a5d70
Aug 26 18:59:36 Tower kernel: RBP: ffff8881023a5d70 R08: 0000000000000000 R09: ffffc90003a47d88
Aug 26 18:59:36 Tower kernel: R10: 0000000000000000 R11: 0000000000000002 R12: ffff8881023a5800
Aug 26 18:59:36 Tower kernel: R13: ffff8881587573d8 R14: ffff888158757500 R15: ffff888100e42718
Aug 26 18:59:36 Tower kernel: FS:  0000000000000000(0000) GS:ffff8887fec80000(0000) knlGS:0000000000000000
Aug 26 18:59:36 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug 26 18:59:36 Tower kernel: CR2: 0000148d1ca83548 CR3: 000000000520a000 CR4: 0000000000350ee0
Aug 26 18:59:36 Tower kernel: Call Trace:
Aug 26 18:59:36 Tower kernel: <TASK>
Aug 26 18:59:36 Tower kernel: do_raw_spin_lock+0x14/0x1a
Aug 26 18:59:36 Tower kernel: release_stripe+0x20/0x37 [md_mod]
Aug 26 18:59:36 Tower kernel: unraidd+0x10ce/0x1140 [md_mod]
Aug 26 18:59:36 Tower kernel: md_thread+0x103/0x12e [md_mod]
Aug 26 18:59:36 Tower kernel: ? _raw_spin_rq_lock_irqsave+0x20/0x20
Aug 26 18:59:36 Tower kernel: ? md_seq_show+0x720/0x720 [md_mod]
Aug 26 18:59:36 Tower kernel: kthread+0xe7/0xef
Aug 26 18:59:36 Tower kernel: ? kthread_complete_and_exit+0x1b/0x1b
Aug 26 18:59:36 Tower kernel: ret_from_fork+0x22/0x30
Aug 26 18:59:36 Tower kernel: </TASK>
Aug 26 19:02:36 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Aug 26 19:02:36 Tower kernel: rcu:      18-...0: (11 ticks this GP) idle=511/1/0x4000000000000000 softirq=309203/309204 fqs=518479 
Aug 26 19:02:36 Tower kernel:   (detected by 7, t=2220084 jiffies, g=2671273, q=2318236 ncpus=24)
Aug 26 19:02:36 Tower kernel: Sending NMI from CPU 7 to CPUs 18:
Aug 26 19:02:36 Tower kernel: NMI backtrace for cpu 18
Aug 26 19:02:36 Tower kernel: CPU: 18 PID: 355 Comm: unraidd0 Tainted: G      D           5.19.17-Unraid #2
Aug 26 19:02:36 Tower kernel: Hardware name: Micro-Star International Co., Ltd. MS-7C35/MEG X570 ACE (MS-7C35), BIOS 1.M0 06/28/2023
Aug 26 19:02:36 Tower kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x87/0x1d0
Aug 26 19:02:36 Tower kernel: Code: c2 0f b6 d2 c1 e2 08 30 e4 09 d0 3d ff 00 00 00 76 0c 0f ba e0 08 72 1e c6 43 01 00 eb 18 85 c0 74 0a 8b 03 84 c0 74 04 f3 90 <eb> f6 66 c7 03 01 00 e9 32 01 00 00 e8 0f 9d 77 00 49 c7 c4 00 ce
Aug 26 19:02:36 Tower kernel: RSP: 0018:ffffc90003a47da0 EFLAGS: 00000002
Aug 26 19:02:36 Tower kernel: RAX: 0000000000000101 RBX: ffff8881023a5d70 RCX: 0000000000000000
Aug 26 19:02:36 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8881023a5d70
Aug 26 19:02:36 Tower kernel: RBP: ffff8881023a5d70 R08: 0000000000000000 R09: ffffc90003a47d88
Aug 26 19:02:36 Tower kernel: R10: 0000000000000000 R11: 0000000000000002 R12: ffff8881023a5800
Aug 26 19:02:36 Tower kernel: R13: ffff8881587573d8 R14: ffff888158757500 R15: ffff888100e42718
Aug 26 19:02:36 Tower kernel: FS:  0000000000000000(0000) GS:ffff8887fec80000(0000) knlGS:0000000000000000
Aug 26 19:02:36 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug 26 19:02:36 Tower kernel: CR2: 0000148d1ca83548 CR3: 000000000520a000 CR4: 0000000000350ee0
Aug 26 19:02:36 Tower kernel: Call Trace:
Aug 26 19:02:36 Tower kernel: <TASK>
Aug 26 19:02:36 Tower kernel: do_raw_spin_lock+0x14/0x1a
Aug 26 19:02:36 Tower kernel: release_stripe+0x20/0x37 [md_mod]
Aug 26 19:02:36 Tower kernel: unraidd+0x10ce/0x1140 [md_mod]
Aug 26 19:02:36 Tower kernel: md_thread+0x103/0x12e [md_mod]
Aug 26 19:02:36 Tower kernel: ? _raw_spin_rq_lock_irqsave+0x20/0x20
Aug 26 19:02:36 Tower kernel: ? md_seq_show+0x720/0x720 [md_mod]
Aug 26 19:02:36 Tower kernel: kthread+0xe7/0xef
Aug 26 19:02:36 Tower kernel: ? kthread_complete_and_exit+0x1b/0x1b
Aug 26 19:02:36 Tower kernel: ret_from_fork+0x22/0x30
Aug 26 19:02:36 Tower kernel: </TASK>
Aug 26 19:04:15 Tower  apcupsd[3774]: Communications with UPS lost.
Aug 26 19:05:36 Tower kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Aug 26 19:05:36 Tower kernel: rcu:      18-...0: (11 ticks this GP) idle=511/1/0x4000000000000000 softirq=309203/309204 fqs=560711 
Aug 26 19:05:36 Tower kernel:   (detected by 7, t=2400089 jiffies, g=2671273, q=2541172 ncpus=24)
Aug 26 19:05:36 Tower kernel: Sending NMI from CPU 7 to CPUs 18:
Aug 26 19:05:36 Tower kernel: NMI backtrace for cpu 18
Aug 26 19:05:36 Tower kernel: CPU: 18 PID: 355 Comm: unraidd0 Tainted: G      D           5.19.17-Unraid #2
Aug 26 19:05:36 Tower kernel: Hardware name: Micro-Star International Co., Ltd. MS-7C35/MEG X570 ACE (MS-7C35), BIOS 1.M0 06/28/2023
Aug 26 19:05:36 Tower kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x87/0x1d0
Aug 26 19:05:36 Tower kernel: Code: c2 0f b6 d2 c1 e2 08 30 e4 09 d0 3d ff 00 00 00 76 0c 0f ba e0 08 72 1e c6 43 01 00 eb 18 85 c0 74 0a 8b 03 84 c0 74 04 f3 90 <eb> f6 66 c7 03 01 00 e9 32 01 00 00 e8 0f 9d 77 00 49 c7 c4 00 ce
Aug 26 19:05:36 Tower kernel: RSP: 0018:ffffc90003a47da0 EFLAGS: 00000002
Aug 26 19:05:36 Tower kernel: RAX: 0000000000000101 RBX: ffff8881023a5d70 RCX: 0000000000000000
Aug 26 19:05:36 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8881023a5d70
Aug 26 19:05:36 Tower kernel: RBP: ffff8881023a5d70 R08: 0000000000000000 R09: ffffc90003a47d88
Aug 26 19:05:36 Tower kernel: R10: 0000000000000000 R11: 0000000000000002 R12: ffff8881023a5800
Aug 26 19:05:36 Tower kernel: R13: ffff8881587573d8 R14: ffff888158757500 R15: ffff888100e42718
Aug 26 19:05:36 Tower kernel: FS:  0000000000000000(0000) GS:ffff8887fec80000(0000) knlGS:0000000000000000
Aug 26 19:05:36 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug 26 19:05:36 Tower kernel: CR2: 0000148d1ca83548 CR3: 000000000520a000 CR4: 0000000000350ee0
Aug 26 19:05:36 Tower kernel: Call Trace:

 

Edited August 26, 20232 yr by Ctug

Unraid driver still crashing, since it happens with two different kernels it's not a specif kernel issue, possibly it's still a hardware issue or the board just doesn't work well with Linux, did you look for a BIOS update?

I just updated the BIOS before posting this, didn't seem to help. Today I noticed that two CPU Threads were 100% and stuck as the server got slower and slower and slower. 

I've problems with my server after upgrade to b650 & 7600x. Before I replace board, cpu and mem in server I make 2-3 days of testing setting on windows - perfectly stable.

 

After change - unraid restarts itself at random occasion, after messing with bios settings (C-State, other PowerSaving, memory settings), reverting back to stock I still get restarts until I disable ReBAR and 4G decoder. 

 

After that no more restarts.

Hey Raptor,

 

I finally fixed me server by following this guide in FAQ, I should have started there but didn't know!!

 

What can I do to keep my Ryzen based server from crashing/locking up with Unraid?

 

After disabling C-states globally in bios my server has been online for almost 24hrs. The parity has completed and i have now moved it back into the basement where it belongs!!!

 

You may try those two options in the FAQ.

I've try disable C-States and Power Supply Idle Control - didn't work.

I would suggest starting your own thread with sys logs and diagnostic data uploaded!

53 minutes ago, Ctug said:

I should have started there but didn't know!!

Well, I also should have mentioned that, but your issues looked more serious than the usually issues caused by that, but glad it worked.

So i have gotten it to stop crashing for the most part, I am now seeing this. I still think its something to do with one of the two NIC cards on this motherboard, It has a realtek 2.5GBS and a Intel NIC, I have it crash my entire network multiple times over the last 5 days, Everything stops working and as soon as I unplug the NIC on the unraid server it all starts working again. 

 

The NIC's performed flawlessly for 4.5 years as my main desktop.

 

 

Aug 30 10:34:14 Tower kernel: WARNING: CPU: 19 PID: 19606 at net/netfilter/nf_conntrack_core.c:1210 __nf_conntrack_confirm+0xa4/0x2b0 [nf_conntrack]
Aug 30 10:34:14 Tower kernel: Modules linked in: macvlan md_mod udp_diag xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_iotlb xt_nat xt_tcpudp ipvlan xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype br_netfilter xfs zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) tcp_diag inet_diag ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs bonding tls bridge stp llc igb i2c_algo_bit r8169 realtek edac_mce_amd edac_core intel_rapl_msr intel_rapl_common iosf_mbi crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel sha512_ssse3 aesni_intel btusb btrtl btbcm crypto_simd btintel cryptd wmi_bmof mxm_wmi bluetooth rapl joydev k10temp ahci nvme i2c_piix4 libahci input_leds ecdh_generic led_class nvme_core i2c_core ecc tpm_crb tpm_tis tpm_tis_core tpm wmi button acpi_cpufreq unix [last unloaded: tun]
Aug 30 10:34:14 Tower kernel: CPU: 19 PID: 19606 Comm: kworker/u64:4 Tainted: P           O       6.1.38-Unraid #2
Aug 30 10:34:14 Tower kernel: Hardware name: Micro-Star International Co., Ltd. MS-7C35/MEG X570 ACE (MS-7C35), BIOS 1.M0 06/28/2023
Aug 30 10:34:14 Tower kernel: Workqueue: events_unbound macvlan_process_broadcast [macvlan]
Aug 30 10:34:14 Tower kernel: RIP: 0010:__nf_conntrack_confirm+0xa4/0x2b0 [nf_conntrack]
Aug 30 10:34:14 Tower kernel: Code: 44 24 10 e8 e2 e1 ff ff 8b 7c 24 04 89 ea 89 c6 89 04 24 e8 7e e6 ff ff 84 c0 75 a2 48 89 df e8 9b e2 ff ff 85 c0 89 c5 74 18 <0f> 0b 8b 34 24 8b 7c 24 04 e8 18 dd ff ff e8 93 e3 ff ff e9 72 01
Aug 30 10:34:14 Tower kernel: RSP: 0018:ffffc9000063cd98 EFLAGS: 00010202
Aug 30 10:34:14 Tower kernel: RAX: 0000000000000001 RBX: ffff8881ee550700 RCX: 5f4866f83266a484
Aug 30 10:34:14 Tower kernel: RDX: 0000000000000000 RSI: 00000000000002b8 RDI: ffff8881ee550700
Aug 30 10:34:14 Tower kernel: RBP: 0000000000000001 R08: 011a3107fcb2396c R09: 627969c67e2cc09a
Aug 30 10:34:14 Tower kernel: R10: 2eac412dd25da258 R11: ffffc9000063cd60 R12: ffffffff82a11d00
Aug 30 10:34:14 Tower kernel: R13: 0000000000016972 R14: ffff8881013d4e00 R15: 0000000000000000
Aug 30 10:34:14 Tower kernel: FS:  0000000000000000(0000) GS:ffff8887fecc0000(0000) knlGS:0000000000000000
Aug 30 10:34:14 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug 30 10:34:14 Tower kernel: CR2: 000014e5a33dc484 CR3: 000000015c654000 CR4: 0000000000350ee0
Aug 30 10:34:14 Tower kernel: Call Trace:
Aug 30 10:34:14 Tower kernel: <IRQ>
Aug 30 10:34:14 Tower kernel: ? __warn+0xab/0x122
Aug 30 10:34:14 Tower kernel: ? report_bug+0x109/0x17e
Aug 30 10:34:14 Tower kernel: ? __nf_conntrack_confirm+0xa4/0x2b0 [nf_conntrack]
Aug 30 10:34:14 Tower kernel: ? handle_bug+0x41/0x6f
Aug 30 10:34:14 Tower kernel: ? exc_invalid_op+0x13/0x60
Aug 30 10:34:14 Tower kernel: ? asm_exc_invalid_op+0x16/0x20
Aug 30 10:34:14 Tower kernel: ? __nf_conntrack_confirm+0xa4/0x2b0 [nf_conntrack]
Aug 30 10:34:14 Tower kernel: ? __nf_conntrack_confirm+0x9e/0x2b0 [nf_conntrack]
Aug 30 10:34:14 Tower kernel: ? nf_nat_inet_fn+0x126/0x1a8 [nf_nat]
Aug 30 10:34:14 Tower kernel: nf_conntrack_confirm+0x25/0x54 [nf_conntrack]
Aug 30 10:34:14 Tower kernel: nf_hook_slow+0x3d/0x96
Aug 30 10:34:14 Tower kernel: ? ip_protocol_deliver_rcu+0x164/0x164
Aug 30 10:34:14 Tower kernel: NF_HOOK.constprop.0+0x79/0xd9
Aug 30 10:34:14 Tower kernel: ? ip_protocol_deliver_rcu+0x164/0x164
Aug 30 10:34:14 Tower kernel: __netif_receive_skb_one_core+0x77/0x9c
Aug 30 10:34:14 Tower kernel: process_backlog+0x8c/0x116
Aug 30 10:34:14 Tower kernel: __napi_poll.constprop.0+0x2b/0x124
Aug 30 10:34:14 Tower kernel: net_rx_action+0x159/0x24f
Aug 30 10:34:14 Tower kernel: __do_softirq+0x129/0x288
Aug 30 10:34:14 Tower kernel: do_softirq+0x7f/0xab
Aug 30 10:34:14 Tower kernel: </IRQ>
Aug 30 10:34:14 Tower kernel: <TASK>
Aug 30 10:34:14 Tower kernel: __local_bh_enable_ip+0x4c/0x6b
Aug 30 10:34:14 Tower kernel: netif_rx+0x52/0x5a
Aug 30 10:34:14 Tower kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Aug 30 10:34:14 Tower kernel: ? _raw_spin_unlock+0x14/0x29
Aug 30 10:34:14 Tower kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]
Aug 30 10:34:14 Tower kernel: process_one_work+0x1ab/0x295
Aug 30 10:34:14 Tower kernel: worker_thread+0x18b/0x244
Aug 30 10:34:14 Tower kernel: ? rescuer_thread+0x281/0x281
Aug 30 10:34:14 Tower kernel: kthread+0xe7/0xef
Aug 30 10:34:14 Tower kernel: ? kthread_complete_and_exit+0x1b/0x1b
Aug 30 10:34:14 Tower kernel: ret_from_fork+0x22/0x30
Aug 30 10:34:14 Tower kernel: </TASK>
Aug 30 10:34:14 Tower kernel: ---[ end trace 0000000000000000 ]---

 

tower-diagnostics-20230830-1529.zip

1 hour ago, Ctug said:

Aug 30 10:34:14 Tower kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Aug 30 10:34:14 Tower kernel: ? _raw_spin_unlock+0x14/0x29
Aug 30 10:34:14 Tower kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

This is a different known issue, switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)).

I switched over hopefully it stays stable!!!!

Still experiencing issues. About to pull my hair out, I ran memtest86 everything passed... 

 

syslog

 

Ive enalbed IPvlan and still freezing

tower-diagnostics-20230904-0901.zip

There are constant call traces logged, suggesting a hardware issue, did you take care of this?

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=819173

 

 

I've disabled C-states and XMP is currently off.

I've ran memtest now for over 24hrs with 16 passes and no errors. Any other suggestions on something I can use to further test the system?

One thing you can try is to boot the server in safe mode with all docker/VMs disabled, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one. 

So I swapped PSU's I will update how it goes, It is only part that has changed since moving it to unraid box. Seems odd for it to cause a kernel panic but I've see weirder shit.

Still crashing at this point. Super Frustrating!!!

Change the network type from macvlan to ipvlan? It was causing kernal panics.

So I've picked up a i7-7700k build, unraid runs fine on that. I took the 3900x build and installed windows and have been running prime 95 for 15 hours on torture not a single crash.