Hello All,
I have been having issues since going to 6.7+ with the server only staying up for roughly 24-48 hours then hanging. I have to hard reboot everytime and it doesn't display any errors on screen. I am seeing the following in the logs, anyone have thoughts?
t 1 04:08:25 utgard01 kernel: WARNING: CPU: 6 PID: 109 at net/netfilter/nf_conntrack_core.c:945 __nf_conntrack_confirm+0x97/0x686
Oct 1 04:08:25 utgard01 kernel: Modules linked in: xt_nat xt_CHECKSUM ipt_REJECT macvlan ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap ipt_MASQUERADE iptable_nat nf_nat_ipv4 iptable_filter ip_tables nf_nat xfs nfsd lockd grace sunrpc md_mod bonding bnx2 sr_mod cdrom ipmi_ssif i2c_core intel_powerclamp coretemp kvm_intel kvm crc32c_intel mpt3sas acpi_power_meter intel_cstate intel_uncore ipmi_si button ata_piix i7core_edac raid_class scsi_transport_sas [last unloaded: bnx2]
Oct 1 04:08:25 utgard01 kernel: CPU: 6 PID: 109 Comm: kworker/6:1 Tainted: G I 4.19.56-Unraid #1
Oct 1 04:08:25 utgard01 kernel: Hardware name: Dell Inc. PowerEdge R710/00W9X3, BIOS 3.0.0 01/31/2011
Oct 1 04:08:25 utgard01 kernel: Workqueue: events macvlan_process_broadcast [macvlan]
Oct 1 04:08:25 utgard01 kernel: RIP: 0010:__nf_conntrack_confirm+0x97/0x686
Oct 1 04:08:25 utgard01 kernel: Code: c1 ed 20 89 2c 24 e8 67 fb ff ff 8b 54 24 04 89 ef 89 c6 41 89 c4 e8 29 f9 ff ff 84 c0 75 b9 49 8b 86 80 00 00 00 a8 08 74 25 <0f> 0b 44 89 e6 89 ef 45 31 ff e8 b2 f1 ff ff be 00 02 00 00 48 c7
Oct 1 04:08:25 utgard01 kernel: RSP: 0018:ffff888e63ac3d98 EFLAGS: 00010202
Oct 1 04:08:25 utgard01 kernel: RAX: 0000000000000188 RBX: ffff888e5c7e0a00 RCX: 0000000067c4b2fb
Oct 1 04:08:25 utgard01 kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffff81e093c0
Oct 1 04:08:25 utgard01 kernel: RBP: 0000000000009370 R08: ffff88870ce367f0 R09: 0000000092501b2e
Oct 1 04:08:25 utgard01 kernel: R10: 0000000000000000 R11: ffff888e3d0b9401 R12: 0000000000002103
Oct 1 04:08:25 utgard01 kernel: R13: ffffffff81e8e080 R14: ffff88870ce36780 R15: ffff88870ce367d8
Oct 1 04:08:25 utgard01 kernel: FS: 0000000000000000(0000) GS:ffff888e63ac0000(0000) knlGS:0000000000000000
Oct 1 04:08:25 utgard01 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 1 04:08:25 utgard01 kernel: CR2: 00001472d8447800 CR3: 0000000001e0a000 CR4: 00000000000006e0
Oct 1 04:08:25 utgard01 kernel: Call Trace:
Oct 1 04:08:25 utgard01 kernel: <IRQ>
Oct 1 04:08:25 utgard01 kernel: ipv4_confirm+0xaf/0xb7
Oct 1 04:08:25 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct 1 04:08:25 utgard01 kernel: ip_local_deliver+0xa9/0xd7
Oct 1 04:08:25 utgard01 kernel: ? ip_sublist_rcv_finish+0x53/0x53
Oct 1 04:08:25 utgard01 kernel: ip_rcv+0xa0/0xbe
Oct 1 04:08:25 utgard01 kernel: ? ip_rcv_finish_core.isra.0+0x2e2/0x2e2
Oct 1 04:08:25 utgard01 kernel: __netif_receive_skb_one_core+0x4d/0x69
Oct 1 04:08:25 utgard01 kernel: process_backlog+0x7c/0x116
Oct 1 04:08:25 utgard01 kernel: net_rx_action+0x10b/0x274
Oct 1 04:08:25 utgard01 kernel: __do_softirq+0xce/0x1e2
Oct 1 04:08:25 utgard01 kernel: do_softirq_own_stack+0x2a/0x40
Oct 1 04:08:25 utgard01 kernel: </IRQ>
Oct 1 04:08:25 utgard01 kernel: do_softirq+0x4d/0x59
Oct 1 04:08:25 utgard01 kernel: netif_rx_ni+0x1c/0x22
Oct 1 04:08:25 utgard01 kernel: macvlan_broadcast+0x10f/0x153 [macvlan]
Oct 1 04:08:25 utgard01 kernel: ? __switch_to_asm+0x41/0x70
Oct 1 04:08:25 utgard01 kernel: macvlan_process_broadcast+0xeb/0x128 [macvlan]
Oct 1 04:08:25 utgard01 kernel: process_one_work+0x16e/0x24f
Oct 1 04:08:25 utgard01 kernel: ? pwq_unbound_release_workfn+0xb7/0xb7
Oct 1 04:08:25 utgard01 kernel: worker_thread+0x1dc/0x2ac
Oct 1 04:08:25 utgard01 kernel: kthread+0x10b/0x113
Oct 1 04:08:25 utgard01 kernel: ? kthread_park+0x71/0x71
Oct 1 04:08:25 utgard01 kernel: ret_from_fork+0x35/0x40
Oct 1 04:08:25 utgard01 kernel: ---[ end trace 3e440b525ec08ddd ]---
Oct 1 04:09:54 utgard01 ntpd[1864]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
Oct 1 08:10:48 utgard01 kernel: WARNING: CPU: 9 PID: 0 at net/netfilter/nf_nat_core.c:420 nf_nat_setup_info+0x6b/0x5fb [nf_nat]
Oct 1 08:10:48 utgard01 kernel: Modules linked in: xt_nat xt_CHECKSUM ipt_REJECT macvlan ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap ipt_MASQUERADE iptable_nat nf_nat_ipv4 iptable_filter ip_tables nf_nat xfs nfsd lockd grace sunrpc md_mod bonding bnx2 sr_mod cdrom ipmi_ssif i2c_core intel_powerclamp coretemp kvm_intel kvm crc32c_intel mpt3sas acpi_power_meter intel_cstate intel_uncore ipmi_si button ata_piix i7core_edac raid_class scsi_transport_sas [last unloaded: bnx2]
Oct 1 08:10:48 utgard01 kernel: CPU: 9 PID: 0 Comm: swapper/9 Tainted: G W I 4.19.56-Unraid #1
Oct 1 08:10:48 utgard01 kernel: Hardware name: Dell Inc. PowerEdge R710/00W9X3, BIOS 3.0.0 01/31/2011
Oct 1 08:10:48 utgard01 kernel: RIP: 0010:nf_nat_setup_info+0x6b/0x5fb [nf_nat]
Oct 1 08:10:48 utgard01 kernel: Code: 48 89 fb 48 8b 87 80 00 00 00 49 89 f7 41 89 d6 76 04 0f 0b eb 0b 85 d2 75 07 25 80 00 00 00 eb 05 25 00 01 00 00 85 c0 74 07 <0f> 0b e9 ac 04 00 00 48 8b 83 90 00 00 00 4c 8d 64 24 30 48 8d 73
Oct 1 08:10:48 utgard01 kernel: RSP: 0018:ffff888763b03790 EFLAGS: 00010202
Oct 1 08:10:48 utgard01 kernel: RAX: 0000000000000080 RBX: ffff8886ed657540 RCX: 0000000000000000
Oct 1 08:10:48 utgard01 kernel: RDX: 0000000000000000 RSI: ffff888763b0387c RDI: ffff8886ed657540
Oct 1 08:10:48 utgard01 kernel: RBP: ffff888763b03868 R08: ffff8886ed657540 R09: ffff888e5c743000
Oct 1 08:10:48 utgard01 kernel: R10: 0000000000000158 R11: ffffffff81e8e001 R12: ffff88875ddcf600
Oct 1 08:10:48 utgard01 kernel: R13: 0000000000000000 R14: 0000000000000000 R15: ffff888763b0387c
Oct 1 08:10:48 utgard01 kernel: FS: 0000000000000000(0000) GS:ffff888763b00000(0000) knlGS:0000000000000000
Oct 1 08:10:48 utgard01 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 1 08:10:48 utgard01 kernel: CR2: 000014f508897000 CR3: 0000000001e0a000 CR4: 00000000000006e0
Oct 1 08:10:48 utgard01 kernel: Call Trace:
Oct 1 08:10:48 utgard01 kernel: <IRQ>
Oct 1 08:10:48 utgard01 kernel: ? fib_validate_source+0xaf/0xd6
Oct 1 08:10:48 utgard01 kernel: ? ipt_do_table+0x58e/0x5db [ip_tables]
Oct 1 08:10:48 utgard01 kernel: nf_nat_alloc_null_binding+0x6f/0x86 [nf_nat]
Oct 1 08:10:48 utgard01 kernel: nf_nat_inet_fn+0xa0/0x192 [nf_nat]
Oct 1 08:10:48 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct 1 08:10:48 utgard01 kernel: ip_local_deliver+0xa9/0xd7
Oct 1 08:10:48 utgard01 kernel: ? ip_sublist_rcv_finish+0x53/0x53
Oct 1 08:10:48 utgard01 kernel: ip_sabotage_in+0x38/0x3e
Oct 1 08:10:48 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct 1 08:10:48 utgard01 kernel: ip_rcv+0x8e/0xbe
Oct 1 08:10:48 utgard01 kernel: ? ip_rcv_finish_core.isra.0+0x2e2/0x2e2
Oct 1 08:10:48 utgard01 kernel: __netif_receive_skb_one_core+0x4d/0x69
Oct 1 08:10:48 utgard01 kernel: netif_receive_skb_internal+0x9f/0xba
Oct 1 08:10:48 utgard01 kernel: br_pass_frame_up+0x123/0x145
Oct 1 08:10:48 utgard01 kernel: ? br_port_flags_change+0x29/0x29
Oct 1 08:10:48 utgard01 kernel: br_handle_frame_finish+0x330/0x375
Oct 1 08:10:48 utgard01 kernel: ? ipt_do_table+0x58e/0x5db [ip_tables]
Oct 1 08:10:48 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct 1 08:10:48 utgard01 kernel: br_nf_hook_thresh+0xa3/0xc3
Oct 1 08:10:48 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct 1 08:10:48 utgard01 kernel: br_nf_pre_routing_finish+0x239/0x260
Oct 1 08:10:48 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct 1 08:10:48 utgard01 kernel: ? nf_nat_ipv4_in+0x1d/0x64 [nf_nat_ipv4]
Oct 1 08:10:48 utgard01 kernel: br_nf_pre_routing+0x2fc/0x321
Oct 1 08:10:48 utgard01 kernel: ? br_nf_forward_ip+0x352/0x352
Oct 1 08:10:48 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct 1 08:10:48 utgard01 kernel: br_handle_frame+0x290/0x2d3
Oct 1 08:10:48 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct 1 08:10:48 utgard01 kernel: ? br_handle_local_finish+0xe/0xe
Oct 1 08:10:48 utgard01 kernel: __netif_receive_skb_core+0x466/0x798
Oct 1 08:10:48 utgard01 kernel: ? udp_gro_receive+0x4c/0x134
Oct 1 08:10:48 utgard01 kernel: __netif_receive_skb_one_core+0x31/0x69
Oct 1 08:10:48 utgard01 kernel: netif_receive_skb_internal+0x9f/0xba
Oct 1 08:10:48 utgard01 kernel: napi_gro_receive+0x42/0x76
Oct 1 08:10:48 utgard01 kernel: bnx2_poll_work+0xf83/0x1059 [bnx2]
Oct 1 08:10:48 utgard01 kernel: bnx2_poll_msix+0x2e/0xb1 [bnx2]
Oct 1 08:10:48 utgard01 kernel: net_rx_action+0x10b/0x274
Oct 1 08:10:48 utgard01 kernel: __do_softirq+0xce/0x1e2
Oct 1 08:10:48 utgard01 kernel: irq_exit+0x5e/0x9d
Oct 1 08:10:48 utgard01 kernel: do_IRQ+0xa9/0xc7
Oct 1 08:10:48 utgard01 kernel: common_interrupt+0xf/0xf
Oct 1 08:10:48 utgard01 kernel: </IRQ>
Oct 1 08:10:48 utgard01 kernel: RIP: 0010:cpuidle_enter_state+0xe5/0x141
Oct 1 08:10:48 utgard01 kernel: Code: 97 7e ba ff 45 84 ff 74 1d 9c 58 66 66 90 66 90 0f ba e0 09 73 09 0f 0b fa 66 66 90 66 66 90 31 ff e8 ae 0c be ff fb 66 66 90 <66> 66 90 48 2b 1c 24 b8 ff ff ff 7f 48 b9 ff ff ff ff f3 01 00 00
Oct 1 08:10:48 utgard01 kernel: RSP: 0018:ffffc900062fbea0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffdc
Oct 1 08:10:48 utgard01 kernel: RAX: ffff888763b20b00 RBX: 00000d82f12e87bb RCX: 000000000000001f
Oct 1 08:10:48 utgard01 kernel: RDX: 00000d82f12e87bb RSI: 0000000035778ed4 RDI: 0000000000000000
Oct 1 08:10:48 utgard01 kernel: RBP: ffff888763b2bb00 R08: 0000000000000002 R09: 00000000000203c0
Oct 1 08:10:48 utgard01 kernel: R10: 00000000014d4b84 R11: 000020a373b2a030 R12: 0000000000000004
Oct 1 08:10:48 utgard01 kernel: R13: 0000000000000004 R14: ffffffff81e5a018 R15: 0000000000000000
Oct 1 08:10:48 utgard01 kernel: do_idle+0x192/0x20e
Oct 1 08:10:48 utgard01 kernel: cpu_startup_entry+0x6a/0x6c
Oct 1 08:10:48 utgard01 kernel: start_secondary+0x197/0x1b2
Oct 1 08:10:48 utgard01 kernel: secondary_startup_64+0xa4/0xb0
Oct 1 08:10:48 utgard01 kernel: ---[ end trace 3e440b525ec08dde ]---
Oct 1 19:57:11 utgard01 kernel: md: sync done. time=57148sec
Oct 1 19:57:11 utgard01 kernel: md: recovery thread: exit status: 0