Server keeps crashing

em1917

By em1917
in General Support

Recommended Posts

em1917 Noob

em1917

Posted
Posted

Hello All, 

 

I have been having issues since going to 6.7+ with the server only staying up for roughly 24-48 hours then hanging. I have to hard reboot everytime and it doesn't display any errors on screen. I am seeing the following in the logs, anyone have thoughts?

  

t  1 04:08:25 utgard01 kernel: WARNING: CPU: 6 PID: 109 at net/netfilter/nf_conntrack_core.c:945 __nf_conntrack_confirm+0x97/0x686
Oct  1 04:08:25 utgard01 kernel: Modules linked in: xt_nat xt_CHECKSUM ipt_REJECT macvlan ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap ipt_MASQUERADE iptable_nat nf_nat_ipv4 iptable_filter ip_tables nf_nat xfs nfsd lockd grace sunrpc md_mod bonding bnx2 sr_mod cdrom ipmi_ssif i2c_core intel_powerclamp coretemp kvm_intel kvm crc32c_intel mpt3sas acpi_power_meter intel_cstate intel_uncore ipmi_si button ata_piix i7core_edac raid_class scsi_transport_sas [last unloaded: bnx2]
Oct  1 04:08:25 utgard01 kernel: CPU: 6 PID: 109 Comm: kworker/6:1 Tainted: G          I       4.19.56-Unraid #1
Oct  1 04:08:25 utgard01 kernel: Hardware name: Dell Inc. PowerEdge R710/00W9X3, BIOS 3.0.0 01/31/2011
Oct  1 04:08:25 utgard01 kernel: Workqueue: events macvlan_process_broadcast [macvlan]
Oct  1 04:08:25 utgard01 kernel: RIP: 0010:__nf_conntrack_confirm+0x97/0x686
Oct  1 04:08:25 utgard01 kernel: Code: c1 ed 20 89 2c 24 e8 67 fb ff ff 8b 54 24 04 89 ef 89 c6 41 89 c4 e8 29 f9 ff ff 84 c0 75 b9 49 8b 86 80 00 00 00 a8 08 74 25 <0f> 0b 44 89 e6 89 ef 45 31 ff e8 b2 f1 ff ff be 00 02 00 00 48 c7
Oct  1 04:08:25 utgard01 kernel: RSP: 0018:ffff888e63ac3d98 EFLAGS: 00010202
Oct  1 04:08:25 utgard01 kernel: RAX: 0000000000000188 RBX: ffff888e5c7e0a00 RCX: 0000000067c4b2fb
Oct  1 04:08:25 utgard01 kernel: RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffffffff81e093c0
Oct  1 04:08:25 utgard01 kernel: RBP: 0000000000009370 R08: ffff88870ce367f0 R09: 0000000092501b2e
Oct  1 04:08:25 utgard01 kernel: R10: 0000000000000000 R11: ffff888e3d0b9401 R12: 0000000000002103
Oct  1 04:08:25 utgard01 kernel: R13: ffffffff81e8e080 R14: ffff88870ce36780 R15: ffff88870ce367d8
Oct  1 04:08:25 utgard01 kernel: FS:  0000000000000000(0000) GS:ffff888e63ac0000(0000) knlGS:0000000000000000
Oct  1 04:08:25 utgard01 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct  1 04:08:25 utgard01 kernel: CR2: 00001472d8447800 CR3: 0000000001e0a000 CR4: 00000000000006e0
Oct  1 04:08:25 utgard01 kernel: Call Trace:
Oct  1 04:08:25 utgard01 kernel: <IRQ>
Oct  1 04:08:25 utgard01 kernel: ipv4_confirm+0xaf/0xb7
Oct  1 04:08:25 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct  1 04:08:25 utgard01 kernel: ip_local_deliver+0xa9/0xd7
Oct  1 04:08:25 utgard01 kernel: ? ip_sublist_rcv_finish+0x53/0x53
Oct  1 04:08:25 utgard01 kernel: ip_rcv+0xa0/0xbe
Oct  1 04:08:25 utgard01 kernel: ? ip_rcv_finish_core.isra.0+0x2e2/0x2e2
Oct  1 04:08:25 utgard01 kernel: __netif_receive_skb_one_core+0x4d/0x69
Oct  1 04:08:25 utgard01 kernel: process_backlog+0x7c/0x116
Oct  1 04:08:25 utgard01 kernel: net_rx_action+0x10b/0x274
Oct  1 04:08:25 utgard01 kernel: __do_softirq+0xce/0x1e2
Oct  1 04:08:25 utgard01 kernel: do_softirq_own_stack+0x2a/0x40
Oct  1 04:08:25 utgard01 kernel: </IRQ>
Oct  1 04:08:25 utgard01 kernel: do_softirq+0x4d/0x59
Oct  1 04:08:25 utgard01 kernel: netif_rx_ni+0x1c/0x22
Oct  1 04:08:25 utgard01 kernel: macvlan_broadcast+0x10f/0x153 [macvlan]
Oct  1 04:08:25 utgard01 kernel: ? __switch_to_asm+0x41/0x70
Oct  1 04:08:25 utgard01 kernel: macvlan_process_broadcast+0xeb/0x128 [macvlan]
Oct  1 04:08:25 utgard01 kernel: process_one_work+0x16e/0x24f
Oct  1 04:08:25 utgard01 kernel: ? pwq_unbound_release_workfn+0xb7/0xb7
Oct  1 04:08:25 utgard01 kernel: worker_thread+0x1dc/0x2ac
Oct  1 04:08:25 utgard01 kernel: kthread+0x10b/0x113
Oct  1 04:08:25 utgard01 kernel: ? kthread_park+0x71/0x71
Oct  1 04:08:25 utgard01 kernel: ret_from_fork+0x35/0x40
Oct  1 04:08:25 utgard01 kernel: ---[ end trace 3e440b525ec08ddd ]---
Oct  1 04:09:54 utgard01 ntpd[1864]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
Oct  1 08:10:48 utgard01 kernel: WARNING: CPU: 9 PID: 0 at net/netfilter/nf_nat_core.c:420 nf_nat_setup_info+0x6b/0x5fb [nf_nat]
Oct  1 08:10:48 utgard01 kernel: Modules linked in: xt_nat xt_CHECKSUM ipt_REJECT macvlan ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap ipt_MASQUERADE iptable_nat nf_nat_ipv4 iptable_filter ip_tables nf_nat xfs nfsd lockd grace sunrpc md_mod bonding bnx2 sr_mod cdrom ipmi_ssif i2c_core intel_powerclamp coretemp kvm_intel kvm crc32c_intel mpt3sas acpi_power_meter intel_cstate intel_uncore ipmi_si button ata_piix i7core_edac raid_class scsi_transport_sas [last unloaded: bnx2]
Oct  1 08:10:48 utgard01 kernel: CPU: 9 PID: 0 Comm: swapper/9 Tainted: G        W I       4.19.56-Unraid #1
Oct  1 08:10:48 utgard01 kernel: Hardware name: Dell Inc. PowerEdge R710/00W9X3, BIOS 3.0.0 01/31/2011
Oct  1 08:10:48 utgard01 kernel: RIP: 0010:nf_nat_setup_info+0x6b/0x5fb [nf_nat]
Oct  1 08:10:48 utgard01 kernel: Code: 48 89 fb 48 8b 87 80 00 00 00 49 89 f7 41 89 d6 76 04 0f 0b eb 0b 85 d2 75 07 25 80 00 00 00 eb 05 25 00 01 00 00 85 c0 74 07 <0f> 0b e9 ac 04 00 00 48 8b 83 90 00 00 00 4c 8d 64 24 30 48 8d 73
Oct  1 08:10:48 utgard01 kernel: RSP: 0018:ffff888763b03790 EFLAGS: 00010202
Oct  1 08:10:48 utgard01 kernel: RAX: 0000000000000080 RBX: ffff8886ed657540 RCX: 0000000000000000
Oct  1 08:10:48 utgard01 kernel: RDX: 0000000000000000 RSI: ffff888763b0387c RDI: ffff8886ed657540
Oct  1 08:10:48 utgard01 kernel: RBP: ffff888763b03868 R08: ffff8886ed657540 R09: ffff888e5c743000
Oct  1 08:10:48 utgard01 kernel: R10: 0000000000000158 R11: ffffffff81e8e001 R12: ffff88875ddcf600
Oct  1 08:10:48 utgard01 kernel: R13: 0000000000000000 R14: 0000000000000000 R15: ffff888763b0387c
Oct  1 08:10:48 utgard01 kernel: FS:  0000000000000000(0000) GS:ffff888763b00000(0000) knlGS:0000000000000000
Oct  1 08:10:48 utgard01 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct  1 08:10:48 utgard01 kernel: CR2: 000014f508897000 CR3: 0000000001e0a000 CR4: 00000000000006e0
Oct  1 08:10:48 utgard01 kernel: Call Trace:
Oct  1 08:10:48 utgard01 kernel: <IRQ>
Oct  1 08:10:48 utgard01 kernel: ? fib_validate_source+0xaf/0xd6
Oct  1 08:10:48 utgard01 kernel: ? ipt_do_table+0x58e/0x5db [ip_tables]
Oct  1 08:10:48 utgard01 kernel: nf_nat_alloc_null_binding+0x6f/0x86 [nf_nat]
Oct  1 08:10:48 utgard01 kernel: nf_nat_inet_fn+0xa0/0x192 [nf_nat]
Oct  1 08:10:48 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct  1 08:10:48 utgard01 kernel: ip_local_deliver+0xa9/0xd7
Oct  1 08:10:48 utgard01 kernel: ? ip_sublist_rcv_finish+0x53/0x53
Oct  1 08:10:48 utgard01 kernel: ip_sabotage_in+0x38/0x3e
Oct  1 08:10:48 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct  1 08:10:48 utgard01 kernel: ip_rcv+0x8e/0xbe
Oct  1 08:10:48 utgard01 kernel: ? ip_rcv_finish_core.isra.0+0x2e2/0x2e2
Oct  1 08:10:48 utgard01 kernel: __netif_receive_skb_one_core+0x4d/0x69
Oct  1 08:10:48 utgard01 kernel: netif_receive_skb_internal+0x9f/0xba
Oct  1 08:10:48 utgard01 kernel: br_pass_frame_up+0x123/0x145
Oct  1 08:10:48 utgard01 kernel: ? br_port_flags_change+0x29/0x29
Oct  1 08:10:48 utgard01 kernel: br_handle_frame_finish+0x330/0x375
Oct  1 08:10:48 utgard01 kernel: ? ipt_do_table+0x58e/0x5db [ip_tables]
Oct  1 08:10:48 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct  1 08:10:48 utgard01 kernel: br_nf_hook_thresh+0xa3/0xc3
Oct  1 08:10:48 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct  1 08:10:48 utgard01 kernel: br_nf_pre_routing_finish+0x239/0x260
Oct  1 08:10:48 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct  1 08:10:48 utgard01 kernel: ? nf_nat_ipv4_in+0x1d/0x64 [nf_nat_ipv4]
Oct  1 08:10:48 utgard01 kernel: br_nf_pre_routing+0x2fc/0x321
Oct  1 08:10:48 utgard01 kernel: ? br_nf_forward_ip+0x352/0x352
Oct  1 08:10:48 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct  1 08:10:48 utgard01 kernel: br_handle_frame+0x290/0x2d3
Oct  1 08:10:48 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct  1 08:10:48 utgard01 kernel: ? br_handle_local_finish+0xe/0xe
Oct  1 08:10:48 utgard01 kernel: __netif_receive_skb_core+0x466/0x798
Oct  1 08:10:48 utgard01 kernel: ? udp_gro_receive+0x4c/0x134
Oct  1 08:10:48 utgard01 kernel: __netif_receive_skb_one_core+0x31/0x69
Oct  1 08:10:48 utgard01 kernel: netif_receive_skb_internal+0x9f/0xba
Oct  1 08:10:48 utgard01 kernel: napi_gro_receive+0x42/0x76
Oct  1 08:10:48 utgard01 kernel: bnx2_poll_work+0xf83/0x1059 [bnx2]
Oct  1 08:10:48 utgard01 kernel: bnx2_poll_msix+0x2e/0xb1 [bnx2]
Oct  1 08:10:48 utgard01 kernel: net_rx_action+0x10b/0x274
Oct  1 08:10:48 utgard01 kernel: __do_softirq+0xce/0x1e2
Oct  1 08:10:48 utgard01 kernel: irq_exit+0x5e/0x9d
Oct  1 08:10:48 utgard01 kernel: do_IRQ+0xa9/0xc7
Oct  1 08:10:48 utgard01 kernel: common_interrupt+0xf/0xf
Oct  1 08:10:48 utgard01 kernel: </IRQ>
Oct  1 08:10:48 utgard01 kernel: RIP: 0010:cpuidle_enter_state+0xe5/0x141
Oct  1 08:10:48 utgard01 kernel: Code: 97 7e ba ff 45 84 ff 74 1d 9c 58 66 66 90 66 90 0f ba e0 09 73 09 0f 0b fa 66 66 90 66 66 90 31 ff e8 ae 0c be ff fb 66 66 90 <66> 66 90 48 2b 1c 24 b8 ff ff ff 7f 48 b9 ff ff ff ff f3 01 00 00
Oct  1 08:10:48 utgard01 kernel: RSP: 0018:ffffc900062fbea0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffdc
Oct  1 08:10:48 utgard01 kernel: RAX: ffff888763b20b00 RBX: 00000d82f12e87bb RCX: 000000000000001f
Oct  1 08:10:48 utgard01 kernel: RDX: 00000d82f12e87bb RSI: 0000000035778ed4 RDI: 0000000000000000
Oct  1 08:10:48 utgard01 kernel: RBP: ffff888763b2bb00 R08: 0000000000000002 R09: 00000000000203c0
Oct  1 08:10:48 utgard01 kernel: R10: 00000000014d4b84 R11: 000020a373b2a030 R12: 0000000000000004
Oct  1 08:10:48 utgard01 kernel: R13: 0000000000000004 R14: ffffffff81e5a018 R15: 0000000000000000
Oct  1 08:10:48 utgard01 kernel: do_idle+0x192/0x20e
Oct  1 08:10:48 utgard01 kernel: cpu_startup_entry+0x6a/0x6c
Oct  1 08:10:48 utgard01 kernel: start_secondary+0x197/0x1b2
Oct  1 08:10:48 utgard01 kernel: secondary_startup_64+0xa4/0xb0
Oct  1 08:10:48 utgard01 kernel: ---[ end trace 3e440b525ec08dde ]---
Oct  1 19:57:11 utgard01 kernel: md: sync done. time=57148sec
Oct  1 19:57:11 utgard01 kernel: md: recovery thread: exit status: 0

 

Link to comment
em1917 Noob

em1917

Posted
  • Author
Posted

this one today

  

Oct  3 18:42:51 utgard01 kernel: WARNING: CPU: 9 PID: 0 at net/netfilter/nf_nat_core.c:420 nf_nat_setup_info+0x6b/0x5fb [nf_nat]
Oct  3 18:42:51 utgard01 kernel: Modules linked in: xt_nat xt_CHECKSUM ipt_REJECT macvlan ip6table_mangle ip6table_nat nf_nat_ipv6 iptable_mangle ip6table_filter ip6_tables vhost_net tun vhost tap ipt_MASQUERADE iptable_nat nf_nat_ipv4 iptable_filter ip_tables nf_nat xfs nfsd lockd grace sunrpc md_mod bonding bnx2 sr_mod cdrom ipmi_ssif i2c_core intel_powerclamp coretemp kvm_intel kvm crc32c_intel mpt3sas acpi_power_meter intel_cstate intel_uncore ipmi_si button ata_piix i7core_edac raid_class scsi_transport_sas [last unloaded: bnx2]
Oct  3 18:42:51 utgard01 kernel: CPU: 9 PID: 0 Comm: swapper/9 Tainted: G        W I       4.19.56-Unraid #1
Oct  3 18:42:51 utgard01 kernel: Hardware name: Dell Inc. PowerEdge R710/00W9X3, BIOS 3.0.0 01/31/2011
Oct  3 18:42:51 utgard01 kernel: RIP: 0010:nf_nat_setup_info+0x6b/0x5fb [nf_nat]
Oct  3 18:42:51 utgard01 kernel: Code: 48 89 fb 48 8b 87 80 00 00 00 49 89 f7 41 89 d6 76 04 0f 0b eb 0b 85 d2 75 07 25 80 00 00 00 eb 05 25 00 01 00 00 85 c0 74 07 <0f> 0b e9 ac 04 00 00 48 8b 83 90 00 00 00 4c 8d 64 24 30 48 8d 73
Oct  3 18:42:51 utgard01 kernel: RSP: 0018:ffff888763b03790 EFLAGS: 00010202
Oct  3 18:42:51 utgard01 kernel: RAX: 0000000000000080 RBX: ffff888d98cb8780 RCX: 0000000000000000
Oct  3 18:42:51 utgard01 kernel: RDX: 0000000000000000 RSI: ffff888763b0387c RDI: ffff888d98cb8780
Oct  3 18:42:51 utgard01 kernel: RBP: ffff888763b03868 R08: ffff888d98cb8780 R09: ffff888e5c743000
Oct  3 18:42:51 utgard01 kernel: R10: 0000000000000158 R11: ffffffff81e8e001 R12: ffff88875ddcfb00
Oct  3 18:42:51 utgard01 kernel: R13: 0000000000000000 R14: 0000000000000000 R15: ffff888763b0387c
Oct  3 18:42:51 utgard01 kernel: FS:  0000000000000000(0000) GS:ffff888763b00000(0000) knlGS:0000000000000000
Oct  3 18:42:51 utgard01 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct  3 18:42:51 utgard01 kernel: CR2: 000014ee7293d320 CR3: 0000000001e0a000 CR4: 00000000000006e0
Oct  3 18:42:51 utgard01 kernel: Call Trace:
Oct  3 18:42:51 utgard01 kernel: <IRQ>
Oct  3 18:42:51 utgard01 kernel: ? fib_validate_source+0xaf/0xd6
Oct  3 18:42:51 utgard01 kernel: ? ipt_do_table+0x58e/0x5db [ip_tables]
Oct  3 18:42:51 utgard01 kernel: nf_nat_alloc_null_binding+0x6f/0x86 [nf_nat]
Oct  3 18:42:51 utgard01 kernel: nf_nat_inet_fn+0xa0/0x192 [nf_nat]
Oct  3 18:42:51 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct  3 18:42:51 utgard01 kernel: ip_local_deliver+0xa9/0xd7
Oct  3 18:42:51 utgard01 kernel: ? ip_sublist_rcv_finish+0x53/0x53
Oct  3 18:42:51 utgard01 kernel: ip_sabotage_in+0x38/0x3e
Oct  3 18:42:51 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct  3 18:42:51 utgard01 kernel: ip_rcv+0x8e/0xbe
Oct  3 18:42:51 utgard01 kernel: ? ip_rcv_finish_core.isra.0+0x2e2/0x2e2
Oct  3 18:42:51 utgard01 kernel: __netif_receive_skb_one_core+0x4d/0x69
Oct  3 18:42:51 utgard01 kernel: netif_receive_skb_internal+0x9f/0xba
Oct  3 18:42:51 utgard01 kernel: br_pass_frame_up+0x123/0x145
Oct  3 18:42:51 utgard01 kernel: ? br_port_flags_change+0x29/0x29
Oct  3 18:42:51 utgard01 kernel: br_handle_frame_finish+0x330/0x375
Oct  3 18:42:51 utgard01 kernel: ? ipt_do_table+0x58e/0x5db [ip_tables]
Oct  3 18:42:51 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct  3 18:42:51 utgard01 kernel: br_nf_hook_thresh+0xa3/0xc3
Oct  3 18:42:51 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct  3 18:42:51 utgard01 kernel: br_nf_pre_routing_finish+0x239/0x260
Oct  3 18:42:51 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct  3 18:42:51 utgard01 kernel: ? nf_nat_ipv4_in+0x1d/0x64 [nf_nat_ipv4]
Oct  3 18:42:51 utgard01 kernel: br_nf_pre_routing+0x2fc/0x321
Oct  3 18:42:51 utgard01 kernel: ? br_nf_forward_ip+0x352/0x352
Oct  3 18:42:51 utgard01 kernel: nf_hook_slow+0x37/0x96
Oct  3 18:42:51 utgard01 kernel: br_handle_frame+0x290/0x2d3
Oct  3 18:42:51 utgard01 kernel: ? br_pass_frame_up+0x145/0x145
Oct  3 18:42:51 utgard01 kernel: ? br_handle_local_finish+0xe/0xe
Oct  3 18:42:51 utgard01 kernel: __netif_receive_skb_core+0x466/0x798
Oct  3 18:42:51 utgard01 kernel: ? udp_gro_receive+0x4c/0x134
Oct  3 18:42:51 utgard01 kernel: __netif_receive_skb_one_core+0x31/0x69
Oct  3 18:42:51 utgard01 kernel: netif_receive_skb_internal+0x9f/0xba
Oct  3 18:42:51 utgard01 kernel: napi_gro_receive+0x42/0x76
Oct  3 18:42:51 utgard01 kernel: bnx2_poll_work+0xf83/0x1059 [bnx2]
Oct  3 18:42:51 utgard01 kernel: bnx2_poll_msix+0x2e/0xb1 [bnx2]
Oct  3 18:42:51 utgard01 kernel: net_rx_action+0x10b/0x274
Oct  3 18:42:51 utgard01 kernel: __do_softirq+0xce/0x1e2
Oct  3 18:42:51 utgard01 kernel: irq_exit+0x5e/0x9d
Oct  3 18:42:51 utgard01 kernel: do_IRQ+0xa9/0xc7
Oct  3 18:42:51 utgard01 kernel: common_interrupt+0xf/0xf
Oct  3 18:42:51 utgard01 kernel: </IRQ>
Oct  3 18:42:51 utgard01 kernel: RIP: 0010:cpuidle_enter_state+0xe5/0x141
Oct  3 18:42:51 utgard01 kernel: Code: 97 7e ba ff 45 84 ff 74 1d 9c 58 66 66 90 66 90 0f ba e0 09 73 09 0f 0b fa 66 66 90 66 66 90 31 ff e8 ae 0c be ff fb 66 66 90 <66> 66 90 48 2b 1c 24 b8 ff ff ff 7f 48 b9 ff ff ff ff f3 01 00 00
Oct  3 18:42:51 utgard01 kernel: RSP: 0018:ffffc900062fbea0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffdc
Oct  3 18:42:51 utgard01 kernel: RAX: ffff888763b20b00 RBX: 0000cd27a382a5b5 RCX: 000000000000001f
Oct  3 18:42:51 utgard01 kernel: RDX: 0000cd27a382a5b5 RSI: 0000000035778ed4 RDI: 0000000000000000
Oct  3 18:42:51 utgard01 kernel: RBP: ffff888763b2bb00 R08: 0000000000000002 R09: 00000000000203c0
Oct  3 18:42:51 utgard01 kernel: R10: 0000000012ae0c9c R11: 0001eb6eb5ea9fb8 R12: 0000000000000004
Oct  3 18:42:51 utgard01 kernel: R13: 0000000000000004 R14: ffffffff81e5a018 R15: 0000000000000000
Oct  3 18:42:51 utgard01 kernel: do_idle+0x192/0x20e
Oct  3 18:42:51 utgard01 kernel: cpu_startup_entry+0x6a/0x6c
Oct  3 18:42:51 utgard01 kernel: start_secondary+0x197/0x1b2
Oct  3 18:42:51 utgard01 kernel: secondary_startup_64+0xa4/0xb0
Oct  3 18:42:51 utgard01 kernel: ---[ end trace 3e440b525ec08ddf ]---

 

Link to comment
LammeN3rd Rookie

LammeN3rd

Posted
Posted (edited)

Ive been noticing the same kind of kernel warnings, system keeps running though. (I also run Unraid on a Dell Poweredge T630)

I've been running 6.7.2 since it's release and these error's are more recent so it can't be due to a core Unraid change, only updated plugin's or dockers are suspect.

 

  

Oct 17 06:09:47 unRAID kernel: WARNING: CPU: 18 PID: 0 at net/netfilter/nf_nat_core.c:420 nf_nat_setup_info+0x6b/0x5fb [nf_nat]
Oct 17 06:09:47 unRAID kernel: Modules linked in: xt_CHECKSUM veth ipt_REJECT ip6table_mangle ip6table_nat nf_nat_ipv6 xt_nat iptable_mangle ip6table_filter ip6_tables macvlan vhost_net tun vhost tap ipt_MASQUERADE iptable_nat nf_nat_ipv4 iptable_filter ip_tables nf_nat nfsd lockd grace sunrpc md_mod ipmi_devintf igb i2c_algo_bit sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd cryptd glue_helper ipmi_ssif intel_cstate intel_uncore nvme intel_rapl_perf i2c_core mxm_wmi megaraid_sas nvme_core wmi ipmi_si acpi_power_meter acpi_pad pcc_cpufreq button [last unloaded: i2c_algo_bit]
Oct 17 06:09:47 unRAID kernel: CPU: 18 PID: 0 Comm: swapper/18 Tainted: G        W         4.19.56-Unraid #1
Oct 17 06:09:47 unRAID kernel: Hardware name: Dell Inc. PowerEdge T630/0NT78X, BIOS 2.10.5 08/07/2019
Oct 17 06:09:47 unRAID kernel: RIP: 0010:nf_nat_setup_info+0x6b/0x5fb [nf_nat]
Oct 17 06:09:47 unRAID kernel: Code: 48 89 fb 48 8b 87 80 00 00 00 49 89 f7 41 89 d6 76 04 0f 0b eb 0b 85 d2 75 07 25 80 00 00 00 eb 05 25 00 01 00 00 85 c0 74 07 <0f> 0b e9 ac 04 00 00 48 8b 83 90 00 00 00 4c 8d 64 24 30 48 8d 73
Oct 17 06:09:47 unRAID kernel: RSP: 0018:ffff88903fa83808 EFLAGS: 00010202
Oct 17 06:09:47 unRAID kernel: RAX: 0000000000000080 RBX: ffff88a0256ba640 RCX: 0000000000000000
Oct 17 06:09:47 unRAID kernel: RDX: 0000000000000000 RSI: ffff88903fa838f4 RDI: ffff88a0256ba640
Oct 17 06:09:47 unRAID kernel: RBP: ffff88903fa838e0 R08: ffff88a0256ba640 R09: 0000000000000000
Oct 17 06:09:47 unRAID kernel: R10: 0000000000000158 R11: ffffffff81e8e001 R12: ffff88859e47ff00
Oct 17 06:09:47 unRAID kernel: R13: 0000000000000000 R14: 0000000000000000 R15: ffff88903fa838f4
Oct 17 06:09:47 unRAID kernel: FS:  0000000000000000(0000) GS:ffff88903fa80000(0000) knlGS:0000000000000000
Oct 17 06:09:47 unRAID kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 17 06:09:47 unRAID kernel: CR2: 000000c42017a000 CR3: 0000000001e0a001 CR4: 00000000003626e0
Oct 17 06:09:47 unRAID kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 17 06:09:47 unRAID kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Oct 17 06:09:47 unRAID kernel: Call Trace:
Oct 17 06:09:47 unRAID kernel: <IRQ>
Oct 17 06:09:47 unRAID kernel: ? ipt_do_table+0x58e/0x5db [ip_tables]
Oct 17 06:09:47 unRAID kernel: nf_nat_alloc_null_binding+0x6f/0x86 [nf_nat]
Oct 17 06:09:47 unRAID kernel: nf_nat_inet_fn+0xa0/0x192 [nf_nat]
Oct 17 06:09:47 unRAID kernel: nf_hook_slow+0x37/0x96
Oct 17 06:09:47 unRAID kernel: ip_local_deliver+0xa9/0xd7
Oct 17 06:09:47 unRAID kernel: ? ip_sublist_rcv_finish+0x53/0x53
Oct 17 06:09:47 unRAID kernel: ip_sabotage_in+0x38/0x3e
Oct 17 06:09:47 unRAID kernel: nf_hook_slow+0x37/0x96
Oct 17 06:09:47 unRAID kernel: ip_rcv+0x8e/0xbe
Oct 17 06:09:47 unRAID kernel: ? ip_rcv_finish_core.isra.0+0x2e2/0x2e2
Oct 17 06:09:47 unRAID kernel: __netif_receive_skb_one_core+0x4d/0x69
Oct 17 06:09:47 unRAID kernel: netif_receive_skb_internal+0x9f/0xba
Oct 17 06:09:47 unRAID kernel: br_pass_frame_up+0x123/0x145
Oct 17 06:09:47 unRAID kernel: ? br_port_flags_change+0x29/0x29
Oct 17 06:09:47 unRAID kernel: br_handle_frame_finish+0x330/0x375
Oct 17 06:09:47 unRAID kernel: ? ipt_do_table+0x58e/0x5db [ip_tables]
Oct 17 06:09:47 unRAID kernel: ? br_pass_frame_up+0x145/0x145
Oct 17 06:09:47 unRAID kernel: br_nf_hook_thresh+0xa3/0xc3
Oct 17 06:09:47 unRAID kernel: ? br_pass_frame_up+0x145/0x145
Oct 17 06:09:47 unRAID kernel: br_nf_pre_routing_finish+0x239/0x260
Oct 17 06:09:47 unRAID kernel: ? br_pass_frame_up+0x145/0x145
Oct 17 06:09:47 unRAID kernel: ? nf_nat_ipv4_in+0x1d/0x64 [nf_nat_ipv4]
Oct 17 06:09:47 unRAID kernel: br_nf_pre_routing+0x2fc/0x321
Oct 17 06:09:47 unRAID kernel: ? br_nf_forward_ip+0x352/0x352
Oct 17 06:09:47 unRAID kernel: nf_hook_slow+0x37/0x96
Oct 17 06:09:47 unRAID kernel: br_handle_frame+0x290/0x2d3
Oct 17 06:09:47 unRAID kernel: ? br_pass_frame_up+0x145/0x145
Oct 17 06:09:47 unRAID kernel: ? br_handle_local_finish+0xe/0xe
Oct 17 06:09:47 unRAID kernel: __netif_receive_skb_core+0x466/0x798
Oct 17 06:09:47 unRAID kernel: ? udp_gro_receive+0x4c/0x134
Oct 17 06:09:47 unRAID kernel: __netif_receive_skb_one_core+0x31/0x69
Oct 17 06:09:47 unRAID kernel: netif_receive_skb_internal+0x9f/0xba
Oct 17 06:09:47 unRAID kernel: napi_gro_receive+0x42/0x76
Oct 17 06:09:47 unRAID kernel: igb_poll+0xb96/0xbbc [igb]
Oct 17 06:09:47 unRAID kernel: net_rx_action+0x10b/0x274
Oct 17 06:09:47 unRAID kernel: __do_softirq+0xce/0x1e2
Oct 17 06:09:47 unRAID kernel: irq_exit+0x5e/0x9d
Oct 17 06:09:47 unRAID kernel: do_IRQ+0xa9/0xc7
Oct 17 06:09:47 unRAID kernel: common_interrupt+0xf/0xf
Oct 17 06:09:47 unRAID kernel: </IRQ>
Oct 17 06:09:47 unRAID kernel: RIP: 0010:cpuidle_enter_state+0xe8/0x141
Oct 17 06:09:47 unRAID kernel: Code: ff 45 84 ff 74 1d 9c 58 0f 1f 44 00 00 0f ba e0 09 73 09 0f 0b fa 66 0f 1f 44 00 00 31 ff e8 ae 0c be ff fb 66 0f 1f 44 00 00 <48> 2b 1c 24 b8 ff ff ff 7f 48 b9 ff ff ff ff f3 01 00 00 48 39 cb
Oct 17 06:09:47 unRAID kernel: RSP: 0018:ffffc900064b3ea0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd2
Oct 17 06:09:47 unRAID kernel: RAX: ffff88903faa0b00 RBX: 00006e7edafc25ba RCX: 000000000000001f
Oct 17 06:09:47 unRAID kernel: RDX: 00006e7edafc25ba RSI: 0000000035652651 RDI: 0000000000000000
Oct 17 06:09:47 unRAID kernel: RBP: ffff88903faab400 R08: 0000000000000002 R09: 00000000000203c0
Oct 17 06:09:47 unRAID kernel: R10: 00000000006ac764 R11: 0015366f8d483640 R12: 0000000000000004
Oct 17 06:09:47 unRAID kernel: R13: 0000000000000004 R14: ffffffff81e5a018 R15: 0000000000000000
Oct 17 06:09:47 unRAID kernel: do_idle+0x192/0x20e
Oct 17 06:09:47 unRAID kernel: cpu_startup_entry+0x6a/0x6c
Oct 17 06:09:47 unRAID kernel: start_secondary+0x197/0x1b2
Oct 17 06:09:47 unRAID kernel: secondary_startup_64+0xa4/0xb0
Oct 17 06:09:47 unRAID kernel: ---[ end trace 6ab3ef74e0b3e5a4 ]---

 

unraid-diagnostics-20191017-1333.zip

Edited by LammeN3rd
Link to comment
  • 2 weeks later...
LammeN3rd Rookie

LammeN3rd

Posted
Posted

Quick update, I've done a lot of testing removing plugins, disabling Virtual machines and Docker before I rolled back the firmware of my intel NIC (i350) and Bios that I had updated recently. After the BIOS downgrade to 2.9.1 I've not seen the issue for days upgrading tot 2.10.5 brings back the crashes!

 

I've not reinstalled my plugins so I can't be 100% sure its the BIOS but if anyone else has this issue I highly recommend looking at your BIOS!!

 

Link to comment
LammeN3rd Rookie

LammeN3rd

Posted
Posted
On 10/10/2019 at 10:27 PM, em1917 said:

here are the diagnostic files. there are no hardware errors. this is running on a Dell R710 Server and I checked for ECC issues and other hardware failures that the drac would have picked up.

utgard01-diagnostics-20191010-2026.zip 94.26 kB · 0 downloads

I suggest looking at your BIOS! from the above log it seems you are running a 2011 version the last version available is from feb 2019 

https://www.dell.com/support/home/nl/nl/nlbsdt1/drivers/driversdetails?driverid=0f4yy&oscode=ws8r2&productcode=poweredge-r710

 

for my Poweredge T630 the BIOS from Feb 2019 fixed my issue!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
Go to topic listing