stultus Posted August 12, 2022 Share Posted August 12, 2022 6.9.1 was beautifully stable with 6+ month uptimes for me. I upgraded to 6.10 around July 4th, got a lot of PCIe errors, and so moved to 6.10.3 to solve that known bug. Ever since then my system dies every week or too. This is the last thing I see on the monitor, always related to nf_conntrack which appears to be a docker/network issue. Last weekend I changed from macvlan to ipvlan but that didn't solve my issue. Any ideas where to start poking around? Aug 3 01:54:07 Sophos kernel: md: recovery thread: check P Q ... Aug 3 01:54:56 Sophos flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update Aug 3 02:25:19 Sophos kernel: ------------[ cut here ]------------ Aug 3 02:25:19 Sophos kernel: WARNING: CPU: 1 PID: 14065 at net/netfilter/nf_conntrack_core.c:1192 __nf_conntrack_confirm+0xb8/0x254 [nf_conntrack] Aug 3 02:25:19 Sophos kernel: Modules linked in: tun nvidia_modeset(PO) nvidia_uvm(PO) veth macvlan xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xfs md_mod nvidia(PO) drm backlight nct6775 hwmon_vid efivarfs iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables btusb btrtl btbcm edac_mce_amd kvm_amd kvm crct10dif_pclmul wmi_bmof mxm_wmi btintel crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd bluetooth igb rapl input_leds i2c_piix4 i2c_algo_bit ccp ecdh_generic hid_lg_g15 k10temp i2c_core ahci led_class ecc libahci wmi button acpi_cpufreq Aug 3 02:25:19 Sophos kernel: CPU: 1 PID: 14065 Comm: kworker/1:1 Tainted: P O 5.15.46-Unraid #1 Aug 3 02:25:19 Sophos kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./X370 Taichi, BIOS P6.40 07/28/2020 Aug 3 02:25:19 Sophos kernel: Workqueue: events macvlan_process_broadcast [macvlan] Aug 3 02:25:19 Sophos kernel: RIP: 0010:__nf_conntrack_confirm+0xb8/0x254 [nf_conntrack] Aug 3 02:25:19 Sophos kernel: Code: 89 c6 48 89 44 24 18 e8 53 e4 ff ff 44 89 f2 44 89 ef 89 c6 89 c5 e8 2c e8 ff ff 84 c0 75 9f 49 8b 87 80 00 00 00 a8 08 74 19 <0f> 0b 89 ee 44 89 ef 45 31 e4 e8 dc df ff ff e8 ae e4 ff ff e9 71 Aug 3 02:25:19 Sophos kernel: RSP: 0018:ffffc900001dcdb0 EFLAGS: 00010202 Aug 3 02:25:19 Sophos kernel: RAX: 0000000000000188 RBX: ffffffff828e1500 RCX: 0000000000000000 Aug 3 02:25:19 Sophos kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffa02c1648 Aug 3 02:25:19 Sophos kernel: RBP: 000000000000fca9 R08: 3705f11d51bb3b46 R09: 9409ee1e63995df3 Aug 3 02:25:19 Sophos kernel: R10: aef9ca04f70f31cc R11: 8711e21c188c2656 R12: 0000000000005102 Aug 3 02:25:19 Sophos kernel: R13: 0000000000005102 R14: 0000000000000000 R15: ffff88814e840dc0 Aug 3 02:25:19 Sophos kernel: FS: 0000000000000000(0000) GS:ffff8887fe840000(0000) knlGS:0000000000000000 Aug 3 02:25:19 Sophos kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 3 02:25:19 Sophos kernel: CR2: 000000c00014d000 CR3: 0000000148b04000 CR4: 0000000000350ee0 Aug 3 02:25:19 Sophos kernel: Call Trace: Aug 3 02:25:19 Sophos kernel: <IRQ> Aug 3 02:25:19 Sophos kernel: nf_conntrack_confirm+0x26/0x3f [nf_conntrack] Aug 3 02:25:19 Sophos kernel: nf_hook_slow+0x3e/0x93 Aug 3 02:25:19 Sophos kernel: ? ip_protocol_deliver_rcu+0x135/0x135 Aug 3 02:25:19 Sophos kernel: NF_HOOK.constprop.0+0x73/0xce Aug 3 02:25:19 Sophos kernel: ? ip_protocol_deliver_rcu+0x135/0x135 Aug 3 02:25:19 Sophos kernel: __netif_receive_skb_one_core+0x79/0x9a Aug 3 02:25:19 Sophos kernel: process_backlog+0xab/0x143 Aug 3 02:25:19 Sophos kernel: __napi_poll.constprop.0+0x2a/0x114 Aug 3 02:25:19 Sophos kernel: net_rx_action+0xe8/0x1f2 Aug 3 02:25:19 Sophos kernel: __do_softirq+0xef/0x218 Aug 3 02:25:19 Sophos kernel: do_softirq+0x50/0x68 Aug 3 02:25:19 Sophos kernel: </IRQ> Aug 3 02:25:19 Sophos kernel: <TASK> Aug 3 02:25:19 Sophos kernel: netif_rx_ni+0x53/0x85 Aug 3 02:25:19 Sophos kernel: macvlan_broadcast+0x116/0x144 [macvlan] Aug 3 02:25:19 Sophos kernel: macvlan_process_broadcast+0xc7/0x110 [macvlan] Aug 3 02:25:19 Sophos kernel: process_one_work+0x198/0x27a Aug 3 02:25:19 Sophos kernel: worker_thread+0x19c/0x240 Aug 3 02:25:19 Sophos kernel: ? rescuer_thread+0x28b/0x28b Aug 3 02:25:19 Sophos kernel: kthread+0xde/0xe3 Aug 3 02:25:19 Sophos kernel: ? set_kthread_struct+0x32/0x32 Aug 3 02:25:19 Sophos kernel: ret_from_fork+0x22/0x30 Aug 3 02:25:19 Sophos kernel: </TASK> Aug 3 02:25:19 Sophos kernel: ---[ end trace f391be142f2ac721 ]--- sophos-diagnostics-20220812-1638.zip Quote Link to comment
JonathanM Posted August 13, 2022 Share Posted August 13, 2022 3 hours ago, stultus said: Last weekend I changed from macvlan to ipvlan but that didn't solve my issue. Are you sure the change was applied? 3 hours ago, stultus said: Aug 3 02:25:19 Sophos kernel: Modules linked in: tun nvidia_modeset(PO) nvidia_uvm(PO) veth macvlan xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xfs md_mod nvidia(PO) drm backlight nct6775 hwmon_vid efivarfs iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables btusb btrtl btbcm edac_mce_amd kvm_amd kvm crct10dif_pclmul wmi_bmof mxm_wmi btintel crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd bluetooth igb rapl input_leds i2c_piix4 i2c_algo_bit ccp ecdh_generic hid_lg_g15 k10temp i2c_core ahci led_class ecc libahci wmi button acpi_cpufreq Aug 3 02:25:19 Sophos kernel: CPU: 1 PID: 14065 Comm: kworker/1:1 Tainted: P O 5.15.46-Unraid #1 Aug 3 02:25:19 Sophos kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./X370 Taichi, BIOS P6.40 07/28/2020 Aug 3 02:25:19 Sophos kernel: Workqueue: events macvlan_process_broadcast [macvlan] Quote Link to comment
JorgeB Posted August 13, 2022 Share Posted August 13, 2022 12 hours ago, stultus said: Aug 3 02:25:19 Sophos kernel: macvlan_broadcast+0x116/0x144 [macvlan] Aug 3 02:25:19 Sophos kernel: macvlan_process_broadcast+0xc7/0x110 [macvlan] It's still using macvlan. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.