6.12.3 daily crashes - switched to IPVLAN - still have macvlan traces

[WWGT]

--- EDIT ---

I did "docker network ls" from the command line and I get this back. But my setting in the GUI is set to IPVLAN..

NETWORK ID     NAME      DRIVER    SCOPE
c09454c9248f   br0       macvlan   local
838447d880bf   bridge    bridge    local
dbcc6f485253   host      host      local
9dfb19da0e21   none      null      local

 

--- EDIT 2 ---

I looked into the logs when docker service was starting and saw this

Jul 28 12:03:34 Tower root: Error response from daemon: error while removing network: network br0 id c09454c9248f8adcc24ac6895002ee6f16edd4e69b0c90380a2ec6d2f87d8f20 has active endpoints
Jul 28 12:03:34 Tower root: Error response from daemon: network with name br0 already exists
Jul 28 12:03:34 Tower root: Error response from daemon: endpoint with name MS-SQL-Server already exists in network br0

 

So I temporarily switched the network of that container to bridge and restarted docker. And voila, it's set to ipvlan now.

So it's possible this changes in the GUI but does not actually change the network. So if you are somneone that gets these crashes even after switching to IPVLAN make sure to double check this!

 

--- ORIGINAL POST ---

 

After the crashes I switched my docker settings from macvlan to ipvlan, but it doesn't help. It still crashes between hours and a day or 2.

I enabled syslog server and see that I STILL get macvlan traces. How is this possible? Is there anywhere else macvlan is used?

 

 

Jul 28 09:36:53 Tower kernel: ------------[ cut here ]------------
Jul 28 09:36:53 Tower kernel: WARNING: CPU: 4 PID: 25775 at net/netfilter/nf_conntrack_core.c:1210 __nf_conntrack_confirm+0xa4/0x2b0 [nf_conntrack]
Jul 28 09:36:53 Tower kernel: Modules linked in: xt_connmark xt_comment iptable_raw xt_mark xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net vhost vhost_iotlb tap udp_diag tun macvlan veth xt_nat xt_tcpudp xt_conntrack nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype br_netfilter xfs dm_crypt dm_mod md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) tcp_diag inet_diag nct6775 nct6775_core hwmon_vid iptable_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libchacha ip6table_filter ip6_tables iptable_filter ip_tables x_tables efivarfs bridge stp llc bonding tls i915 intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm iosf_mbi drm_buddy i2c_algo_bit ttm drm_display_helper drm_kms_helper crct10dif_pclmul crc32_pclmul
Jul 28 09:36:53 Tower kernel: crc32c_intel ghash_clmulni_intel sha512_ssse3 aesni_intel drm crypto_simd cryptd rapl intel_cstate joydev i2c_i801 intel_gtt mei_hdcp mei_pxp wmi_bmof mpt3sas intel_uncore nvme agpgart i2c_smbus mei_me r8169 nvme_core ahci i2c_core mei libahci raid_class realtek syscopyarea scsi_transport_sas sysfillrect sysimgblt tpm_crb fb_sys_fops video tpm_tis tpm_tis_core tpm wmi intel_pmc_core backlight acpi_pad acpi_tad button unix [last unloaded: md_mod]
Jul 28 09:36:53 Tower kernel: CPU: 4 PID: 25775 Comm: kworker/u40:5 Tainted: P           O       6.1.38-Unraid #2
Jul 28 09:36:53 Tower kernel: Hardware name: ASRock B760M PG Riptide/B760M PG Riptide, BIOS 4.01 01/06/2023
Jul 28 09:36:53 Tower kernel: Workqueue: events_unbound macvlan_process_broadcast [macvlan]
Jul 28 09:36:53 Tower kernel: RIP: 0010:__nf_conntrack_confirm+0xa4/0x2b0 [nf_conntrack]
Jul 28 09:36:53 Tower kernel: Code: 44 24 10 e8 e2 e1 ff ff 8b 7c 24 04 89 ea 89 c6 89 04 24 e8 7e e6 ff ff 84 c0 75 a2 48 89 df e8 9b e2 ff ff 85 c0 89 c5 74 18 <0f> 0b 8b 34 24 8b 7c 24 04 e8 18 dd ff ff e8 93 e3 ff ff e9 72 01
Jul 28 09:36:53 Tower kernel: RSP: 0018:ffffc900002d0d98 EFLAGS: 00010202
Jul 28 09:36:53 Tower kernel: RAX: 0000000000000001 RBX: ffff8883fea2ed00 RCX: 8ab619927dd5b8f5
Jul 28 09:36:53 Tower kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8883fea2ed00
Jul 28 09:36:53 Tower kernel: RBP: 0000000000000001 R08: f59f596ce459572b R09: cd5731d16da276b0
Jul 28 09:36:53 Tower kernel: R10: 96fc34b9065abae0 R11: ffffc900002d0d60 R12: ffffffff82a11d00
Jul 28 09:36:53 Tower kernel: R13: 0000000000034cde R14: ffff888207d0b200 R15: 0000000000000000
Jul 28 09:36:53 Tower kernel: FS:  0000000000000000(0000) GS:ffff88904f300000(0000) knlGS:0000000000000000
Jul 28 09:36:53 Tower kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 28 09:36:53 Tower kernel: CR2: 000014df7ffde840 CR3: 000000000420a000 CR4: 0000000000752ee0
Jul 28 09:36:53 Tower kernel: PKRU: 55555554
Jul 28 09:36:53 Tower kernel: Call Trace:
Jul 28 09:36:53 Tower kernel: <IRQ>
Jul 28 09:36:53 Tower kernel: ? __warn+0xab/0x122
Jul 28 09:36:53 Tower kernel: ? report_bug+0x109/0x17e
Jul 28 09:36:53 Tower kernel: ? __nf_conntrack_confirm+0xa4/0x2b0 [nf_conntrack]
Jul 28 09:36:53 Tower kernel: ? handle_bug+0x41/0x6f
Jul 28 09:36:53 Tower kernel: ? exc_invalid_op+0x13/0x60
Jul 28 09:36:53 Tower kernel: ? asm_exc_invalid_op+0x16/0x20
Jul 28 09:36:53 Tower kernel: ? __nf_conntrack_confirm+0xa4/0x2b0 [nf_conntrack]
Jul 28 09:36:53 Tower kernel: ? __nf_conntrack_confirm+0x9e/0x2b0 [nf_conntrack]
Jul 28 09:36:53 Tower kernel: ? nf_nat_inet_fn+0xc0/0x1a8 [nf_nat]
Jul 28 09:36:53 Tower kernel: nf_conntrack_confirm+0x25/0x54 [nf_conntrack]
Jul 28 09:36:53 Tower kernel: nf_hook_slow+0x3a/0x96
Jul 28 09:36:53 Tower kernel: ? ip_protocol_deliver_rcu+0x164/0x164
Jul 28 09:36:53 Tower kernel: NF_HOOK.constprop.0+0x79/0xd9
Jul 28 09:36:53 Tower kernel: ? ip_protocol_deliver_rcu+0x164/0x164
Jul 28 09:36:53 Tower kernel: __netif_receive_skb_one_core+0x77/0x9c
Jul 28 09:36:53 Tower kernel: process_backlog+0x8c/0x116
Jul 28 09:36:53 Tower kernel: __napi_poll.constprop.0+0x28/0x124
Jul 28 09:36:53 Tower kernel: net_rx_action+0x159/0x24f
Jul 28 09:36:53 Tower kernel: __do_softirq+0x126/0x288
Jul 28 09:36:53 Tower kernel: do_softirq+0x7f/0xab
Jul 28 09:36:53 Tower kernel: </IRQ>
Jul 28 09:36:53 Tower kernel: <TASK>
Jul 28 09:36:53 Tower kernel: __local_bh_enable_ip+0x4c/0x6b
Jul 28 09:36:53 Tower kernel: netif_rx+0x52/0x5a
Jul 28 09:36:53 Tower kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Jul 28 09:36:53 Tower kernel: ? _raw_spin_unlock+0x14/0x29
Jul 28 09:36:53 Tower kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]
Jul 28 09:36:53 Tower kernel: process_one_work+0x1a8/0x295
Jul 28 09:36:53 Tower kernel: worker_thread+0x18b/0x244
Jul 28 09:36:53 Tower kernel: ? rescuer_thread+0x281/0x281
Jul 28 09:36:53 Tower kernel: kthread+0xe4/0xef
Jul 28 09:36:53 Tower kernel: ? kthread_complete_and_exit+0x1b/0x1b
Jul 28 09:36:53 Tower kernel: ret_from_fork+0x1f/0x30
Jul 28 09:36:53 Tower kernel: </TASK>
Jul 28 09:36:53 Tower kernel: ---[ end trace 0000000000000000 ]---

 

Edited July 28, 2023 by WWGT
Found a bug + fix for it in how network type is changed in docker

[dlandon]

Please post diagnostics so we can investigate this.  We haven't been able to reproduce it yet.