[指南] 如何解决自定义网络上容器的 macvlan 和 ipvlan 问题


Recommended Posts

>>> Updated Aug 31, 2023 to advise upgrading to Unraid 6.12.4 rather than following this guide

If you are getting call traces related to macvlan, as a first step we recommend navigating to Settings > Docker, switch to advanced view, and change the "Docker custom network type" from macvlan to ipvlan. This is the default configuration that Unraid has shipped with since version 6.11.5 and should work for most systems.

 

However, some users have reported issues with port forwarding from certain routers (Fritzbox) and reduced functionality with advanced network management tools (Ubiquity) when in ipvlan mode.

 

For those users, we have a new method that reworks networking to avoid this.  Upgrade to Unraid 6.12.4:

and follow the guide in the Release Notes to avoid macvlan call traces and crashes:

  https://docs.unraid.net/unraid-os/release-notes/6.12.4/

 

----------------------------

 

For historical purposes, below are details on the 2-nic Docker segmentation method for avoiding macvlan call traces. If you choose to follow these instructions, do not mix and match solutions. Either use the method below OR the new method available in 6.12.4

 

 

USE A DEDICATED ETHERNET PORT FOR DOCKER ONLY

 

>>> Updated Aug 8, 2023 to advise that bridging should be disabled on the dedicated interface

 

You will need an additional ethernet port of your server to make this solution work. This additional port is connected to your local router or switch just like the main port. No network modifications are required for your main connection eth0/br0 unless it is configured as bond or bridge interface with multiple ports and you need to free up one port from the bond or bridge interface and turn it into a dedicated port for docker connections.

 

1. Stop the array and navigate to Settings -> Network Settings and configure the dedicated interface

    - This guide assumes eth1 will be the dedicated interface, adjust for the appropriate ethX device as needed

    - Disable bonding and bridging for this interface (important!)

    - Use IPv4 only or IPv4 and IPv6 as per your case

    - No IP addresses are assigned to this interface

 

image.png

 

2. Navigate to Settings -> Docker and configure Docker to use this dedicated interface

    - Set "Docker custom network type" to "macvlan"

 

image.png

 

    - Scroll down and disable the IP assignment(s) of eth0 / br0 which is going to be replaced

 

image.png

image.png

 

   - Assign manually the "old" assignments to the new dedicated interface (use eth1, not br1 as shown in the image)

 

image.png

image.png

 

3. Start the array and configure the Docker containers with the new custom network

    - Any containers that previously had "Custom: br0" should be changed to "Custom: eth1" (not br1 as shown in the image)

    - IP addresses may be fixed (as in the example below) or dynamic using DHCP (configured in the previous step)

 

image.png

 

THAT'S IT - NOW YOUR DOCKER CONTAINER(S) OPERATE ON A DEDICATED CUSTOM NETWORK 

 

  • Like 7
  • Thanks 2
  • Upvote 2
Link to comment
  • 2 weeks later...

I've tried a bunch of different things to get rid of messages like this without any luck. I usually get one or two per day, as well as a random crash per week or so. 

Mar  2 22:21:04 Unraid kernel: WARNING: CPU: 1 PID: 7504 at net/netfilter/nf_conntrack_core.c:1208 __nf_conntrack_confirm+0xa5/0x2cb [nf_conntrack]
Mar  2 22:21:04 Unraid kernel: Modules linked in: xt_CHECKSUM ipt_REJECT nf_reject_ipv4 ip6table_mangle ip6table_nat iptable_mangle vhost_net tun vhost vhost_iotlb tap macvlan xt_nat xt_tcpudp veth xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter xfs md_mod ipmi_devintf jc42 efivarfs ip6table_filter ip6_tables iptable_filter ip_tables x_tables bridge stp llc bonding tls igb intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ast drm_vram_helper drm_ttm_helper ttm aesni_intel crypto_simd drm_kms_helper ipmi_ssif cryptd i2c_i801 intel_cstate joydev i2c_smbus drm input_leds backlight led_class agpgart syscopyarea sysfillrect sysimgblt i2c_algo_bit acpi_ipmi ahci fb_sys_fops i2c_core libahci ipmi_si button acpi_cpufreq unix [last unloaded: igb]

 

Implemented the changes suggested above and no problems so far. Three days ago and counting...

On an ASRock Rack C2750D4I that never gives my enough sleep.

  • Like 1
Link to comment
8 hours ago, Omri said:

Hi

Tried to follow your guide

But I guess I miss something

There isn't br1 in my containers config

Although I configured eth1 like the screenshot you provided and configured docker setting with br1 instead of br0.

 

Make sure to have “IPv4 address assignment:” set to “none” for your eth1. (As well as for ipv6). 

 

Link to comment
12 hours ago, insomnia417 said:

这不是个很完美的解决方案,我折腾了2周,被死机搞烦了,现在已经降级回6.11.5,等正式版内核看看会不会修复macvlan的bug

 

 @insomnia417 said (from deepl): This is not a perfect solution, I tossed 2 weeks, was bored by the crash, and now have downgraded back to 6.11.5, waiting for the official kernel to see if the macvlan bug will be fixed

Edited by wassereimer
@ corrected
Link to comment

I got this working and the macvlan issue remains solved a couple of weeks later. No crashes since I implemented this (on 11.5). Uptime is 3 weeks and counting or the first time ever. Sorry to hear that there seems to be problems for people to get this configured as described by @bonienl

Link to comment
On 3/28/2023 at 7:05 PM, bonienl said:

 

    - Disable the IP assignment(s) of eth0 / br0 which is going to be replaced

 

image.png

image.png

 

   - Assign manually the "old" assignments to the new dedicated interface

 

image.png

image.png

 

 

I found something. You write "Assign manually the "old" assignments to the new dedicated interface" but you didn't do that. Look at the ipv6 Gateway at br0 and br1. And it works for me if I don't enable the ipv6 custom network on br1, but just ipv4. Then Unraid creates the br1 macvlan network in docker (but you can't see it in the routing table). So there must be something wrong with taking the "old" assignments from br0.

Edited by wassereimer
Link to comment

I think it's a bug in 6.12 (rc3)

BR1 isn't being created

even when I disable bridging on eth0 and enabling it only on eth1

docker filling the right subnet/gateway on br1 but docker doesn't create br1 network (although it's visible in ifconfig)

will be glad if someone who got it working will post docker.cfg and network.cfg from /boot/config

 

Thanks in advance

Link to comment
5 hours ago, bonienl said:

 

It works with Unraid version 6.12. Please upload your diagnostics, likely there is a configuration error.

 

 

Of course. Thank you for looking into it. If I change the "IPv4/IPv6 address assignment" to "Automatic", the br1 Network is correctly created and visible in the routing table. If set to none, that doesn't happen. Even after setting everything in the Docker Settings and enabling the Service.

homeserver-diagnostics-20230418-1850.zip

Link to comment

Thanks for posting this, I had been using ipvlan without issue with Untangle and OPNsense, but recently switched to Sophos XG and it immediately had issues with it, I changed it to macvlan and have had no further issues but remember I switched to ipvlan for a reason, I mean I can't remember why now but I was curious why Sophos didn't like ipvlan... but this seems like a much better option regardless.

Link to comment

Well, figured out a solution (although not perfect)

eth0 was using automatic setting for IPV4+IPV6

after setting it to static, docker network br1 appeared and it's working (IPV4+IPV6).

 

eth1 is using automatic setting for IPV4+IPV6 so I didn't need to set addresses in docker settings (which is good because my IPV6 prefix isn't fixed, and a manual setting would not be valid after my prefix will change).

 

The only "problem" now is that Unraid GUI has two IP's in same subnet.

Link to comment
  • anpple changed the title to [指南] 如何解决自定义网络上容器的 macvlan 和 ipvlan 问题
  • JorgeB unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.