Communication errors after updating to 6.10.0-rc2(3) to use ipvlan in docker custom network type instead of macvlan

[mikesp18]

Editted: Updated to rc3, problem still exists with IPVLAN

 

Title says it. I had a previous post about crashes. A suggestion was that since I was getting macvlan call traces, that I update to 6.10.0-rc2 and change the docker network type to ipvlan instead (SETTINGS->DOCKER->DOCKER CUSTOM NETWORK TYPE->IPVLAN). This did eliminate the previous errors.

 

However, now I have some new issues.

 

I have communication problems. The shares are still accessible. Examples: Fix common problems indicates that I cannot communicate with GitHub, I cannot download blacklist for apps. I cannot update plugins or dockers as they cannot communicate.

 

Now, the interesting part (for me, but I'm pretty dumb).  If I go to SETTINGS->DOCKER->ENABLE DOCKER and change to NO, then I can communicate fine.  Also, when I switch the ENABLE DOCKER back to YES, I get an amount of time where communication seems fine, can even update the dockers without getting NOT AVAILABLE for a minute or two.  Testing this just now, about 2-3 minutes.

 

Any ideas? It's worth noting I do have a second Unraid installation in the same house, and it's unchanged 6.9.2 and still presumably using the macvlan setting for custom networks. No problems on the other system, so I don't think it's a router/hardware issue.

orcrist-diagnostics-20220306-1236.zip

Edited March 13, 2022 by mikesp18

[Squid]

Any thoughts @bonienl?

[bonienl]

Make sure there is no IP address overlap between the docker containers on each Unraid server.

Each Unraid server must use a unique DHCP address range.

 

[mikesp18]

7 hours ago, bonienl said:

Make sure there is no IP address overlap between the docker containers on each Unraid server.

Each Unraid server must use a unique DHCP address range.

 

I just looked through and spotted no overlaps. They are on physically different hardware, 192.168.86.113 vs 192.168.86.115

[bonienl]

Docker uses its own dhcp service, which is configured under Docker settings.

When you use multiple Unraid servers, then each server must have a different dhcp range configured.

 

[mikesp18]

OK, I think I am following.  Forgive me as I'm typically a little out of my depth. I've screenshotted my current settings between the respective installations. I've also included the limited DHCP pool settings that my Google Nest Wifi Router will allow.  I figured that the DHCP pool of the Nest should NOT include the IPs of the DHCP pool of unraid to avoid conflicts, does this sound correct? So now the 6.10.0-rc2 (Orcrist) has 192.168.86.240-247, the 6.9.2 (Grond) has 192.168.86.248-255), and the Nest Wifi pool is 192.168.86.20-239.  I think this all sounds correct.

 

 

[bonienl]

Yeah, this should work, the key is indeed no overlap between the different dhcp servers.

 

[mikesp18]

FWIW, this did not clear up the communications problem. Each time the Docker is enable, about 2 minutes later, I lose communication.  I can ping from the affect server console within the 192.168.86.x network, but I cannot ping outside. Also, I can confirm that I can ping those same IP addresses if the ENABLE DOCKER is set to NO, or if ENABLE DOCKER set to YES and the DOCKER CUSTOM NETWORK TYPE is set to MACVLAN instead of IPVLAN.

Edited March 7, 2022 by mikesp18

[greenflash24]

I am experiencing exactly the same kind of issues after switching from macvlan to ipvlan.

 

I can confirm, that unraid is not able to ping any IPV4 address which is outside of my LAN, but i can ping IPV4 addresses inside of my LAN.

Additionally IPV6 addresses can be reached perfectly from unraid (inside LAN and outside).

From within docker containers i can ping everythin on LAN and WAN and it doesn't matter if the containers using br0 or host networking (bridge).

 

My guess was that my Gateway/router does not like the fact, that ipvlan uses multiple IP addresses with only one single MAC address.

Therefore I replaces my router with a pfsense box, but the issue still persists.

 

@bonienl Are you using any special kind of router, which can deal with one single mac address having multiple IP addresses?

[Freender]

 

1) Docker custom network type: ipvlan, Host access to custom networks: Disabled

In this configuration I don't have any issues

 

 

2) Docker custom network type: ipvlan Host access to custom networks: Enabled

After a couple of minutes I cannot ping 8.8.8.8 from unraid SSH. I think there is something wrong with ipvlan + shim interface configuration

[blaine07]

Either/any of yall using pfsense or something by chance?

 

EDIT: with PfSense I had a bunch of issues after switching and my fix was 

Maybe something like this is breaking your setup?

Edited May 9, 2022 by blaine07

[Freender]

I use UDMP,

- removed all old macvlan static IPs

- In my 192.168.86.0/24 I changed DHCP range to 192.168.86.2 - 192.168.86.172

- moved all ipvlan containers to new address space 192.168.86.173 - 192.168.86.253 (not managed by DHCP)

 

However this made 0 effect, still having connectivity issues =(

[calvolson]

The shim interface miss-configuration comment looks to be in the right direction. When I disable "Host access to custom networks" in the docker settings this issue stops for me. Running 6.10.0 rc8

[Lilarcor]

I second one

[TraXter]

+1 with current 6.10.2 stable, cannot use ipvlan because network connectivity drops after some mins. No issues with macvlan

[aarontry]

Any workaround? I’m having the same issue where I needed to enable host access to the docker with a static. Im running pihole in docker with a static ip and I want to point my unraid host to it. I’m running the latest 6.10.3. 

[CorneliousJD]

On 5/12/2022 at 11:48 AM, calvolson said:

The shim interface miss-configuration comment looks to be in the right direction. When I disable "Host access to custom networks" in the docker settings this issue stops for me. Running 6.10.0 rc8

 

I assume this is a very similar issue that I am facing, my network is fine for a week of uptime but when running a CA Appdata backup where all containers are stopped then restarted my network goes kaput and everything goes down, I cannot reach github, I cannot ping out from the server itself, etc. 

 

Just adding my experience to the mix!

[tjb_altf4]

 

On 5/12/2022 at 11:48 PM, calvolson said:

The shim interface miss-configuration comment looks to be in the right direction. When I disable "Host access to custom networks" in the docker settings this issue stops for me. Running 6.10.0 rc8

 

I wonder if there are issues due to the shim network itself being macvlan as noted in help section, which is already been known to cause crashes for some (certainly has for me since moving to 6.10).

 

[smaster]

+1 to this being a problem.

The moment I activate: "Host access to custom networks" my router gets confused about where to attribute my static IP and it causes all sorts of issues. I need this setting for some of my services to work and it causing instability is definitely not great.

[smaster]

On 7/6/2022 at 1:37 AM, tjb_altf4 said:

 

 

I wonder if there are issues due to the shim network itself being macvlan as noted in help section, which is already been known to cause crashes for some (certainly has for me since moving to 6.10).

 

Just to confirm that once I change "Docker custom network type" to "ipvlan" everything works fine when having the "Host access to custom networks" set to "Yes".

On "macvlan" however, everything is off. I currently have the 6.10.3 version (latest at the moment).

 

For now, I shall leave it on ipvlan, maybe I'll test it in macvlan in the next version.

Edited July 19, 2022 by sergio.calheno

[CorneliousJD]

I changed from a UniFi USG to a UDM now and my old macvlan setup with a VLAN ID of 10 for docker networks to give my pihole a static IP while still using macvlan and avoiding crashes has now stopped working for some reason... and this ipvlan issue still exists.

This leaves me in a bad spot now 

 

Has this been properly reported as a bug yet? 

[Dephcon]

+1  I recently added a NVMe drive and different RAM to my server and all of a sudden I was getting macvlan traces, so by advise of a community member I switched to ipvlan.

 

I'm having the same issue, with ipvlan and "Host access to custom networks: Enabled" my unraid sever and any "bridge" containers can no longer route externally.

 

It Disabling "Host access to custom networks" seems to have "fixed" the issue, but I'm not sure what I'm losing here.  I recall turning it on for a good reaason

Edited October 26, 2022 by Dephcon

[Dephcon]

On 9/5/2022 at 9:48 AM, CorneliousJD said:

Has this been properly reported as a bug yet? 

 

 

[macmanluke]

Looks like there was never a solution to this mess?

macvlan = network works, box crashes every few days (out of no where, has been stable for ages)

 

ipvlan + allow hosts = messed up network connectivity, stable server

 

ip vlan + no allow hosts = working network, stable server but my reverse proxy does not work if a docker is bridged.

[nik82]

On 3/28/2023 at 11:01 AM, macmanluke said:

Looks like there was never a solution to this mess?

macvlan = network works, box crashes every few days (out of no where, has been stable for ages)

 

ipvlan + allow hosts = messed up network connectivity, stable server

 

ip vlan + no allow hosts = working network, stable server but my reverse proxy does not work if a docker is bridged.

 

I can confirm that this is exactly how it is currently working for me. I find this ridiculous, is a fix being worked on in a new version of Unraid? If so how long until this is released?