[6.12] Networking not working correctly after upgrading to 6.12 and changing docker to ipvlan

32 minutes ago, thecode said:

What is the meaning of this parameter?

 

It tells how many interfaces are present and need to be configured

When this line is not present, it is assumed the network.cfg file holds a legacy configuration (pre V6) and it results in a basic network configuration without all the extras which is supported in V6 and later.

 

When modifying the network settings using the GUI, it will automatically add the SYSNICS setting.

 

Since @bonienl mentioned my config is missing "SYSNICS", renamed the network.cfg file and rebooted the server. This doesn't make any problem since the server IPv4 is also reserved on the router DHCP.

 

After reboot now I noticed two changes:

1. br0 interface metric changed from 1 to 1006

2. broadcast address for br0 changed from 0.0.0.0 to 192.168.x.255

I was already suspecting that having both shim-br0 and br0 using the same metric can cause a problem with routing (thous maybe creating the incomplete arp table after server is accessing an internet resource) 

 

Checking my other server (which uses macvlan for now) both br0 and shim-br0 metric is set to 1. I can't change settings on this server since it runs the house now but I assume even with macvlan it should have different metric for br0.

 

For now I did not succeed to reproduce the incomplete arp, before this change it happen within few minutes. I still have the problematic config file, I can try to switch back to it and find the problematic setting if it can help others. I also wonder if it worth switching to macvlan and check if it makes any changes there now.

 

Note: My system doesn't have a network.cfg file now, since I deleted it and did not create any changes from the GUI.

Edited July 14, 2023 by thecode

Quote

After reboot now I noticed two changes:

1. br0 interface metric changed from 1 to 1006

2. broadcast address for br0 changed from 0.0.0.0 to 192.168.x.255

 

This information comes from your DHCP server

 

macvlan / ipvlan setting is for docker, it doesn't change network settings.

 

Quote

For now I did not succeed to reproduce the incomplete arp,

 

An incomplete ARP occurs when the target host (your router in this case) is no longer responding to arp requests from the server.

 

17 minutes ago, bonienl said:

 

This information comes from your DHCP server

 

 

To check it I disabled the DHCP from the GUI (set to static), did not change anything else (the DHCP IP is already offered as the correct static IP), rebooted the server, right after reboot the server is missing the arp for br0.

 

 

 

Edited July 14, 2023 by thecode

2 minutes ago, thecode said:

To check it I disabled the DHCP from the GUI (set to static), did not change anything else (the DHCP IP is already offered as the correct static IP), rebooted the server, right after reboot the server is missing the arp for br0.

 

To me this confirms your router is the source of the problem.

 

This is my routing table

I use static routing with explicit settings for metrics.

Docker uses ipvlan and has "host access" enabled, hence the shim interface

 

 

Doesn't give any problems in my network set up.

 

2 minutes ago, bonienl said:

 

To me this confirms your router is the source of the problem.

 

This doesn't make any sense, how would setting a static IP in unraid (with the same IP) related to the router? If a problem would be when DHCP is used it can explain, but when setting a static IP the router should not have any effect.

 

I have another unraid server which is set to "macvlan" which doesn't suffer from this (with a static IP) and more than 100 network devices and only this server suffer from this problem. I have also tried to setup a linux machine with docker and manually create shim interface (using https://blog.oddbit.com/post/2018-03-12-using-docker-macvlan-networks/) and it did not have any problem. Only this server has the incomplete arp problem and only with ipvlan.

I also can't understand why would a switch/router that has both servers connected to it will be missing the arp for one server. I can capture traffic directly on the router to check.

I am not the only one having this, not sure if this helps but there is a thread about it on reddit:

 

13 minutes ago, bonienl said:

This is my routing table

I use static routing with explicit settings for metrics.

Docker uses ipvlan and has "host access" enabled, hence the shim interface

 

 

Doesn't give any problems in my network set up.

 

There is one difference between this and my config, my config is missing the default route for the shim-br0:

To test my assumption about the interface metric I have manually set the interface metric to 1.

 

With this set to 1 my routing table in the GUI now looks like this:

 

output from route:

 

There may be one little GUI bug since the GUI shows metric as "1" while route shows 0 or 1.

With this setting I think the issue of the incomplete arp is fixed for me. I will let it run for 1-2 days to test.

I see what is happening

When you create a static IP address + gateway without explicitely setting a metric value, then a default route with metric 0 is created for br0. This is indeed wrongly displayed in the GUI as value 1 (a display error).

 

When creating the interface shim-br0, another default route with metric 0 is added, but this fails because two times the same metric is not allowed and hence there is a missing default route for the shim-br0 interface as a result, and this causes communication loss to the outside world whenever the shim-interface is used.

 

The solution is indeed to set a metric value other then zero for the br0 gateway when a static IP assignment is used and "host access" is enabled. Btw this is not Unraid 6.12 specific but is true for earlier versions too.

 

Need to think of a way to make this clear to the user and avoid this situation.

 

Thanks for all the testing.

 

4 minutes ago, bonienl said:

I see what is happening

When you create a static IP address + gateway without explicitely setting a metric value, then a default route with metric 0 is created for br0. This is indeed wrongly displayed in the GUI as value 1 (a display error).

 

When creating the interface shim-br0, another default route with metric 0 is added, but this fails because two times the same metric is not allowed and hence there is a missing default route for the shim-br0 interface as a result, and this causes communication loss to the outside world whenever the shim-interface is used.

 

The solution is indeed to set a metric value other then zero for the br0 gateway when a static IP assignment is used and "host access" is enabled.

 

Need to think of a way to make this clear to the user and avoid this situation.

 

Thanks for all the testing.

 

Many thanks for your helping me debugging this issue. I think that a first step (but not urgent) would be to fix the GUI to reduce confusion. To fixed it, maybe if metric is not explicitly set for br0 it can be automatically set to 1 when the script creates "shim-br0"

 

About making it clear to the user, since the 6.12 Known issues points to your excellent post, I would suggest to edit the post and suggest to first try ipvlan with metric set to 1 for br0.

 

 

And thanks again 👍

I made an update which automatically resolves the issue when the shim interface is created.

This should help anyone with static network settings and "host access" enabled.

 

5 hours ago, bonienl said:

I made an update which automatically resolves the issue when the shim interface is created.

This should help anyone with static network settings and "host access" enabled.

 

This fix is included in 6.12.3, be sure to read the announce post when upgrading:

 

I can confirm the fix works in 6.12.3, I removed the static metric before upgrade and route is set with the correct metric after.

 

Thanks for the efforts resolving this issue.

Changed Status to Solved

Changed Priority to Other