[6.10.0-6.10.2] - Network lost after some days working

Unfortunately looks like there's nothing relevant logged, I assumed the problem occurred between these last lines?

 

Jun 14 08:14:24 vkhpsrv01 root: Fix Common Problems: Warning: Syslog mirrored to flash
Jun 14 12:47:14 vkhpsrv01 webGUI: Successful login user root from 31.4.184.37
Jun 14 19:55:17 vkhpsrv01 webGUI: Successful login user root from 192.168.10.18

 

Yes, the problem occurred between

2 hours ago, JorgeB said:

Jun 14 12:47:14 vkhpsrv01 webGUI: Successful login user root from 31.4.184.37
Jun 14 19:55:17 vkhpsrv01 webGUI: Successful login user root from 192.168.10.18

 

Now, new occurance. Log shows some new registers. 

 

I test I have made is setting up a ping loop from my unraid server to google.com. The result of the ping were Destination Host Unreachable. While the ping was executing I disabled docker service and as soon as docker service was completely disabled, ping started to reach destination. With that it seems that the issue is related to something in docker service network config.

 

I have some dockers on same subnet as my unraid servers. So i have enable host access to custom networks... Perhaps here is the problem... 

 

I have upgrade from 6.10.2 to 6.10.3 stable release.

 

Attached syslog.

 

Thank you for help.

 

syslog_20220614_v2.txt

Edited June 14, 2022 by vvancreij

12 hours ago, vvancreij said:

I have some dockers on same subnet as my unraid servers. So i have enable host access to custom networks... Perhaps here is the problem... 

Possibly, that's something I don't know much about, hopefully someone else can help you.

Thank you for your answer... Now I have the issue narrowed down. I have reading articles and I am going to perform some test related this configs. After them, I will share results and conclusions.

 

In the meanwhile, if someone could help I would be grateful.

 

Thank you.

Hi guys,

 

I am back on track. After investigating and surfing through many articles and performing some tests, finally I was able to fix the incident. Let me share with you all of this and wrap all steps made.

 

First of all, I have to say that I am going to focus on docker network part, as the inicident was cause at this level. Having said that, my initital setup for docker was as follows:

 

• Docker custom network type: macvlan

• Host access to custom networks: Enabled

• IPv4 custom network on interface br0 (optional): "Yes" (Option tick)

• Dockers started with bridge default address and dockers started with bro interface subnet.

 

As described on top of this post, after upgrading from 6.9.2 to 6.10.0, 6.10.1 and 6.10.2 my unraid server was experimenting network issues. To be precise, after some days working as expected I lost completely the connection. There was no webgui, no ssh neither ping. The only solution to recover connection was performing a hard reset. At this point, and after sharing this behaviour in the ocmmunity @JorgeB helped us to detect that the main cause of that issue was related to Docker custom network type driver macvlan.

 

After changing that setup to ipvlan internal network became as stable as was before the upgrade. Unfortunatelly for me, with that setup I started to suffer problems losing connection from my unraid server to internet. After reading and performing some tests I realized that when my conection was lost (no ping through console to google.com) if I stopped docker engine internet connection recovered. 

 

I kept performing test and finally I detected that I when my dockers started in the same subnet as my unraid servers is no network problems appeared despite having docker engine up and running. Having detected that I changed the otpion IPv4 custom network on interface br0 (optional) from "Yes" (ticket) to "No" (unticket) and reconfiguring my dockers to use bridge and custom docker network created by me (using bridge drive). After that, everything is running perfectly fine and smooth. I can tell that I have recovered completely my unraid server.

 

Just a comment regarding custom docker network and bridge. Using bridge for docker, I never find a way to control docker ip assignement (is not specially needed as you can work with ports). However, if you want to control that assignement, you can crete a docker custom network in a specific subnet and the you can control it. The command I use is as follows:

 

docker network create --subnet=x.x.x.x/16 --gateway x.x.x.1 -o com.docker.network.bridge.enable_icc=true -o com.docker.network.bridge.enable_ip_masquerade=true -o com.docker.network.bridge.host_binding_ipv4=0.0.0.0 -o com.docker.network.bridge.name=docker_bridge_name -o com.docker.network.driver.mtu=1500 docker_network_name

 

I found a lot of information here: https://docs.docker.com/network/ipvlan/

Edited June 18, 2022 by vvancreij

Changed Status to Solved

Good to hear you solved your issue, just want to say that I am using custom networks extensively without any problems.

 

Usually these kind of issues are a combination of things, including the network equipment in use.

 

Correct! This is my final conclusion about that issue... Combination of things.

 

I am using docker custom networks too but using the bridge drive to create them. What is important is not to create the custom network or use br interface on the same subnet as your unraid server (using ipvlan network type).I made some tests with ipvdrive, but here I did not found the way to passthrough the dockers dns resolution. Ping work perfectly fine between networks but dns resolution nope.

 

@bonienl, a couple of questions just for a better understanding for me.

 

Are you using macvlan or ipvlan? If you use ipvlan, are you binding docker address into the same subnet as your unraid server?

 

I ask this because with macvlan, I have no problems binding docker into the same network subnet than my unraid server. The problem started after macvlan issue...

 

Thank you!

 

On 6/19/2022 at 3:26 PM, vvancreij said:

What is important is not to create the custom network or use br interface on the same subnet as your unraid server (using ipvlan network type)

I need to run pihole in docker with a static ip on the same network as my unraid server. Is this still possible?

On 6/20/2022 at 1:51 PM, aarontry said:

I need to run pihole in docker with a static ip on the same network as my unraid server. Is this still possible?

 

I answer under my experience. If you are using macvlan network type for dockers I think there is no problem. If you are using ipvlan, my experience as a user says that I have network problems (when using same subnet as unraid server).

 

Why do you need to run a pihole static ip docker on the same subnet as you unraid? I mean, you can run a pihole docker on a custom docker network and add a static route to it. 

Edited August 16, 2022 by vvancreij