Can't get container to act as gateway in internal network

[xorinzor]

I created an internal network using "docker network create --internal --subnet=10.10.0.0/16 vpn" with the goal to use my VPN docker container as gateway, and have all internet traffic of other containers in this network to flow through this gateway.
 

Unfortunately for whatever reason the container just won't forward the traffic, and I'm not sure why. Tried many different tutorials, stackoverflow posts, etc. but none work. Even  though when I used "--link" instead of the internal network bridge, it worked great. (I should add that I configured the correct gateway IP on the containers, and they can ping eachother just fine too).

Are there any limitations to using the "--internal" parameter?

Tried multiple iptables commands, different openvpn "redirect-gateway" options, but really at a loss right now.

 

The openvpn container (gateway) has the following setup:

eth0 - internal network - 10.10.0.2
eth1 - default bridge - 172.17.0.2
tun0 - vpn connection - <dynamic ip>

 

and the other client only has:

eth0 - internal network - 10.10.0.3

 

Tried using these iptables commands:

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT
iptables -A FORWARD -i eth1 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT

as well as using:

echo 1 > /proc/sys/net/ipv4/ip_forward

Which I confirmed by getting the output, to be sure this got set to "true".

Right now, the containers can ping eachother, and the gateway container has access to the internet (confirmed via the vpn connection by using ipify.org).

The other container is unable to ping 1.1.1.1 or 8.8.8.8 and can only ping 10.10.0.3. Using "ip route get 1.1.1.1" shows the route is set correctly, as it tries to contact the gateway container.

Edited January 1, 2019 by xorinzor

[ken-ji]

did you run the gateway container with the privileged option?

and check your gateway on the client containers... are they pointing to .2 vs .1?

[xorinzor]

10 minutes ago, ken-ji said:

did you run the gateway container with the privileged option?

and check your gateway on the client containers... are they pointing to .2 vs .1?

I double checked just to be sure, it's set to privileged.
The gateway is set correctly (and are set automatically using the post-parameter field executing a script). I confirmed this using "ip route" which shows the default gateway to be configured to 10.10.0.2

Edited January 1, 2019 by xorinzor

[xorinzor]

So it turns out that, creating this network, caused Docker to add the following rules to the iptables on the host, preventing traffic.

 

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
num  target     prot opt source               destination
1    DROP       all  -- !10.10.0.0/16         anywhere
2    DROP       all  --  anywhere            !10.10.0.0/16

Removing these 2 rules caused everything to work as intended.
I presume these got added because of the "--internal" parameter upon network creation.

Interestingly, you'd think that by removing these, the "--internal" parameter would be obsolete, but setting the gateway on the container, to that of the host, doesn't give the container internet access. Setting it back to the VPN container, does (via the VPN).

EDIT: it looks like removing these rules did give containers in the default bridge network the ability to ping containers inside this "internal" vpn network though.. guess more research is needed.

EDIT 2: Looks like all "--internal" does is add these iptables rules if I am to believe a serverfault user.

Edited January 2, 2019 by xorinzor