Jump to content
bonienl

Dynamix WireGuard VPN

217 posts in this topic Last Reply

Recommended Posts

On 10/22/2019 at 7:16 PM, mraneri said:

I cannot figure out how to get my phone to see the pi-hole docker container on 10.0.2.3. I have the DNS configured on the client side. 

Setting NAT doesn't seem to change anything either.

I can see other physical nodes on the network, just not the docker assigned a custom IP.

 

 

Screenshot_20191022-130514.thumb.png.58c546e897a7f2af8f90f2af41a332e2.pngScreenshot_20191022-130401.thumb.png.d32a9e4e423604189736730af4e3efdc.pngRoutes.PNG.9bbbc20decc5888d37489619a1c5f24d.PNGVPN.thumb.PNG.513744d4082fad1731a0d41069b53c57.PNG

Is the pihole in your regular (unraid) network?

 

Then you dont need to do anything, ecepct enter ip of dns on client.

Share this post


Link to post
8 minutes ago, nuhll said:

Is the pihole in your regular (unraid) network?

 

Then you dont need to do anything, ecepct enter ip of dns on client.

 

Unraid is 10.0.2.6

The pi-hole container is on br0 with a custom ip of 10.0.2.3.

If I connect through wireguard I can't see 10.0.2.3 to use it as a DNS.

Share this post


Link to post
15 minutes ago, mraneri said:

Unraid is 10.0.2.6

The pi-hole container is on br0 with a custom ip of 10.0.2.3.

If I connect through wireguard I can't see 10.0.2.3 to use it as a DNS.

u dont need to see to use.. xD

 

look some pages earlier i posted my config which works... just differnet ips

 

Share this post


Link to post
33 minutes ago, CrimsonTyphoon said:

Through wireguard, I cannot access dockers that have custom IPs.....

I can. (see some pages prevois)

Edited by nuhll

Share this post


Link to post
42 minutes ago, nuhll said:

u dont need to see to use.. xD

 

look some pages earlier i posted my config which works... just differnet ips

 

You most certainly need to be able to access it to use it. What are you talking about?

 

You haven't added anything to this thread except say that it works for you and you don't know why.

Share this post


Link to post
3 minutes ago, mraneri said:

You most certainly need to be able to access it to use it. What are you talking about?

 

You haven't added anything to this thread except say that it works for you and you don't know why.

Why so pissy? Im just here to help for free.  Youre right it was the other wireguard thread- SORRY! 

 

 

What i meant was that i cant ping the VPN addresses, but it still works.

 

 

Edited by nuhll

Share this post


Link to post
4 minutes ago, nuhll said:

Why so pissy? Im just here to help for free.  Youre right it was the other wireguard thread- SORRY! 

 

 

What i meant was that i cant ping the VPN addresses, but it still works.

 

 

That works. I can use my router as the client DNS just fine, that doesn't solve the problem of not being able to access Dockers with custom IPs. 

 

Share this post


Link to post
2 minutes ago, mraneri said:

That works. I can use my router as the client DNS just fine, that doesn't solve the problem of not being able to access Dockers with custom IPs. 

 

So ill guess, that would explain why it works for me. My guess would be that unraid needs to get a new route.

 

 

Share this post


Link to post
2 hours ago, nuhll said:

I can. (see some pages prevois)

Okay

 

Let me explain my situation and summarize my last few posts...lets get some new light in here!

 

pfsense router - 192.168.8.1 (pfsense also acts as DNS server) (network is 192.168.8.0/24)

unraid - 192.168.8.151

 

br0 - docker's on unraid's IP (192.168.8.151:XXXX) & VMs

br1.15 - dockers with their own IP. br1.15 is linked to VLAN 15, which I can access no problem throughout my LAN, (i.e sabnzbd.domain.com, sonarr.domain.com)

 

Wireguard network: 10.253.0.0/24

WIreguard is set for remote tunnel access (Allow 0.0.0.0/24) and using pfsense as DNS server (192.168.8.1)

 

pfsense has a static route setup:

 

Network               Gateway                         Interface

10.253.0.0/16         unRAID - 192.168.8.151           Vlan_Internal

 

Things that work through wireguard VPN:

  • Access the internet with unraid/pfsense's WAN IP
  • Access my lan (192.168.8.0/24)
  • Access unraid server
  • Access unraid VMs (They have their own IP on 192.168.8.0/24)
  • Access dockers running on unraid (192.168.8.151:XXXX)
  • DNS lookup dockers with there own IP (I.e. sabnzbd.domain.com resolves to 192.168.8.175)
    • This is a host override in pfsense settings; it will not resolve to any other DNS server, only on my network.

 

Things that dont work through wireguard VPN:

  • Access dockers w/ there own IP (I.e. sabnzbd.domain.com or 192.168.8.175 does not work)
  • Ping docker's with there own custom IP (see below for example)

 

I am looking to be able to access my docker's with there own IP through the wireguard VPN connection.

 

Things I have tried / notes:

  • wireguard settings 
    • Nat OFF or Nat ON. Nat OFF nothing works, Nat ON is the behavior above.
  • unraid's routing table:
ROTOCOL	ROUTE				GATEWAY					METRIC	
IPv4		default				192.168.8.1 via br0			1	
IPv4		10.253.0.2			wg0					1	
IPv4		172.17.0.0/16			docker0					1	
IPv4		192.168.8.0/24			br0					1	
IPv4		192.168.122.0/24		virbr0					1	
  • Ping on my local LAN:
$ traceroute 192.168.8.175
 1?: [LOCALHOST]                      pmtu 1500
 1:  sabnzbd.domain.com                                        10.045ms reached
 1:  sabnzbd.domain.com                                       2.332ms reached
     Resume: pmtu 1500 hops 1 back 1
  • Ping through the wireguard tunnel:
$ traceroute 192.168.8.175
 1?: [LOCALHOST]                      pmtu 1280
 1:  10.253.0.1                                           52.842ms
 1:  10.253.0.1                                           63.555ms
 2:  no reply
 3:  no reply
^C
  • Rebooted pfsense and unraid
  • Made sure there were allow rules pfsense
  • Pfsense does not show anything in logs blocking the 10.253.0.0/16 network.

 

So...who's got ideas?

Edited by CrimsonTyphoon
formatting

Share this post


Link to post
12 hours ago, CrimsonTyphoon said:

So...who's got ideas?

It isn't just you. I complicated my network a bit to try and reproduce this, and I'm seeing it too. 

 

I amended the guide to acknowledge this. Still looking for a solution.

 

15 hours ago, nuhll said:

I can. (see some pages prevois)

I'm glad everything is working for you @nuhll, but your network is rather unique :) I'm not sure how we can leverage that into a solution that will work for everybody.

Share this post


Link to post
47 minutes ago, ljm42 said:

It isn't just you. I complicated my network a bit to try and reproduce this, and I'm seeing it too. 

 

I amended the guide to acknowledge this. Still looking for a solution.

 

I'm glad everything is working for you @nuhll, but your network is rather unique :) I'm not sure how we can leverage that into a solution that will work for everybody.

No, i made a mistake.

 

My pihole dns server is working, but its not reachable via VPN..... because ive entered my routers DNS IP (which is obv. reachable) which then relay to the pihole dns (br0)...

 

I think there must be a route added to unraid to get VPN -> unraid -> docker working...? But how, i dont know.

Share this post


Link to post
56 minutes ago, ljm42 said:
13 hours ago, CrimsonTyphoon said:

So...who's got ideas?

It isn't just you. I complicated my network a bit to try and reproduce this, and I'm seeing it too. 

 

I amended the guide to acknowledge this. Still looking for a solution.

Add me to the list.  I have 'Local gateway uses NAT' set to No.  I have the static route in my router between the VPN tunnel and unRAID server.  My peer access type is 'Remote access to LAN.'

 

Directly on the server, I can launch the webGUI for all docker containers on a VLAN with own IP address and in a subnet other than the unRAID subnet.  Through the WireGuard tunnel, none of them are accessible or pingable.

 

I am still experimenting.

Share this post


Link to post
13 minutes ago, Hoopster said:

Add me to the list.

Do you use UPnP or manual port forwarding?

 

Also is bridging enabled for eth0 (it should be)?

Edited by bonienl

Share this post


Link to post
1 minute ago, bonienl said:

Do you use UPnP or manual port forwarding?

Manual port forwarding.  I have UPnP turned off in my USG.

Share this post


Link to post

I should also add that I see this same behavior with OpenVPN.  No docker containers with custom IPs are accessible. 

 

Clearly, this is not a WireGuard issue (not that anyone claimed it is).  I assume it is just a matter of finding the right VPN/Router/Network configuration.

Share this post


Link to post
33 minutes ago, Hoopster said:

I should also add that I see this same behavior with OpenVPN.  No docker containers with custom IPs are accessible. 

 

Clearly, this is not a WireGuard issue (not that anyone claimed it is).  I assume it is just a matter of finding the right VPN/Router/Network configuration.

That is a good data point, thanks!

 

Oh and for the record, I am using UPnP. With bridging (not bonding) on br0.

Edited by ljm42

Share this post


Link to post
41 minutes ago, bonienl said:

Also is bridging enabled for eth0 (it should be)?

Yes, bridging is enabled for eth0.

Share this post


Link to post
16 minutes ago, ljm42 said:

That is a good data point, thanks!

 

Oh and for the record, I am using UPnP

Here is the routing configuration for my OpenVPN server:

 

image.thumb.png.b4a7726f22c57475e45955dda93424e4.png

 

the .1.x subnet is where the server is.

 

Nothing on the .3.x subnet is accessible.  That is my VLAN to which I have docker containers with custom IPs assigned.

 

Interestingly, with NAT enabled I can access the unRAID server; with routing enabled I cannot. 

 

With WireGuard the server and anything on the .1.x subnet is accessible with or without NAT enabled.

Share this post


Link to post

I did (a lot of) testing and made line traces to monitor what is happening exactly.

 

1. If you have docker containers with a custom IP address assigned to br0 (the management interface of Unraid) then containers will never be reachable thru WG. The reason: the tunnel terminates on the host, but docker doesn't allow communication between host (Unraid) and containers. The solution: create a separate interface or VLAN for docker containers

 

2. When the setting "Local gateway uses NAT" is set to YES, it will cause Unraid to use its own LAN address as source for communication to other devices on the LAN network. I am going to change the name of this setting, because it has nothing to do with the local gateway (router). With NAT enabled on the Unraid server, all devices respond directly to Unraid (i.e. not via the default gateway). In my testing, however, the NAT setting causes issues when talking to containers with custom IP addresses. These reply wrongly and consequently are not reachable, but any other device in the LAN works correctly, just not containers.

 

3. When the setting "Local gateway uses NAT" is set to NO, it will cause Unraid to use the WG tunnel address as source for communication. In this case the default gateway (your router) needs a static route added to point tunnel addresses back to the Unraid server. With this set up both docker containers and other devices are reachable.

 

4. I have issues when using UPnP on my router (Ubiquiti) and ended up making manual forwarding port rules instead. User mistake, UPnP is working correctly with Ubiquiti.

 

Edited by bonienl
  • Like 1
  • Thanks 1

Share this post


Link to post

I have made an update to the WireGuard plugin, which allows more selective NAT rules when multiple tunnels exist.

 

You need to re-apply all existing tunnel configurations to change the settings.

Afterwards a reboot is required unless you know how to manipulate iptables manually (a crime).

 

Share this post


Link to post
On 10/21/2019 at 1:20 PM, nuhll said:

Can you explain me why i dont need any new routes and it just works out of the box without any changes? U said it doesnt know a way back... but how is the requestet data (e.g. webtraffic) routed back to the mobile... then?

 

Also why is my (br0) pihole working, also without any extra rules...?

Your set up is working because you configured your clients to use the gateway as DNS forwarder, however this solution renders pi-hole useless because of the DNS caching your gateway will do.

Share this post


Link to post
9 hours ago, bonienl said:

1. If you have docker containers with a custom IP address assigned to br0 (the management interface of Unraid) then containers will never be reachable thru WG. The reason: the tunnel terminates on the host, but docker doesn't allow communication between host (Unraid) and containers. The solution: create a separate interface or VLAN for docker containers

My docker containers with custom IP addresses are on br0.3 (192.168.3.x subnet).  None are on br0 (if you recall, br0 custom IP addresses caused call traces with my hardware).

 

A couple of examples

image.png.5312f4bbf3145d661b9bbea15d8f18ea.png

 

The WireGuard peer configuration shows the following Allowed IPs:  AllowedIPs=192.168.1.0/24, 192.168.3.0/24, 10.253.0.1/32

 

I have updated to the latest version of the WireGuard plugin.

 

9 hours ago, bonienl said:

3. When the setting "Local gateway uses NAT" is set to NO, it will cause Unraid to use the WG tunnel address as source for communication. In this case the default gateway (your router) needs a static route added to point tunnel addresses back to the Unraid server. With this set up both docker containers and other devices are reachable.

Local server uses NAT (new wording) is set to No.  There is a static route in my USG router from 10.253.0.0/24 to 192.168.1.10 (unRAID server IP)

Note: in your example earlier in this thread you had the Netmask as /16. I oirginally set it this way in the route.  The updated plugin prompt reads: "Remark: configure your router with a static route of 10.253.0.0/24 to 192.168.1.10" so I have now changed it to /24.

 

image.png.f05aea25cff83410c0bf11436b423d2c.png

 

9 hours ago, bonienl said:

4. I have issues when using UPnP on my router (Ubiquiti) and ended up making manual forwarding port rules instead. User mistake, UPnP is working correctly with Ubiquiti.

I have UPnP disabled and am using manual port forwarding.  Does it matter either way?

 

5 hours ago, bonienl said:

I have made an update to the WireGuard plugin, which allows more selective NAT rules when multiple tunnels exist.

 

You need to re-apply all existing tunnel configurations to change the settings.

Afterwards a reboot is required unless you know how to manipulate iptables manually (a crime).

I have reapplied existing configuration.  I have rebooted the server.

 

I still have no access to docker container UIs on the .3.x VLAN.

 

Traceroute on 192.168.3.101 (HandBrake container)

image.png.3fffc9d0b97501b41cb553c09ed15169.png

 Issue here?

 

Docker VLAN configuration

image.png.866190a8166b70ffcd727129fe1e31a4.png

 

Did I miss something or do I have something improperly configured?

 

Whether or not I get this to work, I still love the WireGuard implementation and the work you have done on this.  Much appreciated!

Share this post


Link to post
39 minutes ago, Hoopster said:

Whether or not I get this to work,

It should work! I have a similar set up with containers on separate VLANs and interfaces and these are reachable (I just tried Heimdall which for me is on br0.5).

 

I'll have a look at your settings.

 

Q: what do you use as peer device (sorry if already mentioned)?

 

Q: Can you show the routing table of the gateway (you need to ssh into the USG). Below is mine (LAN = eth0, WAN = eth2)

admin@Blaster:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       I - ISIS, B - BGP, > - selected route, * - FIB route

S>* 0.0.0.0/0 [1/0] via xxx.xxx.xxx.xxx, eth2
C>* 10.0.101.0/24 is directly connected, eth0
C>* 10.0.102.0/24 is directly connected, eth0.2
C>* 10.0.103.0/24 is directly connected, eth0.3
C>* 10.0.104.0/24 is directly connected, eth0.4
C>* 10.0.105.0/24 is directly connected, eth0.5
C>* 10.0.106.0/24 is directly connected, eth0.6
S>* 10.253.0.0/16 [1/0] via 10.0.101.5, eth0
C>* 127.0.0.0/8 is directly connected, lo

 

Edited by bonienl

Share this post


Link to post
17 minutes ago, bonienl said:

It should work!

Agreed! It is obviously something on the router side of things that perhaps I have misconfigured.  Since both OpenVPN and WireGuard behave the same way with respect to the 192.168.3.x subnet, even though both are configured to allow it, I must have some sort of routing issue.

 

I appreciate you taking a look at my config and perhaps it will help someone else as well.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.