IP routing does not work properly! Need a direction to look at


Recommended Posts

Hello everyone. I have the problem that my VPN clients cannot access any service on my unraid system that runs over the bridge adapter br0. 

Docker containers and VMs with a separate ip address work just fine. 

As a router I use pfSense firewall as a VM with a passed 4-port NIC. So unraid itself has 2 lan ports. The built in 1 gbe and a second added 10 gbe card. The 10 gbe card is directly connected to my workstation. 

I setup a vpn connection to my local network (10.0.0.1/24) for roughly 4 - 7 devices simultaneously. The connection and access to any other device on my lan works perfectly. The vpn clients are inside the network 10.0.1.0/24.

My problem is as follows. 

From the vpn clients I can ping and traceroute to any device on my lan except the unraid system. I already did a tcpdump on unraid, my pfsense firewall (on two different nics) while pinging from a vpn client to my unraid system e.g. 10.0.1.2 -> 10.0.0.22 I see the icmp requests arriving at the unraid server. But after that nothing happens. 

Trying it the other way around, ping from unraid (10.0.0.22) to vpn client (10.0.1.2) works just fine. The target devices replies to the request. 

 

After identifying the issue I need some help to setup the route. I tried different combinations but my unraid system just don't want to route the packets to the right interface. 

 

I read a lot about routing and route table entries in combination with vlans. I think my problem is pretty similar. 

 

TLDR; 

Unraid (10.0.0.22 | 10.0.0.0/24) won't respond to my vpn clients (10.0.1.2 | 10.0.1.0/24) ping requests from another net address space. 

I already had a conversation about this over at reddit with all the necessary details. (https://www.reddit.com/r/unRAID/comments/bynnc7/unraid_ignores_ping_requests_from_another_subnet/

My route table on the network settings page is all default except one entry 10.0.1.0/24 gw br0.

 

Can someone point me in the direction maybe with a reason why this isn't working?

 

Thanks in advance

 

Link to comment

@bonienl here's the screenshot:

 

Bildschirmfoto 2019-06-13 um 19.53.45.png

 

At the beginning I thought that the default route should handle the request and send a reponse. Because the target is not in the same network it would send the reply to the router.

 

EDIT:

To make the route across restart persistent I added a user script scheduled for startup of the array with this command to add the route: 

ip route add 10.0.1.0/24 dev br0

 

Edited by giusemr
Link to comment

From my experience the bridge label always matches the interface label, eth0 = br0, eth1 = br1, eth2 = br2. I have had so many networking issues like this since the last release of unRAID and I thought upgrading would fix it. For me the main issue is my default gateway changes to the wrong gateway every time I reboot now. My 10 gbe card isn't connected to the internet but it always uses that as the default gateway now. I know I can write a script to add the correct gateway but I would just like to know why. 

Link to comment

@giusemr The reason your unraid system cannot talk to the VPN clients is accdg to your diagnostics is because you have a docker network with that subnet and it will take priority.

br-cb2a9c697938: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.0.1.1  netmask 255.255.255.0  broadcast 10.0.1.255
        ether 02:42:79:17:f9:72  txqueuelen 0  (Ethernet)

not sure why it didn't show in your routing table screenshot.

Also, running the specified ip route command is odd... as you are telling unraid to be able to reach the VPN clients by trying directly on the br0 broadcast domain.

Maybe the odd broadcast domain is helping it to work...

br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.0.22  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::541a:5dff:fe73:4e39  prefixlen 64  scopeid 0x20<link>
        ether 64:00:6a:5d:f2:3e  txqueuelen 1000  (Ethernet)

@bonienl Do you think having the Diagnostics dump the docker networks is also a great idea?

@ucliker A linux system can only have one default gateway unless the gateway metrics is correctly configured or you have detailed routing rules setup - google "policy routing". The way unraid configures the network is that it sequentially enables and configure interfaces, so if you managed to define 2 gateways without defining the metrics, the second gateway will win, unless its down/not working.

  • Upvote 2
Link to comment
6 hours ago, ken-ji said:

@giusemr The reason your unraid system cannot talk to the VPN clients is accdg to your diagnostics is because you have a docker network with that subnet and it will take priority.


br-cb2a9c697938: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.0.1.1  netmask 255.255.255.0  broadcast 10.0.1.255
        ether 02:42:79:17:f9:72  txqueuelen 0  (Ethernet)

not sure why it didn't show in your routing table screenshot.

Also, running the specified ip route command is odd... as you are telling unraid to be able to reach the VPN clients by trying directly on the br0 broadcast domain.

Maybe the odd broadcast domain is helping it to work...


br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.0.22  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::541a:5dff:fe73:4e39  prefixlen 64  scopeid 0x20<link>
        ether 64:00:6a:5d:f2:3e  txqueuelen 1000  (Ethernet)

 

Thanks for your clue. Actually I deleted the route manually while trying to figuring out the problem but I totally forget that it corresponds to a docker network. I pruned my docker networks (the network wasn't in use anyway) and now everything is working, I also deleted the manual ip route command at system startup as the router handles the routing to the vpn clients. Thanks a lot.

 

Can I change the broadcast address in the webgui or can I change it manually? I didn't found the settings regarding this.

Link to comment

@ken-ji I never had to change the metrics in the past, I always left everything as the default (1). I'm just surprised that after all these years of using unRAID that this is happening. Its not creating two default gateways when I reboot its assigning the gateway from eth2 as the default. I guess I will set the metrics and see what happens. 

Edited by ucliker
Link to comment

@bonienl I am not configuring anything. Every time I reboot the systems sets a default gateway like its supposed to. It only sets one but it is the wrong one every time. I then need to delete that one and add the correct one. It using my 10gbe adapter (eth2) which isn't connected to the internet instead of using eth0.

Link to comment

I think adding the correct metrics worked. Since my 10gbe card is directly connected to another card in my workstation unRAID sees that and assumes it needs its own default route. I just set that to metric 2 and my default gateway (192.168.1.1) to metric 1 and it seemed to fix everything. Thanks @bonienl for comment regarding the router setup, that helped. Also thanks @ken-ji for the help with the metrics, the policy-based routing definitely made more sense once I read about it on the Cisco website. I may have had metrics setup prior to upgrading at some point and never changed it back after swapping nic cards or something. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.