giusemr Posted June 13, 2019 Share Posted June 13, 2019 Hello everyone. I have the problem that my VPN clients cannot access any service on my unraid system that runs over the bridge adapter br0. Docker containers and VMs with a separate ip address work just fine. As a router I use pfSense firewall as a VM with a passed 4-port NIC. So unraid itself has 2 lan ports. The built in 1 gbe and a second added 10 gbe card. The 10 gbe card is directly connected to my workstation. I setup a vpn connection to my local network (10.0.0.1/24) for roughly 4 - 7 devices simultaneously. The connection and access to any other device on my lan works perfectly. The vpn clients are inside the network 10.0.1.0/24. My problem is as follows. From the vpn clients I can ping and traceroute to any device on my lan except the unraid system. I already did a tcpdump on unraid, my pfsense firewall (on two different nics) while pinging from a vpn client to my unraid system e.g. 10.0.1.2 -> 10.0.0.22 I see the icmp requests arriving at the unraid server. But after that nothing happens. Trying it the other way around, ping from unraid (10.0.0.22) to vpn client (10.0.1.2) works just fine. The target devices replies to the request. After identifying the issue I need some help to setup the route. I tried different combinations but my unraid system just don't want to route the packets to the right interface. I read a lot about routing and route table entries in combination with vlans. I think my problem is pretty similar. TLDR; Unraid (10.0.0.22 | 10.0.0.0/24) won't respond to my vpn clients (10.0.1.2 | 10.0.1.0/24) ping requests from another net address space. I already had a conversation about this over at reddit with all the necessary details. (https://www.reddit.com/r/unRAID/comments/bynnc7/unraid_ignores_ping_requests_from_another_subnet/) My route table on the network settings page is all default except one entry 10.0.1.0/24 gw br0. Can someone point me in the direction maybe with a reason why this isn't working? Thanks in advance Quote Link to comment
bonienl Posted June 13, 2019 Share Posted June 13, 2019 Post a screenshot of the routing table (see Settings -> Network Settings) Quote Link to comment
giusemr Posted June 13, 2019 Author Share Posted June 13, 2019 (edited) @bonienl here's the screenshot: At the beginning I thought that the default route should handle the request and send a reponse. Because the target is not in the same network it would send the reply to the router. EDIT: To make the route across restart persistent I added a user script scheduled for startup of the array with this command to add the route: ip route add 10.0.1.0/24 dev br0 Edited June 13, 2019 by giusemr Quote Link to comment
bonienl Posted June 13, 2019 Share Posted June 13, 2019 The route you added should not be necessary, the default route should take care of this. I expected to see br1 and not br3. Can you post the full diagnostics (see Tools -> Diagnostics). Quote Link to comment
giusemr Posted June 13, 2019 Author Share Posted June 13, 2019 I also thought about this. Maybe the gateway is br3 because the 10gbe nic is labelled as eth3. server-diagnostics-20190613-1836.zip Quote Link to comment
ucliker Posted June 17, 2019 Share Posted June 17, 2019 From my experience the bridge label always matches the interface label, eth0 = br0, eth1 = br1, eth2 = br2. I have had so many networking issues like this since the last release of unRAID and I thought upgrading would fix it. For me the main issue is my default gateway changes to the wrong gateway every time I reboot now. My 10 gbe card isn't connected to the internet but it always uses that as the default gateway now. I know I can write a script to add the correct gateway but I would just like to know why. Quote Link to comment
ken-ji Posted June 18, 2019 Share Posted June 18, 2019 @giusemr The reason your unraid system cannot talk to the VPN clients is accdg to your diagnostics is because you have a docker network with that subnet and it will take priority. br-cb2a9c697938: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 10.0.1.1 netmask 255.255.255.0 broadcast 10.0.1.255 ether 02:42:79:17:f9:72 txqueuelen 0 (Ethernet) not sure why it didn't show in your routing table screenshot. Also, running the specified ip route command is odd... as you are telling unraid to be able to reach the VPN clients by trying directly on the br0 broadcast domain. Maybe the odd broadcast domain is helping it to work... br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.0.22 netmask 255.255.255.0 broadcast 0.0.0.0 inet6 fe80::541a:5dff:fe73:4e39 prefixlen 64 scopeid 0x20<link> ether 64:00:6a:5d:f2:3e txqueuelen 1000 (Ethernet) @bonienl Do you think having the Diagnostics dump the docker networks is also a great idea? @ucliker A linux system can only have one default gateway unless the gateway metrics is correctly configured or you have detailed routing rules setup - google "policy routing". The way unraid configures the network is that it sequentially enables and configure interfaces, so if you managed to define 2 gateways without defining the metrics, the second gateway will win, unless its down/not working. 2 Quote Link to comment
bonienl Posted June 18, 2019 Share Posted June 18, 2019 3 hours ago, ken-ji said: Do you think having the Diagnostics dump the docker networks is also a great idea? It could be added, though this is a corner case situation, but troubleshooting can include such cases too ... Quote Link to comment
giusemr Posted June 18, 2019 Author Share Posted June 18, 2019 6 hours ago, ken-ji said: @giusemr The reason your unraid system cannot talk to the VPN clients is accdg to your diagnostics is because you have a docker network with that subnet and it will take priority. br-cb2a9c697938: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 inet 10.0.1.1 netmask 255.255.255.0 broadcast 10.0.1.255 ether 02:42:79:17:f9:72 txqueuelen 0 (Ethernet) not sure why it didn't show in your routing table screenshot. Also, running the specified ip route command is odd... as you are telling unraid to be able to reach the VPN clients by trying directly on the br0 broadcast domain. Maybe the odd broadcast domain is helping it to work... br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.0.22 netmask 255.255.255.0 broadcast 0.0.0.0 inet6 fe80::541a:5dff:fe73:4e39 prefixlen 64 scopeid 0x20<link> ether 64:00:6a:5d:f2:3e txqueuelen 1000 (Ethernet) Thanks for your clue. Actually I deleted the route manually while trying to figuring out the problem but I totally forget that it corresponds to a docker network. I pruned my docker networks (the network wasn't in use anyway) and now everything is working, I also deleted the manual ip route command at system startup as the router handles the routing to the vpn clients. Thanks a lot. Can I change the broadcast address in the webgui or can I change it manually? I didn't found the settings regarding this. Quote Link to comment
ucliker Posted June 18, 2019 Share Posted June 18, 2019 (edited) @ken-ji I never had to change the metrics in the past, I always left everything as the default (1). I'm just surprised that after all these years of using unRAID that this is happening. Its not creating two default gateways when I reboot its assigning the gateway from eth2 as the default. I guess I will set the metrics and see what happens. Edited June 18, 2019 by ucliker Quote Link to comment
bonienl Posted June 18, 2019 Share Posted June 18, 2019 Changes are made in the kernel resulting in different network behavior. In general do not configure more than one default gateway unless your network environment (router) is set up such that different gateways can be used. 1 Quote Link to comment
ucliker Posted June 18, 2019 Share Posted June 18, 2019 @bonienl I am not configuring anything. Every time I reboot the systems sets a default gateway like its supposed to. It only sets one but it is the wrong one every time. I then need to delete that one and add the correct one. It using my 10gbe adapter (eth2) which isn't connected to the internet instead of using eth0. Quote Link to comment
ucliker Posted June 18, 2019 Share Posted June 18, 2019 I think adding the correct metrics worked. Since my 10gbe card is directly connected to another card in my workstation unRAID sees that and assumes it needs its own default route. I just set that to metric 2 and my default gateway (192.168.1.1) to metric 1 and it seemed to fix everything. Thanks @bonienl for comment regarding the router setup, that helped. Also thanks @ken-ji for the help with the metrics, the policy-based routing definitely made more sense once I read about it on the Cisco website. I may have had metrics setup prior to upgrading at some point and never changed it back after swapping nic cards or something. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.