casperse Posted March 12, 2020 Share Posted March 12, 2020 Hi All I am trying to setup to separate IP's with different ISP's/Gateways (NIC's) that I can select for my dockers? eth0 IP: 192.168.0.6/24 Gateway ISP1: 102.168.0.1/24 (Unifi USG3 router) eth1 IP: 192.168.0.7/24 Gateway ISP2: 192.168.0.18 (Pfsense VM running on Unraid) But after separating them I got into a lot of trouble with my Dockers which sofar have used ISP1 and kept pointing to 192.168.0.6? Is this possible using the Unraid UI for LAN settings? Current IP configuration: I tried splitting them by disabling bonding and removing eth1 and then adding static IP's to each of the NIC's with different gateways? That really messed things up, and I had to restore my "network.cfg" on my flash drive from a backup to get things working again Is it the routing tables I need to define? It would also be great to have a secondary gateway definition in case the primary ISP went down? - But that is not a priority Right now I would just like to get my new ISP setup for UNRAID dockers Thanks Casperse Quote Link to comment
ken-ji Posted March 13, 2020 Share Posted March 13, 2020 (edited) Docker networks don't support networks with more than one gateway. They also don't support having more than one docker network with the same subnet / gateway What you really should have here is a router (even a linux VM) that will do src based policy routing against the two wans so ip group 1 uses ISP1 and other IPs use ISP2 but they all point to the router. Edited March 13, 2020 by ken-ji Quote Link to comment
casperse Posted March 13, 2020 Author Share Posted March 13, 2020 2 hours ago, ken-ji said: Docker networks don't support networks with more than one gateway. They also don't support having more than one docker network with the same subnet / gateway What you really should have here is a router (even a linux VM) that will do src based policy routing against the two wans so ip group 1 uses ISP1 and other IPs use ISP2 but they all point to the router. Yes only one gateway for the Unraid server (I can manually change it if one of my ISP's goes down.... Thats fine the Unraid server can have the same gateway to ISP2 (Pfsense VM on Unraid server) I have created Firewall Aliases that will route selected Host IP traffic through the ISP2 I just need to use the two NIC's on the server for two different IP's that I can select on each Docker? I can see that in the Docker settings I have this: But I cant get one docker to use 192.168.0.6 and another to use 192.168.0.7 (Same gateway) Is this also not possible? Br Casperse Quote Link to comment
ken-ji Posted March 13, 2020 Share Posted March 13, 2020 (edited) Like I said docker can't have two networks with the same subnet (ip range) Your best bet is to alter your network such that both ISPs gateways have a different subnet ie 10.0.0.1/24 for ISP1 and 10.0.1.1/24 for ISP2, then use a multi-wan capable router or VM (don't like this option) to merge the two and do balance/failover/source routing/etc. and the pfsense VM is ok, idea but you need more network card ports on your Unraid than you have (I think) Edited March 13, 2020 by ken-ji Quote Link to comment
casperse Posted March 14, 2020 Author Share Posted March 14, 2020 I have 2 NIC on the MB for Unraid and I have 4 NIC's on the Pfsense VM would that be enough? Update my Unifi supports 2 x ISP on the USG3 - But I really like all the options I have to do VPN and Alias rules, pfBlockerNG and so much more in PFsense! (Also looking into having a 2x10G card for the Pfsense when my ISP upgrade their infrastructure, cheapest 10G router you can have :-) I think I will use the Pfsense with ISP2 only and keep ISP1 for my Unifi and Home Now I just need to find a way to separate traffic from Dockers in Pfsense by Port traffic? and not IP's.... That should be possible? Quote Link to comment
ken-ji Posted March 14, 2020 Share Posted March 14, 2020 It's probably doable... just not an expert nor a fan of using a VM as a router on the same system as the clients Quote Link to comment
casperse Posted March 26, 2020 Author Share Posted March 26, 2020 (edited) On 3/14/2020 at 1:47 PM, ken-ji said: It's probably doable... just not an expert nor a fan of using a VM as a router on the same system as the clients Same her but since the router and ISP is only for this server it doesn't really matter if the server is down I have run into another problem that I hope you might can answer... The server IP is used and shared by the Docker and the same gateway (subnet) Unraid server IP: 192.168.0.10 There are VM's on the Unraid server with their own IP like 192.168.0.40 on the Br0 (Bridged IP) = 192.168.0.10 If I route any traffic through the Pfsense for the server Unraid IP, dockers etc on the 192.168.0.6 it will overrule any traffic coming from my VMs? and route everything over the rule set for the server IP? So is this only possible to route traffic from my VM's if they have a real physical NIC's that I can use and passthrough to my VM's? Edited March 26, 2020 by casperse Quote Link to comment
ken-ji Posted March 27, 2020 Share Posted March 27, 2020 One way to do this is to configure Unraid to enable VLANs on your NIC, so it will create an interface like eth0.2/br0.2 (VLAN ID=2) Then make sure the configuration for the VLAN interface does not have an IP address (Most people assign an IP which may prevent this solution from working) So the VLAN is also configured and routed by pfSense (ie, the VLAN is a subnet, and pfSense has an IP in that subnet - probably acting as DHCP/DNS server as well). Finally the VM is connected to the VLAN sub-interface eth0.2/br0.2 - it ill get an IP on that VLAN and pfSense can route and filter traffic to and from that IP (or even the VLAN subnet) Quote Link to comment
dalben Posted April 2, 2020 Share Posted April 2, 2020 Not sure if related but as this is the latest Dual NIC thread I'll ask here. I've setup dual NICs in my server. 10.10.0.0/24 and 10.10.3.0/24. 10.10.3 is where all my IOT typed devices live. I've started moving some dockers there and all is good, they can still talk to each other and the server. But... tdm (the server name) doesn't seem to resolve on my PCs / laptops anymore. Actually anything not on 10.10.3.0 subnet can't resolve. Is there any way I can have the 10.10.3.10 advertise a different hostname? Or is there some other setting which I just need to click to resolve it all. I got around this on my main PC by editing the hosts file, but I really don't want something so manual on all devices Quote Link to comment
Gladio Posted February 9, 2021 Share Posted February 9, 2021 Why it shouldn't be possible to add a secondary ip to an interface. Command will be Quote ip addr add 192.168.198.251/24 dev br0 an ip addr show will show: Quote 13: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether d0:50:99:d0:16:1d brd ff:ff:ff:ff:ff:ff inet 192.168.198.249/24 scope global br0 valid_lft forever preferred_lft forever inet 192.168.198.251/24 scope global secondary br0 valid_lft forever preferred_lft forever inet6 2a02:8108:48c0:2400:ccb8:1792:831e:92b/64 scope global mngtmpaddr noprefixroute valid_lft forever preferred_lft forever inet6 fe80::bee3:c157:bef4:7fb5/64 scope link valid_lft forever preferred_lft forever But I don't know how to setup, unraid will remember this Quote Link to comment
Gladio Posted April 5, 2021 Share Posted April 5, 2021 I find a work around to make this semi-permanent Install the plugin: CA User Scripts Here you can implement your own Scripts, which will survive a reboot / reconstruct of the USB-Stick Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.