Dual IP NIC setup in Unraid?


Recommended Posts

Hi All

 

I am trying to setup to separate IP's with different ISP's/Gateways (NIC's) that I can select for my dockers?

eth0
IP: 192.168.0.6/24
Gateway ISP1: 102.168.0.1/24 (Unifi USG3 router)

eth1
IP: 192.168.0.7/24
Gateway ISP2: 192.168.0.18 (Pfsense VM running on Unraid)

 

But after separating them I got into a lot of trouble with my Dockers which sofar have used ISP1 and kept pointing to 192.168.0.6?

Is this possible using the Unraid UI for LAN settings?

 

Current IP configuration:

image.thumb.png.fbb6b498f0585b675b374c670dd07491.png

 

I tried splitting them by disabling bonding and removing eth1 and then adding static IP's to each of the NIC's with different gateways?

That really messed things up, and I had to restore my "network.cfg" on my flash drive from a backup to get things working again

 

Is it the routing tables I need to define?

 

It would also be great to have a secondary gateway definition in case the primary ISP went down? - But that is not a priority

Right now I would just like to get my new ISP setup for UNRAID dockers

 

Thanks

Casperse

 

 

 

Link to comment

Docker networks don't support networks with more than one gateway.

They also don't support having more than one docker network with the same subnet / gateway

 

What you really should have here is a router (even a linux VM) that will do src based policy routing against the two wans so ip group 1 uses ISP1 and other IPs use ISP2 but they all point to the router.

Edited by ken-ji
Link to comment
2 hours ago, ken-ji said:

Docker networks don't support networks with more than one gateway.

They also don't support having more than one docker network with the same subnet / gateway

 

What you really should have here is a router (even a linux VM) that will do src based policy routing against the two wans so ip group 1 uses ISP1 and other IPs use ISP2 but they all point to the router.

Yes only one gateway for the Unraid server (I can manually change it if one of my ISP's goes down.... 

 

Thats fine the Unraid server can have the same gateway to ISP2 (Pfsense VM on Unraid server)

I have created Firewall Aliases that will route selected Host IP traffic through the ISP2

 

I just need to use the two NIC's on the server for two different IP's that I can select on each Docker?

I can see that in the Docker settings I have this:

image.thumb.png.919002c991929791e5ecdf61323adefa.png

But I cant get one docker to use 192.168.0.6 and another to use 192.168.0.7 (Same gateway)

 

Is this also not possible?

 

Br

Casperse

Link to comment

Like I said docker can't have two networks with the same subnet (ip range)

Your best bet is to alter your network such that both ISPs gateways have a different subnet ie 10.0.0.1/24 for ISP1 and 10.0.1.1/24 for ISP2, then use a multi-wan capable router or VM (don't like this option) to merge the two and do balance/failover/source routing/etc. and the pfsense VM is ok, idea but you need more network card ports on your Unraid than you have (I think)

Edited by ken-ji
Link to comment

I have 2 NIC on the MB for Unraid and I have 4 NIC's on the Pfsense VM would that be enough?

Update my Unifi supports 2 x ISP on the USG3 - But I really like all the options I have to do VPN and Alias rules, pfBlockerNG and so much more in PFsense!

(Also looking into having a 2x10G card for the Pfsense when my ISP upgrade their infrastructure, cheapest 10G router you can have :-) 

 

I think I will use the Pfsense with ISP2 only and keep ISP1 for my Unifi and Home

Now I just need to find a way to separate traffic from Dockers in Pfsense by Port traffic? and not IP's.... That should be possible?

Link to comment
  • 2 weeks later...
On 3/14/2020 at 1:47 PM, ken-ji said:

It's probably doable... just not an expert nor a fan of using a VM as a router on the same system as the clients :D

 

Same her but since the router and ISP is only for this server it doesn't really matter if the server is down :D

 

I have run into another problem that I hope you might can answer...

 

The server IP is used and shared by the Docker and the same gateway (subnet) Unraid server IP: 192.168.0.10

There are VM's on the Unraid server with their own IP like 192.168.0.40 on the Br0 (Bridged IP) = 192.168.0.10

 

If I route any traffic through the Pfsense for the server Unraid IP, dockers etc on the 192.168.0.6 it will overrule any traffic coming from my VMs?

and route everything over the rule set for the server IP?

 

So is this only possible to route traffic from my VM's if they have a real physical NIC's that I can use and passthrough to my VM's?

Edited by casperse
Link to comment

One way to do this is to configure Unraid to enable VLANs on your NIC, so it will create an interface like eth0.2/br0.2 (VLAN ID=2)

Then make sure the configuration for the VLAN interface does not have an IP address (Most people assign an IP which may prevent this solution from working)

So the VLAN is also configured and routed by pfSense (ie, the VLAN is a subnet, and pfSense has an IP in that subnet - probably acting as DHCP/DNS server as well). Finally the VM is connected to the VLAN sub-interface eth0.2/br0.2 - it ill get an IP on that VLAN and pfSense can route and filter traffic to and from that IP (or even the VLAN subnet)

 

Link to comment

Not sure if related but as this is the latest Dual NIC thread I'll ask here.

 

I've setup dual NICs in my server.  10.10.0.0/24 and 10.10.3.0/24.  10.10.3 is where all my IOT typed devices live.  I've started moving some dockers there and all is good, they can still talk to each other and the server.

 

But...

 

tdm (the server name) doesn't seem to resolve on my PCs / laptops anymore.  Actually anything not on 10.10.3.0 subnet can't resolve.  Is there any way I can have the 10.10.3.10 advertise a different hostname? Or is there some other setting which I just need to click to resolve it all.  I got around this on my main PC by editing the hosts file, but I really don't want something so manual on all devices

Link to comment
  • 10 months later...

Why it shouldn't be possible to add a secondary ip to an interface.
Command will be

Quote

ip addr add 192.168.198.251/24 dev br0

an ip addr show will show:

Quote

13: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000                                                             
    link/ether d0:50:99:d0:16:1d brd ff:ff:ff:ff:ff:ff                                                                                                 
    inet 192.168.198.249/24 scope global br0                                                                                                          
       valid_lft forever preferred_lft forever                                                                                                                     
    inet 192.168.198.251/24 scope global secondary br0                                                                                                      
       valid_lft forever preferred_lft forever                                                                                                             
    inet6 2a02:8108:48c0:2400:ccb8:1792:831e:92b/64 scope global mngtmpaddr noprefixroute                                                               
       valid_lft forever preferred_lft forever                                                                                                                        
    inet6 fe80::bee3:c157:bef4:7fb5/64 scope link                                                                                                                    
       valid_lft forever preferred_lft forever 

But I don't know how to setup, unraid will remember this

Link to comment
  • 1 month later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.