Network isolation in unRAID 6.4


bonienl

Recommended Posts

On 8/8/2018 at 3:14 PM, bonienl said:

First, when your router has enough LAN ports (usually 4) then no switch is required, which make things easier :)

I have good experience with managed switches from TP-link out of their easy smart series, e.g. their TL-SG108E and TL-SG1016DE models. These have effordable prices and can do what you need.

I  use Ubiquiti routers, these have several features, including VLAN support, but are more targetted at 'prosumers', people with sufficient network knowledge. Good stuff, but you may need to do some learning.

 

I used Netgear routers in the past, but don't remember their exact feature set. Perhaps somebody else has more recent experience with Netgear.

A quick look at the datasheet of Netgear and the features you need are:

IEEE 802.1q VLAN support - this allows an interface to use VLANs and VLAN trunking

Port trunking - another word for port aggregation. Netgear supports LAG protocol statically, which means the 'other' side must also set static LAG without negotation. unRAID supports static LAG (IEEE 802.3ad)

Yeah my router is at one end of the house, and the AP is at the other, so that wouldn't work ha.

I went out and bought a TL-SG108E and now have it connected directly to the router, with unraid, AP and others connected to the switch. I'll have a crack tomorrow at setting up the VLANs in the switch and router.

I've been dabbling deeper and deeper into the networking rabbithole lately, and advice like yours is priceless.

Cheers.

Link to comment

@bonienl one last question: I'm trying to configure my router, however i'm having trouble trying to find the appropriate options in the webui.

As mentioned previously, i'm using a Netgear R7000 running AdvancedTomato firmware. There's network and VLAN menus but i'm not sure about assigning certain networks to a VLAN ID.

I've set up port 3 (which is connected to the switch with unraid), as a tagged VLAN4 port. Is this correct? Although further advanced options as per your tutorial aren't available under VLAN.

I'm sure the option is available somewhere, the verbiage is probably just different, and nestled in another section somewhere.

Would you have any idea how to set this up, or similar setup instructions elsewhere?

network.jpg

vlan.jpg

Link to comment

It looks correct, but

53 minutes ago, Boo-urns said:

I've set up port 3 (which is connected to the switch with unraid), as a tagged VLAN4 port

make sure to set the SG108E to understand that there is VLAN4 and that it should be tagged on the pertinent interfaces,

 

You have to understand the key parts of the terminology.

a network packet can have two states untagged and tagged

  • untagged means its a standard packet in a standard network
  • tagged means there is extra headers labeling the packet. this label tells switches (and devices) that are aware that it belongs to an alternate network (VLAN).

now switches that get a tagged packet check if they know the tag and decide based on settings.

  • security minded switches drop unknown tags (or tags that are in the wrong place)
  • the packet is then sent out associated ports
    • it is left alone if the port says that the VLAN is tagged on the port
    • the tag is stripped if the port says the VLAN is untagged on the port

when a deivce gets the packet, the network stack then checks if the tag is configured or not (subinterfaces ie br0.1) and if it is, it strips the tag and presents it on the subinterface - a packet with VLAN4 tag is received on br0.4; otherwise the packet is ignored.

the inverse happens when you send a packet.- the device generates traffic on br0.4 and the network stack tags it and sends it out of br0, which the switch gets and presents to the associated ports as before 

 

I'm using a mikrotik router, so I define the VLAN subinterface:

image.png.62db64673d2e5e90218afbe81213c511.png

and assign it a subnet 10.0.3.1/24

 

My VLAN table on my switch TP-Link SG2008 is like this:

image.png.fb6cc9d79e271edc0accf3e06a68e91d.png

 

And the docker network:

image.thumb.png.e17aea87a71d1f84adf35170359d0bef.png

 

 

Link to comment

You need at least select the ports to which unRAID and your router are connected as "tagged" on your switch.

 

As explained in the excellent story of @ken-ji tagging those ports creates a separate communication from unRAID to router. AKA an isolated network.

 

On your router configuration of port 3 looks alright. It allows both untagged traffic (br0) and tagged VLAN 4 traffic (br2).

 

You have correctly defined br0 and br2 as different networks and to let them talk to each other, traffic is 'routed' by your router, which allows a container (192.168.4.x) to talk to the server (192.168.1.x) unless you have defined firewall rules on your router which prevents or limits this communication.

 

Edited by bonienl
Link to comment

Thanks @bonienl and @ken-ji for the detailed responses. I believe I have my system set up now...mostly.

My test docker is running off br0.4 and the router reports its IP on LAN2/br2/VLAN4. Success!

I've set up rules in the router for access between LANs for the time being.

Now for the final piece of the puzzle though, the router IP on VLAN4 is now 192.168.4.1 (192.168.1.1 on br0), I put that in the gateway IP in unraid for br0.4. The docker has no internet connection. Using Sonarr as an example, i get "no route to host" errors in the log.

Is there something I need to do specifically to allow internet access to the br0.4 VLAN? I've tried setting the gateway to both 192.168.4.1 and 1.1.

image.thumb.png.a05664293293fa2dc84b900adec1df78.png

image.png.c383c9cc104deeabc4561ce4b79ad0f6.png

image.png.5b8cbdb36dc6f545426f5047d02278b5.png

Edited by Boo-urns
Link to comment

Open the console of the docker from the WebUI

try pinging the router ip from there.

also check the IP of the docker from there with

ip addr

 

if the ping is successful, then check your router rules as it might not be routing properly

Edited by ken-ji
Link to comment
45 minutes ago, ken-ji said:

ok. that's one really bare docker. I'll try to spin it up and have a look later...

I've had the same results on 2 other dockers, running on both the VLAN and bridge. Although DelugeVPN gave a ping result, so I transferred that to the VLAN and got the below results. Can ping other dockers on the VLAN but not the host or gateway. Additionally, the ipp addr are below.

image.png.f8d8b9543a9bcc98557f0c35783d205b.pngimage.png.b8c744cd6fe7408a8fdde3a4bc4fcd48.png

Edited by Boo-urns
Link to comment

Not sure why you can't ping/connect to the gateway yet. but you should not assign an IP to the VLAN interface of unRAID

then go back to the docker settings, and reconfigure the network for the vlan (you need to stop the docker service to do this)

this way all docker/unraid traffic will be via the router.

check your router and make sure that the vlan interface is correctly defined. and try to ping from there to a docker container

 

Link to comment
9 hours ago, ken-ji said:

Not sure why you can't ping/connect to the gateway yet. but you should not assign an IP to the VLAN interface of unRAID

then go back to the docker settings, and reconfigure the network for the vlan (you need to stop the docker service to do this)

this way all docker/unraid traffic will be via the router.

check your router and make sure that the vlan interface is correctly defined. and try to ping from there to a docker container

 

So by assigning an IP to the VLAN interface, i'm assuming you're talking about within settings > network settings. I was under the impression this needed to be defined as per @bonienl's instruction in the tutorial so the docker see's the VLAN as a network choicer. Regardless, I followed your advice and removed the IP from VLAN4(br0.4).

image.png.3720926677a1747323d84d55a06869e8.png

Within the docker settings, i've entered the following for br0.4.

image.thumb.png.1a4c5beaa11736652a64309024968275.png

I get correct IP assignment upon starting the dockers (and the network shows up as a choice within the docker), however I get the same problem, no internet connection from the dockers/no connection to host.

Also I cannot ping the dockers from the router, but i can ping between dockers on the VLAN

Link to comment
25 minutes ago, Boo-urns said:

Also I cannot ping the dockers from the router, but i can ping between dockers on the VLAN

 

This likely means this router itself might be misconfigured with regards to VLAN configurations.

when unRAID had and ip on the VLAN interface, was the router and unRAID able to ping each other using the VLAN IPs?

 

I double checked your previous posts and I realized, that you defined a LAN2 (br2) for the vlan, but what are the member interfaces for this LAN2 interfaces? just port 3?

and we discussed that you have a switch on port 3, before the unRAID server. Did you configure that switch properly? by configuring the VLAN4 to be tagged on the relevant interfaces of the switch?

Link to comment

With Unraid assigned IP of 192.168.4.2 on VLAN4 (in unraid network settings), I cannot ping it from the router, or any other devices on VLAN4. Although a docker on VLAN4 does come up in the device list.

image.png.4058bafecd6bdf7317547c98a122125c.png

I believe I do have the switch set up correctly, the router's port 3 goes to the port 1 of the switch, and port 2 of the switch goes to unraid.

image.png.137169908e683a844cec373e29f92e0a.pngimage.thumb.png.6ed4fbe6276d963654c6653f369538f7.png

 

Link to comment

Odd.

if the docker can be seen from the router with the IP 192.168.4.4, can you ping the docker then?

I'm rather stumped as to why networking isn't working as its usually plug and play...

Does the Access control page allow you to set rules between 192.168.4.1 and 192.168.4.4?

Do you have SSH access to your router? maybe you can try pinging and what not from there...

Link to comment
  • 2 weeks later...
On 2/8/2018 at 12:34 AM, mifronte said:

Installed 6.4.1 and set VLAN interface to no IP for unRAID.  Then configured docker with the appropriate network.  Everything looks good.  The UniFi docker container is in a separate VLAN and unRAID is not available in that VLAN.  Complete segration achieved!  Great job @bonienl!

 

Now, just for curiosity, if I have another VLAN where I wanted to selectively have unRAID available, but with only certain shares and no other services, would that be feasible some time in the future?

 

For example, if I have these shares on my unRAID server:  Sales, Finance, Engineering, Executives.  In the Sales VLAN, unRAID would be available with just the Sales share and so on for the other VLANs.  The management web GUI would not be available nor any other services or ports.  Kind of like having a Docker container that allow you to specify which SMB shares to expose and the permissions allowed.  This way I can run multiple instances of the containers in their own VLAN.

 

Why would I want to do this?  Well I am just looking into the future where I become less trusting of all these "smart devices".  So I can see having a VLAN where these devices are segregated from my main LAN, but I would give them access to just certain shares while unRAID is completely isolated.  Kind of like a Docker container so that if for some reason the container is compromised, my unRAID server is still safe.

 

On 2/8/2018 at 1:39 AM, bonienl said:

Currently unRAID binds its services to all available ports when it starts. What you are looking for is binding to a specific management port only. You could make that a feature request.

 

@mifronte Did you make this a feature request? I'm interested in this as well.

Link to comment

There is a hidden feature you can use (use the latest unRAID version). :)

 

You need to edit the file /config/ident.cfg on your flash device and change BIND_MGT="no" to BIND_MGT="yes"

After the change you need to restart the nginx daemon to make the new setting active.

Login to the console and execute

/etc/rc.d/rc.nginx restart

When the above setting is active, it will limit GUI access to the management interface only (eth0 or br0).

 

Link to comment
  • 8 months later...

kenji, I was able to create a switch port to have multiple tagged VLANs. All fine. Dockers run fine in the new subnet, but can't access the host IP space for older containers that I couldn't switch over yet (by re-doing them, some are home grown)...

 

Would a secondary NIC be a work around to this? Since it would be bound to eth1, and ingress through the switch to the server?

Link to comment
  • 3 months later...

Hi Guys,

 

I've got all my docker containers in a vlan (vlan10) and it is all working great.

 

Only problem is that I running OpenVPN and this has to be on host mode to operate correctly.  Problem i'm having is my LetsEncrypt web server is in the docker vlan and of course it can't talk to the OpenVPN container which is attached to the UNRAID IP.

 

I've read about adding a second NIC and letting routing route to the UNRAID Host so I have added a second network to a USB NIC and put one of my containers onto it.  The container can ping other containers in the docker vlan however i still can't get it to ping the host.

 

Unraid Server is 192.168.1.100

Docker VLAN is tagged VLAN 10 on BR0

New VLAN is untagged (tagged at swtitch VLAN 20) on BR1

 

I'm sure i'm missing something simple.

Link to comment
  • 2 weeks later...
On 8/21/2019 at 8:01 AM, ken-ji said:

Also, if you have VLAN support, your docker network on the vlan is able to talk to unraid.

AFAIK, openVPN works very well with its own dedicated IP ( as long as the docker network is either on a different VLAN, or interface from the Unraid )

FYI @ken-ji I found a post where you said to remove the IP addresses from the Interfaces.  Did this and it worked straight away.

 

Thanks!

Link to comment
  • 1 month later...

has something changed for 6.6.7? I tried this tonight but lost WebUI access (and all access) to my Plex Container. I've been running Plex just fine.  From my secure LAN (VLAN10) desktop I can't ping Plex (192.168.50.3) or Docker (192.168.50.2) but I can ping the unRAID server (192.168.10.69). My router firewall rules are set up so I can ping any subnet from VLAN10. I have an EdgerouterX.

Here is my setup. Please let me know what I'm doing wrong. 

 

Capture.thumb.PNG.ed3889b5ff714f9782cfee78459a8623.PNG

 

Capture2.thumb.PNG.ccb66d31086f90b66052565d7028bc4c.PNG

 

Capture3.thumb.PNG.796fa42f1a8513f9d67f157bfa0810f2.PNG

 

Capture4.PNG.17c583e377958a5d784aa6e261a5b703.PNG

 

Capture5.thumb.PNG.4da2fbbf9ff376144797d5feda9fde51.PNG

 

Capture6.thumb.PNG.13a0f66d227b59192f2479b424951b8b.PNG

 

Capture7.thumb.PNG.1fce3bc20db88da7147f70c34d3f6022.PNG

unRAID is connected to switch port 2. 

 

Capture8.PNG.e378601e9b0417ca07d3304fe0ed97cf.PNG

Edited by adminmat
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.