[6.3.0+] How to setup Dockers without sharing unRAID IP address


ken-ji

Recommended Posts

How to setup Dockers to have own IP address without sharing the host IP address:

This is only valid in unRAID 6.3 series going forward.

6.4.0 has this built into the GUI but currently have a limitation of needing extra IP addresses for each of your interfaces and needs to delete all manually created docker networks. If you don't like that, you can opt to create the network manually like here and disable the docker network auto cleanup.

6.4.1 is now more intelligent about this and presents a relatively powerful UI that covers most simple cases. You don't need to assign unnecessary IP address to additional interfaces anymore.

refer to https://lime-technology.com/forums/topic/62107-network-isolation-in-unraid-64/ for more details

 

 

Single NIC only:

  • Some assumptions:
    We'll be using a shared interface br0 (This allows us to use the same nic with virtual machines, otherwise its alright to use eth0)
    The IP address details are:
    unRAID = 192.168.1.2
    Gateway/router = 192.168.1.1
    Subnet = 192.168.1.0/24
    Docker IP pool = 192.168.1.128/25 (192.168.1.128-254)
    A new docker network will be established called homenet
  • Login via SSH and execute this:
# docker network create \
-o parent=br0 \
--driver macvlan \
--subnet 192.168.1.0/24 \
--ip-range 192.168.1.128/25 \
--gateway 192.168.1.1 \
homenet
 
  • Modify any Docker via the WebUI in Advanced mode
  • Set Network to None
  • Remove any port mappings
  • Fill in the Extra Parameters with: --network homenet
  • Apply and start the docker
  • The docker is assigned an IP from the pool 192.168.1.128 - 192.168.1.254; typically the first docker gets the first IP address
# docker inspect container | grep IPAddress
            "SecondaryIPAddresses": null,
            "IPAddress": "",
                    "IPAddress": "192.168.1.128",
# docker exec container ping www.google.com
PING www.google.com (122.2.129.167): 56 data bytes
64 bytes from 122.2.129.167: seq=0 ttl=57 time=36.842 ms
64 bytes from 122.2.129.167: seq=1 ttl=57 time=36.496 ms
^C
# docker exec container ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2): 56 data bytes
^C
#
  • At this point, your gateway/router will have a first class network citizen with the specified IP address
  • An additional Extra Parameter can be specified to fix the IP address: --ip 192.168.1.128
  • The container will not be allowed to talk to unRAID host due to the underlying security implementation with the macvlan driver used by Docker. This is by design
  • That's it.

 

Secondary NIC is available:

  • Some assumptions:
    We'll be using a dedicated interface br1 (the native eth1 interface can used here too)
    There is no IP address assigned to the interface
    The IP address details are:
    Gateway/router = 10.0.3.1
    Subnet = 10.0.3.0/24
    Docker IP pool = 10.0.3.128/25 (10.0.3.128-254)
    A new docker network will be established called docker1
    unRAID has an ip of 10.0.3.2
  • Login via SSH and execute this:
# docker network create \
-o parent=br1 \
--driver macvlan \
--subnet 10.0.3.0/24 \
--ip-range 10.0.3.128/25 \
--gateway 10.0.3.1 \
docker1
 
  • Modify any Docker via the WebUI in Advanced mode
  • Set Network to None
  • Remove any port mappings
  • Fill in the Extra Parameters with: --network docker1
  • Apply and start the docker
  • The docker is assigned an IP from the pool 10.0.3.128 - 10.0.3.254; typically the first docker gets the first IP address
# docker inspect container | grep IPAddress
            "SecondaryIPAddresses": null,
            "IPAddress": "",
                    "IPAddress": "10.0.3.128",
# docker exec container ping www.google.com
PING www.google.com (122.2.129.167): 56 data bytes
64 bytes from 122.2.129.167: seq=0 ttl=57 time=36.842 ms
64 bytes from 122.2.129.167: seq=1 ttl=57 time=36.496 ms
^C
# docker exec container ping 10.0.3.2
PING 10.0.3.2 (10.0.3.2): 56 data bytes
64 bytes from 10.0.3.2: seq=0 ttl=64 time=0.102 ms
64 bytes from 10.0.3.2: seq=1 ttl=64 time=0.075 ms
64 bytes from 10.0.3.2: seq=2 ttl=64 time=0.065 ms
64 bytes from 10.0.3.2: seq=3 ttl=64 time=0.069 ms
^C
--- 10.0.3.2 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.065/0.077/0.102 ms

  • At this point, your gateway/router will have a first class network citizen with the specified IP address
  • An additional Extra Parameter can be specified to fix the IP address: --ip 10.0.3.128
  • The container can happily talk to unRAID as the packets go out via br1 and talk to the host on br0
  • That's it.

 

Some caveats:

  • With only a single NIC, and no VLAN support on your network, it is impossible for the host unRAID to talk to the containers and vice versa; the macvlan driver specifically prohibits this. This situation prevents a reverse proxy docker from proxying unRAID, but will work with all other containers on the new docker network.
  • We can only have one network defined per gateway. So if you already have docker network br0 on your main LAN (gateway 192.168.1.1), docker will not allow you to create another network referencing the same gateway.
  • I cannot confirm yet what happens in the case of two or more NICs bridged/bonded together (but it should be the same as a single NIC)
  • unRAID 6.4.0 We need to disable the docker network auto generation and cleanup if we want these settings to remain (until the auto cleanup is made more intelligent). The code below should be inserted into the go file before /usr/local/sbin/emhttp is started. The code below will disable docker network auto creation too, so it will behave like 6.3.5. It seems that the docker page will happily show any docker network you have defined, so you can just use them as normal and no longer need to set Extra Parameters. Again not needed with 6.4.1 going forward.
# stop docker network creation and pruning
sed -i -e '/# cleanup/,+3s/^/## /;s/^  custom_networks/## custom_networks/' /etc/rc.d/rc.docker

 

Capture.PNG.eca20b4719650a5a4ff273cb26189830.PNG

 

Edited by ken-ji
Added post 6.4 note to stop docker auto pruning and allow custom netoworks to wrk again.
  • Like 1
  • Upvote 9
Link to comment

Does this need to be done every time unRAID is restarted?

And why 6.3.0+ only? Was it a new undocumented feature or something?

 

I think its permanent (as long as the docker.img is intact) since that's where all the docker related meta data is persisted.

6.3.0 used docker 1.12 which is when the macvlan plugin was released as stable

 

This is quite interesting. I've been looking into network segregation for Dockers in the past but couldn't make it to work properly.

 

I'll have a look at translating your approach to GUI support.

 

Do you have a good reference (URL?) which provides more background information?

 

 

Probably these:

https://github.com/docker/libnetwork/blob/master/docs/macvlan.md

https://docs.docker.com/engine/userguide/networking/get-started-macvlan/

 

  • Upvote 2
Link to comment
  • 2 weeks later...

Brilliant ken-ji!

 

Works a treat and solved some Plex and Crashplan docker issues I was having, I nearly set up new VM's to solve those issues until I found your post.

 

Router is happy and so am I!

 

It seems to help bridge the gap between containers and VM's allowing them to live on the physical network rather than port mapping or NAT'ing.

  • Upvote 1
Link to comment
  • 4 weeks later...

Just starting to play around with this.  Got this working and an IP address allocated from the DHCP pool, using a single NIC.

 

docker run -d --name="plex" --net="none" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "PUID"="99" -e "PGID"="100" -e "VERSION"="plexpass" -v "/mnt/user/movies/":"/movies":rw -v "/mnt/user/tv/":"/tv":rw -v "/mnt/user/music/":"/music":rw -v "/mnt/cache/.appdata/plex":"/config":rw --network nonvpn linuxserver/plex

But if I try to use

 --IP=192.168.0.128

for fix the IP address I get

docker run -d --name="plex" --net="none" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "PUID"="99" -e "PGID"="100" -e "VERSION"="plexpass" -v "/mnt/user/movies/":"/movies":rw -v "/mnt/user/tv/":"/tv":rw -v "/mnt/user/music/":"/music":rw -v "/mnt/cache/.appdata/plex":"/config":rw --network nonvpn --IP 192.168.0.150 linuxserver/plex
unknown flag: --IP

 

EDIT:  Schoolboy error, @Malykai kindly pointed out that it needs to be --ip in lowercase.  I feel kinda stupid now for copy pasta without thinking. 

Edited by CHBMB
Link to comment

@ken-ji This works brilliantly. All my WAN traffic goes over a VPN using pfsense, that however breaks Plex remote access.  Managed to fix it with this macvlan setup.  Thanks man.  I'm going to write a guide on how to do this in pfsense at some point.  Credit to you of course.

Link to comment
9 hours ago, CHBMB said:

Just starting to play around with this.  Got this working and an IP address allocated from the DHCP pool, using a single NIC.

 


docker run -d --name="plex" --net="none" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "PUID"="99" -e "PGID"="100" -e "VERSION"="plexpass" -v "/mnt/user/movies/":"/movies":rw -v "/mnt/user/tv/":"/tv":rw -v "/mnt/user/music/":"/music":rw -v "/mnt/cache/.appdata/plex":"/config":rw --network nonvpn linuxserver/plex

But if I try to use


 --IP=192.168.0.128

for fix the IP address I get


docker run -d --name="plex" --net="none" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "PUID"="99" -e "PGID"="100" -e "VERSION"="plexpass" -v "/mnt/user/movies/":"/movies":rw -v "/mnt/user/tv/":"/tv":rw -v "/mnt/user/music/":"/music":rw -v "/mnt/cache/.appdata/plex":"/config":rw --network nonvpn --IP 192.168.0.150 linuxserver/plex
unknown flag: --IP

 

EDIT:  Schoolboy error, @Malykai kindly pointed out that it needs to be --ip in lowercase.  I feel kinda stupid now for copy pasta without thinking. 

Oops. just noticed now the wrong capitalization in the post. Corrected.

Link to comment
  • 2 weeks later...

I'm using this and it works really well. Everything survives restarts and what not. I'm going to end up using this to point to another port on my nic and have all of that traffic route to a router that vpn's all the traffic. This seems to be useful enough to deserve to be pinned no? or added to the "FAQ for unRAID v6" topic maybe?

Link to comment
[mention=62359]ken-ji[/mention] This works brilliantly. All my WAN traffic goes over a VPN using pfsense, that however breaks Plex remote access.  Managed to fix it with this macvlan setup.  Thanks man.  I'm going to write a guide on how to do this in pfsense at some point.  Credit to you of course.



I must try this out... what you are describing is the exact reason i have plex running in a dedicated vm.. would love to have a docker with a seperate ip address..

I am somewhat reluctant in doing it this way though.. since it is not formally supporter it could break with an update ? Soinds like something that wpuld be great to fit in the gui itself..


Verzonden vanaf mijn iPhone met Tapatalk
Link to comment
11 minutes ago, Helmonder said:

 

 


I must try this out... what you are describing is the exact reason i have plex running in a dedicated vm.. would love to have a docker with a seperate ip address..

I am somewhat reluctant in doing it this way though.. since it is not formally supporter it could break with an update ? Soinds like something that wpuld be great to fit in the gui itself..


Verzonden vanaf mijn iPhone met Tapatalk

 

 

It's supported by docker so unless they deprecate the feature then it shouldn't be a problem.

 

I actually managed to figure out how to do this via pfsense, so I'm not currently using this method, but it worked flawlessly.

Link to comment
On 4/20/2017 at 2:55 AM, CHBMB said:

It's supported by docker so unless they deprecate the feature then it shouldn't be a problem.

 

I actually managed to figure out how to do this via pfsense, so I'm not currently using this method, but it worked flawlessly.

 

I'll be patiently waiting for this guide B|

Link to comment
On 4/21/2017 at 11:32 PM, CHBMB said:

What guide?

Sent from my LG-H815 using Tapatalk
 

 

On 4/3/2017 at 8:09 AM, CHBMB said:

@ken-ji This works brilliantly. All my WAN traffic goes over a VPN using pfsense, that however breaks Plex remote access.  Managed to fix it with this macvlan setup.  Thanks man.  I'm going to write a guide on how to do this in pfsense at some point.  Credit to you of course.

 

Link to comment
3 minutes ago, bonienl said:

Perhaps you would be interested to know that macvlan support is added in the upcoming version of unRAID, it allows you to select additional 'custom' networks from the GUI.

 

Errr, I've kinda changed my approach now, doing it all at the firewall level.  But I did enjoy messing around with the macvlan stuff and I can definitely see how it would be very useful for others.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.