Docker network security/isolation with iptables?


DizRD

Recommended Posts

Hi, New unraid user here.

I'm not sure, but I think I have a fairly basic setup of unraid so far, but before I start getting friends and family to use my server, I'm wanting to tighten up security.

 

It seems docker containers are able to talk to each other on my unraid container networks and the unraid host itself.

Here's the advanced network settings from my unraid docker tab:

Host access to custom networks:

Disabled

Preserve user defined networks:

No

 

However, even with those settings a container on a custom network was able to access other containers via host exposed port and the containers were able to reach other ips in my actual lan range.

 

I tried disabling ICC when creating the network to improve isolation, like:

docker network create -o "com.docker.network.bridge.enable_icc"="false" isonet1
docker network create --internal nonet1

 

I eventually got a block working by using iptables directly:

iptables -A INPUT -s 172.18.0.0/24 -d 192.168.1.0/24 -j DROP

<Thanks to: 

>

What I noticed is that stops containers from accessing exposed ports on the unraid host, but doesn't stop local lan access. So a compromised container would allow an attacker to use that container to attack my router or other network devices.

 

So I tried:

iptables -A FORWARD -s 172.18.0.0/24 -d 192.168.1.0/24 -j DROP

Which seems to work, but also kills network access out to the internet.

 

I'm curious if people have suggestions on better iptables commands to block host and lan access but still allow internet access.

This would allow me to create relatively isolated networks for different container groups that I could route to with NPM.

 

Link to comment

The way to make this work properly is to define different custom networks.

A custom network is by design a isolated environment and only members in this custom network can talk to each other.

Also by design docker containers in a custom network can not communicate with the host (Unraid server), though there is a setting to bypass this restriction (not recommended)

 

Custom networks are either assigned to different physical ethernet ports or different VLANs (virtual networks). This ensures traffic between custom networks is never shared, but completely segregated.

 

Link to comment

Your first paragraph was what I was trying to communicate wasn't true in my case.

 

That being said, I didn't setup the custom networks with different physical ethernet ports. Do you have link(s) to share or would searching with "custom network" "physical ethernet ports" be the next best choice?

Link to comment
  • 9 months later...
On 2/23/2022 at 11:55 AM, bonienl said:

Custom networks are either assigned to different physical ethernet ports or different VLANs (virtual networks). This ensures traffic between custom networks is never shared, but completely segregated.

 

 

Sorry for digging up an old post but I'm trying to do exactly this and I can't find any info on how to do it.

How can I assign a custom network to a different physical port?

After creating the network with "docker network create XXXX" I can't find a way to route it through eth1/br1 instead of eth0/br0.

 

On Discord someone suggested to change the iptables but if there was an "easier" and more "permanent" way to do this it would be great.

 

Ty very much for the help! :D

Edited by SggCnn93
Link to comment
On 12/3/2022 at 12:17 AM, SggCnn93 said:

 

Sorry for digging up an old post but I'm trying to do exactly this and I can't find any info on how to do it.

How can I assign a custom network to a different physical port?

After creating the network with "docker network create XXXX" I can't find a way to route it through eth1/br1 instead of eth0/br0.

 

On Discord someone suggested to change the iptables but if there was an "easier" and more "permanent" way to do this it would be great.

 

Ty very much for the help! :D

Hey
I want the same :D

Is there something new?

 

Link to comment

So yea, I had to figure out what Bonienl was talking about.. Maybe there is a better way, but I had to get a Smart switch that supports vlans, and create a VLAN, attach a network adapter to the VLAN port on the switch and then I attached the relevant docker network to the vlan.. It works.. but it's a pain to setup. Networking in Kubernetes would be easier, but I know that's not officially supported. Ultimately with my time in unraid while I love it as a storage device/internal app server, I wouldn't trust the isolation provided by docker and vlans at the moment for public internet facing apps.. But that's just me as a security person. I'm probably just going to setup a fedora server with kubernetes for any public internet facing apps.

Link to comment
  • 4 months later...

hi there,

 

I'm wondering how to secure Dockers with iptables, too.

 

I have nextcloud running through swag like spaceinvador does in his tutorial.

SWAG and nextcloud can reach my internal network and my Windows10 Client directly on any port which is a security issue in my opinion.

 

Because if SWAG or Nextcloud docker gets hacked, then my whole system can get compromised.

 

So I want to block internal traffic between the dockers and isolate the proxynet completely. Is that possible?

 

I have posted screenshots of my setup and a established connection from the nextcloud docker to RDP Port of my Win10 Client which i wan't to block with iptables.

 

I want to only allow the following communication and block all others:

 

SWAG 172.18.0.5 to NextCloud 172.18.0.3 443 + 80
SWAG 172.18.0.5 to Documentserver 172.18.0.4 443 + 80

NextCloud 172.18.0.3 to SWAG 172.18.0.5 443 + 80
Documentserver 172.18.0.4 to SWAG 172.18.0.5 443 + 80

 

I already tried many different commands on iptables but nothing works.

 

Does anybody have an idea and can help?

 

Would be very grateful :)

 

Thanks!

 

 

 

 

Docker_Containers_with_ips.png

fail.png

Edited by diederich89
Link to comment

Dont try and manipulate iptables manually, its complicated and not worth while in general (particularly because docker uses iptables and doesnt play nice with user defined stuff). You can achieve better isolation using docker's networking capabilities, however you will not be able to do so using unRAIDs web interface as it does not support setting up more complex network layouts. You will either have to use something like compose, portainer, or docker cli commands.

What you likely want to do is put is put the nextcloud and mariadb containers on a separate internal docker bridge network with no port mappings. Then your swag container would connect to both your proxynet network and the network that the other containers are on.

Link to comment

Thanks for reply!

 

I did some deeper research and read a little more about docker container.

 

Are the following information correct?

 

1. Docker container on unraid are not executed with root rights per default

2. so if a Docker container gets hacked, nothing can be installed on the docker container and escaping to another container is not possible

3. using docker.socks directly in an container is indeed very dangerous and could lead to hacked unraid hosts and takeover

 

So if you avoid doing Point 3, there is nothing bad about the containers being able to ping each other and the internal network.

 

Right? :)

Link to comment

I am by no means a docker expert but i will try and answer your questions.

4 hours ago, diederich89 said:

1. Docker container on unraid are not executed with root rights per default

I am not certain i know exactly how to answer your question here, as i think it could be refereeing to multiple things. If you are talking about what they call "rootless" containers then no I dont believe that feature is used. If you are talking about privileged containers then yes generally speaking containers are run with privileged mode disabled unless they really need it.

4 hours ago, diederich89 said:

2. so if a Docker container gets hacked, nothing can be installed on the docker container and escaping to another container is not possible

Two questions in one here. If a container were to be hacked then yes something could be installed within the container. Escaping the container and running something on the host system would be much harder. I wont say impossible and unRAID is not meant to be a high security platform, however generally speaking the assumption is that programs cannot break out of a container.

4 hours ago, diederich89 said:

3. using docker.socks directly in an container is indeed very dangerous and could lead to hacked unraid hosts and takeover

Exposing the docker socket inside a container can be risky and is generally not something that should be done. Generally if an application needs to use the docker socket i recommend pairing it with a docker socket proxy that limits what docker features it can access.

  • Like 1
Link to comment
3 hours ago, primeval_god said:

I am by no means a docker expert but i will try and answer your questions.

I am not certain i know exactly how to answer your question here, as i think it could be refereeing to multiple things. If you are talking about what they call "rootless" containers then no I dont believe that feature is used. If you are talking about privileged containers then yes generally speaking containers are run with privileged mode disabled unless they really need it.

Two questions in one here. If a container were to be hacked then yes something could be installed within the container. Escaping the container and running something on the host system would be much harder. I wont say impossible and unRAID is not meant to be a high security platform, however generally speaking the assumption is that programs cannot break out of a container.

Exposing the docker socket inside a container can be risky and is generally not something that should be done. Generally if an application needs to use the docker socket i recommend pairing it with a docker socket proxy that limits what docker features it can access.

Thanks for your fast reply! 

 

So if there is an exploit on nextcloud itself which leads to a rootshell on the docker container "nextcloud", someone could install for example metasploit on that container and can try to attack my Win10 Client in my home network (since it is pingable and connectable). 


That's a thing I can't get out of my head and I really don't like :(

 

Attacker attacks --> https://nextcloud.OWNSERVER --> runs an exploit to gain access to a root shell on nextcloud docker --> install "hacking tools" on nextcloud docker --> attacks my Win10 VM and hacks to RDP Port (which I assume is open in that scenario) 

Link to comment

In general the safest and recommended option is to only expose services hosted on unRAID locally and not directly to the internet. Then use a VPn to connect to your local nextwork and services. Another option is to use vlans in some manner to ensure that your exposed service does not have access to your local network.

Link to comment
  • 1 month later...
On 4/24/2023 at 12:22 AM, diederich89 said:

thanks for your reply, I will read further on how to use the second network adapter on my unraid for different dockers and throw the second network adapter in a vlan.

While a theoretic possibility, the chance they'll break out is low. However a more secure way of hosting something like nextcloud is to do it in a good ol VM. 

 

A VM does not share it's internals with the host and is much harder to break out of. If you do VM's properly it's going to be nearly as secure as running baremetal servers. You can ofc run docker in a VM if you do still want to use the ease of use that comes with docker. 

 

 

Edited by tessuwu
oops wrong name
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.