VLAN help

[stev067]

Hey guys,

I have moved my Unraid server onto a new network switch, with VLAN + PoE, so that I can add 4 PoE cameras that I will access through a VM on the server, but are otherwise isolated from my network. Since the camera ends are basically ethernet cables dangling outside of my house, what settings can I use on the Unraid side, so that only the VM I have running Blue Iris is able to communicate with the cameras, so that the server isn't exposed. I enabled VLAN in network settings, and created a BR0.1, but I don't know what to do from here.

The way I have it now, I've confirmed I can ping and access a camera from the server, and not from another PC elsewhere on my network, so the cameras are at least currently isolated to this switch.

Thanks I appreciate any advice.

 

Edit: I think I made some progress. Now the camera is not pingable from anywhere on my network, except directly from the VM I want it to be used in. My only concern, is that now the VM's IP is pingable from everywhere. Anyone have experience in this area because I'm really stumbling through it...

 

Edit #2: Ok I think I got it the way it should be now. From other devices on the network, I can ping the VM, and not the camera. From within the VM, I can't ping anything except the camera. Seems safe enough but I would still like some input from an expert. I will likely write up exactly what I did, to have someone check my work, and because I couldn't find a single guide for this.

Edited June 30, 2021 by stev067
progress

[Vr2Io]

Common setup was separate Blue Iris VM and Cameras by VLAN with main network, then only provide remote access from main network to VM by InterVLAN routing or loopback or something .....

 

On 6/30/2021 at 7:53 AM, stev067 said:

Seems safe enough but I would still like some input from an expert. I will likely write up exactly what I did, to have someone check my work, and because I couldn't find a single guide for this.

I am not sure how you implement at all, if you want someone have feedback then you should provide the idea for discussion ( too detail also complicate ), may be you achieve through apply config in Kernal / OS / build-in firewall

 

-------------------------------------

 

BTW, I don't like implement in software level, below are the idea for my setup.

 

- All implement in Switch ACL (MikroTik SwOS), hardware level

- VLAN only use to separate  different network segment, i.e. WAN, LAN_1, LAN_2 etc

- Wireshark docker capture trap packet, I will periodic check that

- All device could limited protect depends on the rule

- Spend lot of time to setup / try / design

 

 

Simple description

 

- ACL rule 1 allow UDP:9 ( WOL ), I use it to count any "hits"

- ACL rule 2 allow UDP:53 to pihole

- ACL rule 3 route other UDP:53 to Wireshark, it capture as record 22-26 ( block DNS lookup other then pihole )

- Other ACL not list

 

Edited July 1, 2021 by Vr2Io

[stev067]

Here are the details of how I did this... The switch is a TL-SG108PE. I'm not certain I did this right. It seems to accomplish what I set out to do, but I am NOT an expert on networking by any means.

Ports 1-4 on the switch are for the cameras. Port 5 is my server. Ports 6-7 are empty but will be general purpose. Port 8 connects to the rest of my network via another switch.

VLAN group 101 contains the 4 camera ports untagged, as well as the server port tagged.

VLAN group 102 contains the server port, the other general ports, and the larger network port untagged.

 

Then here are the PVID settings:

Then on my server, I enabled VLAN in network settings, and created 101 (apparently it was important that the number you create matches the VLAN number from the switch. I wanted the VM to be connecting to the VLAN that only includes the cameras and the server, and not the one with access to the rest of the LAN. Hence, 101 per the 1st image above.

Then on the VM settings, I change the network bridge to br0.101, which I had just created.

Last, in the VM, I set a static IP in Windows, which is different from the rest of my network (ie 192.168.10.X instead of 192.168.1.X). I also changed the IPs of the cameras so they would be on the same group (also 192.168.10.X).

Now the VM and cameras seem to be totally isolated from the rest of the network (including the rest of the Unraid server the VM runs in), but are able to access eachother.

Any comments/pointers?

[Vr2Io]

The switch TL-SG108PE and Cameras part was straightforward, Tag it then connect thr br0.101 .

 

5 hours ago, stev067 said:

Last, in the VM, I set a static IP in Windows, which is different from the rest of my network (ie 192.168.10.X instead of 192.168.1.X). I also changed the IPs of the cameras so they would be on the same group (also 192.168.10.X).

I notice Unraid br0.101 "IPv4 address assignment" were none, but if still pingable by main network to VM, this out of my expectation.

 

5 hours ago, stev067 said:

Now the VM and cameras seem to be totally isolated from the rest of the network (including the rest of the Unraid server the VM runs in), but are able to access eachother.

Then you access VM by VNC ? ( I believe YES )

Suggest you should connect a PC to cameras port and set IP to 192.168.1.X , then try to ping Unraid's IP , Router's IP to confirm truly isolate. 

Edited July 1, 2021 by Vr2Io

[stev067]

9 hours ago, Vr2Io said:

The switch TL-SG108PE and Cameras part was straightforward, Tag it then connect thr br0.101 .

 

I notice Unraid br0.101 "IPv4 address assignment" were none, but if still pingable by main network to VM, this out of my expectation.

 

Then you access VM by VNC ? ( I believe YES )

Suggest you should connect a PC to cameras port and set IP to 192.168.1.X , then try to ping Unraid's IP , Router's IP to confirm truly isolate. 

 

Yep I am only able to access the VM by VNC currently. That might be kind of a problem, because I have no way of viewing the Blue Iris clips, except in VNC, since there is no shared drive and no internet connection. I have to learn more about what you said here:

15 hours ago, Vr2Io said:

then only provide remote access from main network to VM by InterVLAN routing or loopback or something .....

 

[Vr2Io]

5 hours ago, stev067 said:

Yep I am only able to access the VM by VNC currently. That might be kind of a problem, because I have no way of viewing the Blue Iris clips, except in VNC, since there is no shared drive and no internet connection. I have to learn more about what you said here:

That need router support VLAN too, I assume your router not support this.

 

As you have 10G NIC for main network, then onboard NIC could use to connect cameras, no need setting VLAN on bridge (br0), just passthrough the NIC to VM as 2nd NIC, setting in different subnet for cameras network. But this need extra cable and may be far away from POE switch, to overcome this, you could use VLAN to trunk at middle and separate in both end.

Edited July 1, 2021 by Vr2Io

[stev067]

1 minute ago, Vr2Io said:

That need router support VLAN too, I assume your router not support this.

 

As you have 10G NIC for main network, then onboard NIC could use to connect cameras, no need setting VLAN on bridge (br0), just passthrough the NIC to VM as 2nd NIC, setting in different subnet for cameras network.

The 10G NIC actually just goes directly to my other computer which also has 10G NIC. I don't have a 10G switch, so it's just p2p. Also, yes my router probably does not have VLAN. It sounds like a lot of work to un-do all of what I just did, especially since I just bought this VLAN switch for this purpose.

I don't really mind using VNC for interacting with Blue Iris, but it would be nice if I had access to the HDD I'm passing in to that VM, outside of the VM. Any idea if that's possible?

[Vr2Io]

Note.

 

4 minutes ago, stev067 said:

but it would be nice if I had access to the HDD I'm passing in to that VM, outside of the VM. Any idea if that's possible?

Seems you can't do that if VM isolate. 

[stev067]

5 minutes ago, Vr2Io said:

Note.

 

Seems you can't do that if VM isolate. 

Ok so if I change the subnet on the VM and cameras back to match the rest of the network, the VM is no longer isolated. Is there a firewall rule I can specify on the VM that will prevent traffic from the camera ports to the rest of the network besides just the VM?

Edited July 1, 2021 by stev067

[Vr2Io]

21 minutes ago, stev067 said:

It sounds like a lot of work to un-do all of what I just did, especially since I just bought this VLAN switch for this purpose.

That's why I like ACL method, other device in network also have some kind of protect.

 

My switch was CSS326 with 2 10G port, if 2 10G port enough, it could simply your network too. Lowest price ~ USD 130 @ Amazon.

Edited July 1, 2021 by Vr2Io

[Vr2Io]

6 minutes ago, stev067 said:

Is there a firewall rule I can specify on the VM that will prevent traffic from the camera ports to the rest of the network besides just the VM?

The firewall in VM just protect the VM, that means camera port could access rest network.

And I doubt your config really could isolate the network, those ping test unreachable may be because you set both in different subnet only.

Edited July 1, 2021 by Vr2Io

[stev067]

5 minutes ago, Vr2Io said:

The firewall in VM just protect the VM, that means camera port could access rest network.

And I doubt your config really could isolate the network, those ping test unreachable may be because you set both in different subnet only.

No I tested when the cameras and VM were on the same subnet as the rest of the network, and the cameras were unreachable from another PC on the same subnet. Wasn't that the point of the VLAN? My concern is, if the cameras can reach the VM, and the VM can reach the network, can the cameras somehow reach the network via the VM?

Edited July 1, 2021 by stev067

[Vr2Io]

23 minutes ago, stev067 said:

No I tested when the cameras and VM were on the same subnet as the rest of the network, and they were unreachable.

Note and fine.

 

23 minutes ago, stev067 said:

Wasn't that the point of the VLAN?

It depends on case by case.

 

23 minutes ago, stev067 said:

if the cameras can reach the VM, and the VM can reach the network, can the cameras somehow reach the network via the VM?

Yes could be, my idea ( ACL ) as below, so really couldn't hack thr cameras switch port. 

 

c1,c2,c3,c4 <-> ports with ACL ( for example only allow necessary TCP/UDP/MAC to VM ) <-> VM and rest network

 

** Overlook your 10G NIC was twist pair not SFP+ ** 

Edited July 1, 2021 by Vr2Io

[ken-ji]

Ok, we need to make some stuff clear first.
@stev067 you just replaced the switch connecting your Unraid server with the VLAN enabled switch, (POE not important here)

so I assume your network looks like

 

And I think your config looks good - save for using VLAN 102 as the general purpose (stick to VLAN1 unless your router and stuff need this level of control)
so your goal is to limit access between the cameras and blue iris PC vs unraid and and the rest of the network.

 

Judging from your setup, you already cannot reach the cameras and the VM from the rest of the network.

and if the BR0.101 interface does not have an IP address, Unraid will also be unable to reach the VM.

 

You seem to want to access the VM also from the rest of the network and the easiest way (not the securest) is to simply add another NIC to the VM and connect it to BR0

 

Alternatively if you have a router with VLAN support is simply to put the camera in their own VLAN, then program the router to deny access from the cameras to your network, and allow the VM  to connect to the cameras

 

 

[stev067]

10 hours ago, ken-ji said:

Ok, we need to make some stuff clear first.
@stev067 you just replaced the switch connecting your Unraid server with the VLAN enabled switch, (POE not important here)

so I assume your network looks like

 

And I think your config looks good - save for using VLAN 102 as the general purpose (stick to VLAN1 unless your router and stuff need this level of control)
so your goal is to limit access between the cameras and blue iris PC vs unraid and and the rest of the network.

 

Judging from your setup, you already cannot reach the cameras and the VM from the rest of the network.

and if the BR0.101 interface does not have an IP address, Unraid will also be unable to reach the VM.

 

You seem to want to access the VM also from the rest of the network and the easiest way (not the securest) is to simply add another NIC to the VM and connect it to BR0

 

Alternatively if you have a router with VLAN support is simply to put the camera in their own VLAN, then program the router to deny access from the cameras to your network, and allow the VM  to connect to the cameras

 

 

Thanks for taking a look. This networking stuff is all wizardry to me. As for the image, there is another switch between the router and this new switch, but I don't have it set up to do anything special. So the image is accurate besides that. 

This whole VLAN thing has been very frustrating to set up, and I've just disabled it for now. I'm having a hard time understanding the whole Tag vs Untag, vs non-member thing, and PVID settings. I'd rather not replace my router just for this, or add a second NIC. Is there any way I can manage the traffic from within the VM itself? Otherwise, is there some medium level of safety I can accomplish with what I have, while still being able to network with blue iris?

[Vr2Io]

5 hours ago, stev067 said:

Otherwise, is there some medium level of safety I can accomplish with what I have, while still being able to network with blue iris?

I would suggest keep your previous setting, carmars in VLAN br0.101 than add br0 and br0.101 to blue iris VM.

 

- br0 for VM management, file sharing ...

- br0.101 for carmers in different IP ssubnet.

- Apply firewall rule in br0.101 or both.

 

This still better then all stuff in same flat network.

Edited July 2, 2021 by Vr2Io

[Vr2Io]

16 hours ago, ken-ji said:

You seem to want to access the VM also from the rest of the network and the easiest way (not the securest) is to simply add another NIC to the VM and connect it to BR0

Exactly same suggestion as @ken-ji , two vNIC to VM, no extra hardware need.

[stev067]

6 hours ago, Vr2Io said:

I would suggest keep your previous setting, carmars in VLAN br0.101 than add br0 and br0.101 to blue iris VM.

 

- br0 for VM management, file sharing ...

- br0.101 for carmers in different IP ssubnet.

- Apply firewall rule in br0.101 or both.

 

This still better then all stuff in same flat network.

Oh I didn't realize I can add a second virtio adapter on the same VM. This sounds like the way to go. When you say apply firewall rules, you mean within the windows vm? What kind of rules would you suggest?

 

Edit: Ok I got that setup working. I'm really pleased with this. It's basically what I was trying to set up. Thanks both of you for your help. I'm still curious what you recommend for firewall settings, because I don't know what an attacker would be able to do, given this setup.

Edited July 3, 2021 by stev067

[Vr2Io]

3 hours ago, stev067 said:

you mean within the windows vm?

Yes

 

3 hours ago, stev067 said:

what you recommend for firewall settings

In fact no recommend could provide, best should be set as tight as possible, i.e. only allow specific TCP / UDP : port passthrough to br0.101. ( So VM have greatest protection from cameras port )

You could got those info. by cameras communication protocol info. or install Wireshark to capture br0.101 traffic.

 

I use Unifi CAM, so I will ref. below info.

https://help.ui.com/hc/en-us/articles/217875218-UniFi-Video-Ports-Used

Edited July 3, 2021 by Vr2Io

[stev067]

8 hours ago, Vr2Io said:

Yes

 

In fact no recommend could provide, best should be set as tight as possible, i.e. only allow specific TCP / UDP : port passthrough to br0.101. ( So VM have greatest protection from cameras port )

You could got those info. by cameras communication protocol info. or install Wireshark to capture br0.101 traffic.

 

I use Unifi CAM, so I will ref. below info.

https://help.ui.com/hc/en-us/articles/217875218-UniFi-Video-Ports-Used

 

Ok I used wireshark, and it looks like the only communication on br0.101 is between port 1935 of the camera IP, to another single port of the VM IP (both directions). Can I create one rule that allows communication between these 2 ports while blocking everything else on this adapter? Do I have to delete/change any of the existing rules? Maybe you have an example what this looks like in the firewall rules.

[Vr2Io]

5 minutes ago, stev067 said:

 

Can I create one rule that allows communication between these 2 ports while blocking everything else on this adapter? Do I have to delete/change any of the existing rules? Maybe you have an example what this looks like in the firewall rule

As said before,I seldom use software firewall, you need try how to block all and only allow one port pass on br0.101, this wouldn't log you out if any wrong due to management by br0. I think you should disable all default rules, no need delete it.

[stev067]

1 minute ago, Vr2Io said:

As said before,I seldom use software firewall, you need try how to block all and only allow one port pass on br0.101, this wouldn't log you out if any wrong due to management by br0. I think you should disable all default rules, no need delete it.

Alright thanks for all your help. I will research windows firewall rules.

[stev067]

Just wanted to report back. I tried playing with firewall rules, and got frustrated with it. I plugged in an old laptop to one of the VLAN ports, and verified that even when I set the IP to the same subnet as the rest of my network, it does not see anything else there, so it is truly limited to what's plugged into the VLAN. That was really my main concern, and I decided it wasn't worth the time I was spending to try and limit port IO. If someone were to plug into the ethernet cable powering one of my cameras, all they would maybe be able to attack is that VM and nothing else.

[Vr2Io]

That's fine.

 

6 hours ago, stev067 said:

got frustrated with it

That's true, I also spend several days to redesign Switch's ACL for better network monitoring, learn some stuff.