herringms

I spent some time on my flight today outlining what I think might work. Unfortunately, I feel like without a firewall in front of the PfSense, you still have a security risk in the event of a configuration clear as the unraid host could become available through APIPA. I wouldn't be too worried, but the conditions here are that we may trust the infrastructure and topology, but not the occupants of the network. ideally, you want something that makes the host inaccessible in the event the configuration is cleared, so I'm suggesting you must have a layer 2 vlan that's a single step away from PfSense, between the ISP modem and the PfSense host, to make sure the unraid host does not become addressable. I have to configs, one as three routers in a y config, and one is a single router. The single router config I'm less sure of without testing, but would look something like the following: Physical topology ISP Gateway -> [optional: additional layer2 device on VLAN 10 ->] PfSense box eth0[.10] (passthrough) PfSense box eth0.20 & eth1.30 (bridge) -> (LAN)Switch Virtual Connections: Interfaces PfSense outside (eth0) (Statically assigned, no upstream DHCP) OR PfSense outside (eth0.10) (DHCP) PfSense outside (eth0[.10]) -> eth1.20 PfSense outside (eth0[.10]) -> eth1.30 Unraid Config VLANs 10 20 30 Passthrough eth0 Bridge eth1 Inside network is safe if config is cleared because PfSense won't route Unraid becomes exposed if DHCP is served from modem and no firewall, or if there is no Required VLAN for eth0 to get out Putting a hardware device that can serve a VLAN in front of the PfSense seems the only safe way to protect this from a configuration wipe. I'm looking into a $50 TPLink that may be a good candidate Pfsense config (Statically set, no upstream DHCP) outside: public address info inside: 172.18.11.1/24 firewall, block between the two, block to unraid where desired Pfsense config (DHCP) [Additional layer2 device with VLAN ports required] VLAN 10 configured outside: eth0.10 with DHCP (LAN) inside: 172.18.11.1/24 opt int 2 config inside: 172.18.20.1/24 opt int 3 config inside: 172.18.30.1/24 If this is looking good to others, I'll test it when I get home in a week, and write up a guide for PFSense as a border router on a 2 NIC Unraid host

All great points! I needed to think about them for a while, but don't think I can really solve for those problems. I'm going to amend the disclaimer as this is not secure, but rather a proof of concept. I can add a note about the 172 addresses, but in my model I stepped up to 172.17/24s to avoid most conflicts. I was more concerned about conflicts with SD WANs on 10/8 (This is one of the places I expect to use this config), and residential devices on 192/24s. I believe avoiding difficult to troubleshoot routing issues is going to be much more important that confusion over subnets in the 172 range, as this has to be a technical and complex process by nature. For the second, you're right eth0 is the way to go. I'm not using this as an actual edge device and probably wouldn't recommend it I don't list steps in the guide to lock down Unraid from the parent network, and don't think I'd want to. BUT I should mention that. For me, currently, it's about portability of my containers, VMs, and Unraid in general. it was also about recovering a few devices without wiping them that had hard set VLANs and SSIDs. We're inherently needing to put a ton of trust in the upstream as it is. I'm reminded of Steve Gibson's y configuration for routers (https://pcper.com/2016/08/steve-gibsons-three-router-solution-to-iot-insecurity/), and am wondering if it would be worth 3 virtual routers with eth1 and eth0 being available selectively to each router would be a good solution. However, I feel like if you're looking for your PfSense to be the primary router. I'd need to really plan that out.

Greetings all, I saw some previous posts about this and users having difficulties getting this set up. ex: https://forums.unraid.net/topic/115277-how-to-setup-pfsense-vmware-on-unraid-os-with-2-nic-ports I'll start by saying I'm not a Linux guy, but have some basic networking and troubleshooting experience. This is a pretty advanced process no matter who you are, and I doubt many people will get through it without a mistake or two. I'm open to recommendations and corrections, and I may be able to help with some mild questions, especially asking for clarification, but will probably ignore anything too complicated. Disclaimer: Do not attempt this if you do not understand the networking concepts being used. Disclaimer 2: The firewall rules are just what I feel comfortable with and not a recommendation. Think of them as sample code. You should plan, implement and test your own rules to make sure they meet your expectations and requirements. Border Router Disclaimer: If you want PfSense to be your border router, this configuration is insecure. We're inherently putting a lot of trust in the upstream. This guide also allows Unraid to be available from the parent network. I may work on a future guide that offers a more secure configuration for border routers, but as is, this configuration should not be considered secure. It is, perfectly possible to be made secure, My guide would just focus on making the entire "secure border router" configuration more resilient. The challenge: With only two NICs, you need to keep one of them bridged and virtually available so you don't risk taking the entire OS offline and going into some form of recovery. There is not a clear consensus on the best, or even a working method for doing this with a FreeBSD custom kernel like PfSense. My specific case: I'm transitorily homeless and wanted to be able to configure my own VLANs, networks, wireless broadcasts firewall rules, DHCP with DNS options, Pi-hole server, etc. behind my parents, very limited, AT&T gateway. Edge Routers are unavailable and many alternatives don't get the jobs done or are too expensive. I've confirmed this is working exactly as I want it to for provided different DHCP to Docker, Guest, IOT, Main and Parent networks without internetwork connectivity except where I want it. This works via hardwire, direct on Docker/VM assignment, and from the wireless broadcast on my Unifi APs (make sure to configure VLANs within the controller if you use these). General Purpose: Your server only has two NICs, and you'd like a virtualized router to handle VLANs, firewall rules, routes. Provides internet from within another network by masquerade, and the PfSense router can receive a DHCP address on the outside. Conventions used and terms: Parent Network: This is the immediate upstream network. In the case of this configuration it is a semi trusted residential network. VLAN_MAIN : ID - 11 VLAN_DOCKERSANDVMS : ID - 12 VLAN_GUEST : ID - 76 VLAN_IOT : ID - 107 Address space for my VLANs: several subnets in the 172.17.0.0/16 range. Please note that Unraid by default uses some 172.16.0.0/12 addresses and there could be potential for a subnet conflict if you aren't careful All_VLANs: and interface group containing each VLAN. VLANs are configured in the Unraid host, not on PfSense Firewall Rules documentation syntax - Description: "A Description": Int "interfaceOrGroupName" sounce network "network Name" dest address "address name" -port portNumber -allowOrDeny Guide: Setting up an Unraid device with only 2 NICs to be PfSense capable Configure Unraid, configuring the NIC: 1) Disable VMs and Dockers temporarily via settings within Unraid. 2) Remove Bonds and bridges from the NIC you would like to passthrough and And "Bind it to VFIO at boot" via the tools -> device settings menu(see pic) 3) Reboot, then configure your remaining NIC from "Settings -> Network Settings". Name it and enable bridging 4) Set a Static IP or DHCP based on your preference. This is your Unraid server's address on the parent network. 5) Create preferred VLANs. Only assign an IP if you want your host accessible on that network Configure Unraid, configuring the PfSense VM: I'm going to keep this section abbreviated, focusing on making your NICs and HDD available through the VM config 6) Create a PfSense VM with your preferred resources a. Your vDisk BUS needs to be set to SATA b. Your LAN NIC should be br0 and model e1000 c. Add a Mac address, source bridge, and model of e1000 for each VLAN and the LAN (LAN will be for managing PfSense) d. Add your WAN NIC that you enabled passthrough in by clicking it under "Other PCI devices" and create the VM 7) Install PfSense and configure it to your preferences. Configure the PfSense to make the GUI available: I'm going to keep this section abbreviated, focusing only on the necessary steps to get to the GUI After the install completes, open the Unraid VM's console and run through the wizard. If you messed up, or were unsure, follow these steps when you get back to the menu. 9) Press 1 to "Assign interfaces". You can verify the Mac address in the VM config within Unraid (you will be able to name them later) (see pic) a. WAN is the pass-through interface. It should not be virtualized. b. LAN will be one of the em interfaces. c. You can assign the remaining em interfaces to opts (These will be your VLANs), or wait til you are in the GUI. 10) Press 2 to set the IP address. You only need to set the WAN, and LAN network for now a. Assign the WAN (pass-through) interface to receive DHCP b. Assign the LAN interface a static private IP, with no DHCP server, on it's own subnet. c. The LAN interface is where the management GUI for PfSense will be available and you should receive a confirmation from the console it is now available. 11) Give your computer or management device a second IP on the LAN's subnet and navigate to the IP you assigned within the browser. 12) Go through the basic config in the web GUI, you can skip most steps, we'll do them manually. Completing the PfSense configuration in the GUI: 13) Go to "Interface -> Assignments" and verify all interfaces (WAN, LAN, vLANs) are created and assigned correctly. Do not create VLANs on PfSense. 14) You can rename them by clicking on them. This is highly recommended for your sanity and should be consistent with your descriptions in Unraid 15) Under "Interface Groups" add a group for all of your VLAN interfaces to simplify firewall rules later 16) Go to "Services -> DHCP Server" and add your range, DNS server and Gateway IP, and lease time. Add a domain if needed. 17) Go to System -> Routing, and verify the WAN gateway is configured correctly (See pic, mine is receiving DHCP) 18) The default gateway should be that of the IP of the gateway getting your Unraid OS (parent network) to the internet. 19) Verify your PfSense VM has internet by pinging 8.8.8.8 from the "PfSense console" within the "Unraid VMs console". Configure the firewall: After you've confirmed access to the internet, only the LAN interface (used for management) will have access to the internet until you create additional firewall rules 20) Plan your firewall rules so you don't take down your network. All rules can be created on the ALL_VLANs group for a base setup. As your rules will differ from mine, here are some important notes: All networks other than the one declared LAN do not have an inside to outside rule You'll need rules to restrict communication between each VLAN via layer 3 You'll need rules to prevent VLANs from accessing the parent network but still allow them to get out. With my four VLANs (Main, DockersAndVMs, Guest, IOT) it took me 11 firewall rules. Depending on your use case, this may vary. My firewall rules: 1) Description: "Default allow LANs to internet": Int "All_VLANs" sounce network "172.17.0.0/16" dest address "WAN Address" -allow * Note, Since you can't target interface groups in source, I specified all of the 172 space, where my VLANs exist. You could also specify the entire 172 range 172.16.0.0/12 2.1) Optional: Description: "Allow access PfSense management from main": Int "All_VLANs" sounce network "VLAN_MAIN" dest address "VLAN_MAIN address" -port 22,80,443 -allow 2.2) Optional: Description: "Block access PfSense management from VLAN": Int "All_VLANs" sounce network "VLAN_DOCKERSANDVMS" dest address "VLAN_DOCKERSANDVMS address" -port 22,80,443 -block 2.3) Optional: Description: "Block access PfSense management from VLAN": Int "All_VLANs" sounce network "VLAN_GUEST" dest address "VLAN_GUEST address" -port 22,80,443 -block 2.4) Optional: Description: "Block access PfSense management from VLAN": Int "All_VLANs" sounce network "VLAN_IOT" dest address "VLAN_IOT address" -port 22,80,443 -block 3.1) Description: "Allow VLAN 11 to talk to itself": Int "All_VLANs" sounce network "VLAN_MAIN" dest network "VLAN_MAIN" -allow 3.2) Description: "Allow VLAN 12 to talk to itself": Int "All_VLANs" sounce network "VLAN_DOCKERSANDVMS" dest network "VLAN_DOCKERSANDVMS" -allow 3.3) Description: "Allow VLAN 76 to talk to itself": Int "All_VLANs" sounce network "VLAN_GUEST" dest network "VLAN_GUEST" -allow 3.4) Description: "Allow VLAN 107 to talk to it's gateway": Int "All_VLANs" sounce network "VLAN_IOT" dest network "VLAN_IOT" -allow 4.1) Optional: Description: "Block access to printers or network appliances from a VLAN": Int "All_VLANs" sounce network "VLAN_IOT" dest address "NetworkApplianceIPsGroup" -block 4.2) Optional: Description: "Make printers or network appliances available": Int "All_VLANs" sounce network "172.17.0.0/16" dest address "NetworkApplianceIPsGroup" -allow 5.1) Description: "Block VLANs to VLAN 11": Int "All_VLANs" sounce network "172.17.0.0/16" dest network "VLAN_Main" -block 5.2) Description: "Block VLANs to VLAN 12": Int "All_VLANs" sounce network "172.17.0.0/16" dest network "VLAN_DOCKERSANDVMS" -block 5.3) Description: "Block VLANs to VLAN 76": Int "All_VLANs" sounce network "172.17.0.0/16" dest network "VLAN_Guest" -block 5.4) Description: "Block VLANs to VLAN 107": Int "All_VLANs" sounce network "172.17.0.0/16" dest network "VLAN_IOT" -block 6) Description: "Default allow LAN to any other rule": Int "All_VLANs" sounce network "172.17.0.0/16" dest address "any" -allow Finally, change your PfSense password. You should develop a habit of doing this and documenting it in a password manager when you initially set it up, but do it now if you forgot. *crosses fingers* Here's hoping that formatting isn't a disaster