JonathanM Posted April 4, 2020 Share Posted April 4, 2020 6 hours ago, jowi said: I've been running pfsense in a vm on a headless UNRAID server for some time now, and it works great... until the server has an issue. And then you can't do anything... you can't reach pfsense, you can't ssh or even IPMI into unraid... you don't have network, you don't have internet. The only thing you can do is turn the server off HARD and pray that your disks are ok... And IF you are rebooting, you can't IPMI to enter bios or even see bios etc because you can reach the server only after it has booted and pfsense is started... it's a great way get into pfsense, but i'm gonna go the dedicated pfsense hardware route as soon as i can. That's why you keep a basic router programmed to get your basic network back up in a pinch. No port forwards or VPN, just basic NAT with dhcp. Quote Link to comment
jowi Posted April 4, 2020 Share Posted April 4, 2020 (edited) Yeah, i know, and i do. But then you have to rewire everything... and there is not much room in the closet. Maybe i just wire it up anyway, so when it is needed, i only have to turn it on, and switch the cable modem connection to the emergency router in stead of the unraid server... but then what happens if i reboot the unraid server and the pfsense vm starts up? Then there are 2 routers with ip address 192.168.1.1... or can i just use e.g. 192.168.1.1 for pfsense and 192.168.1.2 for my failover router, and just disable dhcp on the failover router while pfsense is running? I would like to 'switch' if unraid/pfsense is down for some reason, without a lot of recabling. Just unplug the cablemodem from pfsense, and plug it in the failover router, enable dhcp and get on with it? Is that possible? Edited April 4, 2020 by jowi Quote Link to comment
Sinister Posted April 22, 2020 Share Posted April 22, 2020 Has anyone managed to get the failover demonstrated in the first video working ? if so i would love to be pointed in the right direction Quote Link to comment
Sinister Posted April 26, 2020 Share Posted April 26, 2020 So after burning my brains i managed to get everything working except the last part: ssh [email protected] /etc/rc.halt You will need to generate some ssh key pairs on unRAID and copy the public key to the admin user in pfsense i have generated my keys pasted the public version into the admin user of pfsense ssh is enabled and works have tested this in puTTY. i created a user script with "ssh [email protected] /etc/rc.halt" no quotes and ip address being the lan IP of pfsense can anyone tell me the last part im missing to make this work Quote Link to comment
AgentXXL Posted April 28, 2020 Share Posted April 28, 2020 (edited) UPDATE @ 3:20pm Mountain: I've finally managed to at least get the Intel dual port 1Gbps NIC passed through to the pfSense VM and for the 1st time I was able to reach the pfSense menu and configure the interfaces. It's not seeing my HP NC523SFP dual port 10Gbps NIC but that's probably a driver/module issue that I can update in the pfSense VM. I've read that others have used the HP (QLogic) 10Gbps adapters so I'm sure I'll eventually figure it out UPDATE #2: did some testing and found that pfSense fails to detect the NICs with QEMU 4.x. QEMU 3.1 is the version that I'm currently using and so far pfSense seems stable. As for the current solution that enabled the Intel NIC: it was as simple as changing the machine type from Q35-4.2 to Q35-2.11 - saw this in one of the many threads that discuss passthrough. It was the only change made to get it to recognize at least the Intel NIC. Original post: I'm having some issues with getting my NICs passed through to the pfSense VM. I've tried both the vfio-pci.cfg/BIND and the vfio-pci.ids=xxxx.yyyy methods. I have two dual port NICs, one an Intel 82571 dual port 1Gbps and the other a HP NC523SFP dual port 10Gbps. Both methods do seem to bind the cards to vfio as unRAID no longer sees them under Network Settings. The initial creation of the VM as per part 3 of the videos works as expected. When the VM reboots at the end of the install, pfSense loads from the vdisk but fails before getting to the pfSense menu options. The error message reported states that 'Configuration references interfaces that do not exist (em0, em1). I never get to the ‘interface assignment’ stage…. I’ve also tried with PCIe ACS override and VFIO allow unsafe interrupts options in VM Settings. When I reboot the VM and hit escape at the loader menu I can issue an 'lsdev' command and it shows the following: Are then net0: and net1: devices 2 of my 4 NIC ports? Where are the other two? I tried with just the Intel dual port card and saw the same result. I then tried with just the HP NC523SFP card by itself, also yielding the same results. As another note, when I try to review the logs of the VM boot, it’s very short and only has the last couple of screens of info prior to the halt. How do I increase the log size for my VM so I can capture all of the messages from start of the boot through to the halt. At least then I can possibly find a cause for my issue. Here's the IOMMU list for my NICs: The onboard dual 1Gbps NICs have a different PCI ID and they are still visible to unRAID and working in a failover bonded pair. Any suggestions on what I need to do to overcome the halt issue for the pfSense VM? TIA! Edited May 4, 2020 by AgentXXL Cleanup formatting and add missing info Quote Link to comment
Armed Ferret Posted May 24, 2020 Share Posted May 24, 2020 Hello. When I launch PFSense for the first time I get stuck on the black screen where it says booting... and nothing else happens. I have attached my logs from my PFsense Vm. I see the following errors 2020-05-24 02:05:07.735+0000: Domain id=1 is tainted: high-privileges 2020-05-24 02:05:07.735+0000: Domain id=1 is tainted: host-cpu char device redirected to /dev/pts/0 (label charserial0) 2020-05-24T02:05:18.252838Z qemu-system-x86_64: vfio-pci: Cannot read device rom at 0000:08:00.0 Device option ROM contents are probably invalid (check dmesg). Skip option ROM probe with rombar=0, or load from file with romfile= 2020-05-24T02:05:18.254745Z qemu-system-x86_64: vfio-pci: Cannot read device rom at 0000:09:00.0 Device option ROM contents are probably invalid (check dmesg). Skip option ROM probe with rombar=0, or load from file with romfile= 2020-05-24T02:05:18.256320Z qemu-system-x86_64: vfio-pci: Cannot read device rom at 0000:08:00.1 Device option ROM contents are probably invalid (check dmesg). Skip option ROM probe with rombar=0, or load from file with romfile= 2020-05-24T02:05:18.257872Z qemu-system-x86_64: vfio-pci: Cannot read device rom at 0000:09:00.1 Device option ROM contents are probably invalid (check dmesg). Skip option ROM probe with rombar=0, or load from file with romfile= I did some searches and someone mentioned it has to do with the CPU pinning but I pinned 2 threads on 1 core. I even tried switching the pinning to a different core. I had 1 and 7 pinned but now I have 5 and 11. So Im a little lost here. I used the vfio plugin to get passthrough working. I also had to enable ACS override to get the IOMMU groups broken up. Here is my pfsense VM config Part 1 Part 2 I have attached my pfsense logs and my IOMMU group in txt files. Any help would be appreciated. PFsense logs.txt IOMMU Groups.txt Quote Link to comment
Chris09 Posted June 10, 2020 Share Posted June 10, 2020 Hello All, I'm having the same issue as @Armed Ferret my current version of unraid is 6.8.3, is this a known issue or is it unsupported? I am a noob at unraid and wanted to see about getting pfsense vitalized as one of my projects after my plex server. Thanks Quote Link to comment
Addy Posted June 12, 2020 Share Posted June 12, 2020 (edited) Hey guys, I could really use some help.. I'm not sure what is going on. I'm trying to get my gbe nic passed through to my VM, but I don't think my syslinux config is working properly This was my original Config kernel /bzimage append pcie_acs_override=downstream initrd=/bzroot pci=nomsi,noaer This is my edited config First I tried kernel /bzimage append vfio-pci.ids=8086:1079 pcie_acs_override=downstream initrd=/bzroot pci=nomsi,noaer This seemed to have 0 effect, the devices were still in the network settings. Then I tried below, as I saw it in a post kernel /bzimage append pci-stub.ids=8086:1079 pcie_acs_override=downstream initrd=/bzroot pci=nomsi,noaer I can no longer see the controllers in my network settings, but I also cannot see them available to passthrough to the VM Any idea's? Edit: I have just been editing the Unraid OS syslinux config, I assume this is fine as this is how I boot it Edited June 12, 2020 by Addy Quote Link to comment
vitorbrito Posted June 19, 2020 Share Posted June 19, 2020 I'm following along with the tutorial, but when I start the VM VNC doesn't show anything. Any tips? Quote Link to comment
david279 Posted June 19, 2020 Share Posted June 19, 2020 Pfsense VM works now without any xml modification on the new 6.9 beta22 Quote Link to comment
Chris09 Posted June 30, 2020 Share Posted June 30, 2020 On 6/19/2020 at 2:15 PM, david279 said: Pfsense VM works now without any xml modification on the new 6.9 beta22 Just confirmed, this is true Quote Link to comment
bdydrp Posted July 1, 2020 Share Posted July 1, 2020 (edited) Have followed along and when its time to assign interface ports, pfsense doesnt see that 4port NIC This is my IOMMU groups I am assuming the ticked items in Group 1 is the 4 port intel NIC, but not sure if the remaining item in thatgroup are ascociated? Do they need to be split? I have tried the overide option on or off, this is what pfsense displays igb0 is the onboard nic? EDIT: In the VM creation it automatically sets the machine type as Q35-5.0 Once i changed it to Q35-2.11 everything worked as per the video And i did not have to set any pcie overrides. My intel nic in group 1 passed through no problems. pfSense now installed!! Edited July 1, 2020 by bdydrp Quote Link to comment
kaiguy Posted July 1, 2020 Share Posted July 1, 2020 I'd be super interested in getting more information about failover. Aside from a brief explanation in the first video I haven't found any other references to it. I'm specifically curious about: Are the configurations identical (e.g., did you take a backup from the primary pfsense instance and restore it on the failover device (obviously updating the interface assignments to WAN and LAN)? Have all the extra packages installed? Does it require any different configurations of the network setup? For example, do devices on the network complain when the switchover occurs since DHCP assignments may get wonky? Does it require a different gateway? Does the changeover require physically changing the modem network cable to the physical box, or do you have something going on with the switch to assign it its own VLAN? If anyone can shed some light, that would be outstanding! Thanks! Quote Link to comment
bdydrp Posted July 2, 2020 Share Posted July 2, 2020 (edited) OK - Dumb question My current setup is: VDSL modem (DHCP) > netgear router > switch With pfSense, what the correct way? VDSL > wan port on nic > lan port on nic > switch? VDSL > switch > wan port on nic > lan port on nic > switch? reason i ask , is that when setting up pfsense, and auto selecting wan/lan ports both these were connected directly to the switch, but pfsense didnt get an external IP from my ISP but instead an IP in my local range (192.168.1.xxx) When i disconnect my current router, cant seem to get pfsense to get an IP from my ISP!! Edited July 2, 2020 by bdydrp Quote Link to comment
Chris09 Posted July 2, 2020 Share Posted July 2, 2020 2 minutes ago, bdydrp said: OK - Dumb question My current setup is: VDSL modem (DHCP) > netgear router > switch With pfSense, what the correct way? VDSL > wan port on nic > lan port on nic > switch? VDSL > switch > nic wan port > nic lan port > switch? reason i ask , is that when setting up pfsense, and auto selecting wan/lan ports both these were connected directly to the switch, but pfsense didnt get an external IP from my ISP but instead an IP in my local range (192.168.1.xxx) When i disconnect my current router, cant seem to get pfsense to get an IP from my ISP!! I had pfsense prior, it is amazing!! But usually its ISP -> pfsense -> switch pfsense handles everything else and can be configured for how you want it to be Quote Link to comment
bdydrp Posted July 2, 2020 Share Posted July 2, 2020 (edited) Thats what i had thought, but i cant get pfsense to get an IP from my ISP - and hence no internet connection Edited July 2, 2020 by bdydrp Quote Link to comment
Chris09 Posted July 2, 2020 Share Posted July 2, 2020 1 minute ago, bdydrp said: Thats what i had thought, but i cant get pfsense to get an IP from my ISP - and hence no internet connection There are too many factors that would apply, but are you trying to run it on vm or bare metal? Quote Link to comment
bdydrp Posted July 2, 2020 Share Posted July 2, 2020 20 minutes ago, Chris09 said: There are too many factors that would apply, but are you trying to run it on vm or bare metal? Sorry, in a VM! Quote Link to comment
Chris09 Posted July 2, 2020 Share Posted July 2, 2020 1 hour ago, bdydrp said: Sorry, in a VM! I remember seeing an issue posted somewhere in the forums about this and I've been trying to find the page for this but no luck so far... I haven't fully gotten mines up in a vm yet, I'm mainly done it bare metal Quote Link to comment
bdydrp Posted July 2, 2020 Share Posted July 2, 2020 all good When i was setting up the vm, at the point of auto selecting wan/lan port, i connected a cable from my switch to these ports. So i might delete and remove vm. Then that time of auto selecting wan port, unplug current router and connect isp straight to wan port, then lan to switch. See how that goes.....Nothing to lose!! Quote Link to comment
sekrit Posted July 20, 2020 Share Posted July 20, 2020 On 4/20/2018 at 8:20 PM, SpaceInvaderOne said: I am starting a series of videos on pfSense. Both physical and VM instances will be used. Topics such as using a failover physical pfSense to work with a VM pfSense. Setting up OpenVPN (both an OpenVPN server and OpenVPN multiple clients). Using VLANs. Blocking ads. Setting up squid and squid guard and other topics. T This part is an introduction part gives an overview of the series of videos and talks about pfSense and its advantages. Second part of is on hardware and network equipment Part 3 install and basic config Part 4 customize backup and aupdate Part 5 DHCP, Interfaces and WIFI Part 6 Pfsense and DNS Part 7 - Firewall rules, Portforwarding/NAT, Aliases and UPnp Part 8 Open NAT for XBOX ONE and PS4 So, How do we get the physical machine to turn on as failover when the PFSENSE VM failis? Also, is there a way to sync settings between the two (I dunno... have both load from a network image or sonething? Quote Link to comment
AlexMex Posted November 12, 2020 Share Posted November 12, 2020 Hi, Is there any chance to get this working on Ryzen with Unraid 6.9? I've not been able to install pfsense (without or with passthrough of my intel nic) on Unraid 6.8. Tested on Unraid 6.8 with Ryzen 1700/Asus Prime X370-Pro, Ryzen 2700X/Asus Prime X470-Pro and Ryzen 3900X/Aorus X570 Ultra. I would like to shutdown the two pc I use to run pfsense and keep only Unraid ones. Quote Link to comment
Whatsinaname Posted December 20, 2020 Share Posted December 20, 2020 Hi Spaceinvader One, Thanks for your excellent videos on pfsense and other helpful videos. I'd like to share that pfsense now has a new developer version available that specifically doesn't require AES-NI. I have "new" processors underway that do support AES-NI but I might give that a try first, and go to a stable version, current 2.45 or later a stable version of 2.5. Thanks again. Cheers. Quote Link to comment
Burnstation19 Posted December 28, 2020 Share Posted December 28, 2020 Just FYI to anyone passing a X520-DA2 10gb NIC through may only have one interfeace load in freebsd pfsense.... I had to follow the following post and edit xml...mutltifunction not enabled was issue. This may save someone hours of time and suffering Cheers, Burnstation19 1 Quote Link to comment
Lebowski Posted January 6, 2021 Share Posted January 6, 2021 Wondering if anyone has any tips around performance. It seems we get a penalty under a VM. I have 1000Mbps/50Mbps but the best I can get via Pfsense in Unraid is 750Mbps, this seems to be common for anyone running in a VM. Anyone had any joy? (I can plug in directly and get full speed. Its related to the VM) Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.