Jump to content
SpaceInvaderOne

*VIDEO GUIDE* A comprehensive guide to pfSense both unRAID VM and physical

102 posts in this topic Last Reply

Recommended Posts

12 hours ago, diarnu said:

based on your earlier posts, I'm not sure how the iPad on the WIFI subnet ever saw the server on the LAN subnet ...

It didn't :) i had to add a bridge in between the to subnets...

When i first installed it i did not understand that each nic would form a separate subnet.

 

I am thinking of using a separate nic/subnet with an extra access point, for IOT stuff, Nest, Alexa etc. But i'm not sure if e.g. the Nest app on the 'normal' wifi will then be able to access the thermostat if that is on another subnet...

Share this post


Link to post

For some reason my quad Intel NIC does not have wol_magic enabeled by default.

After fighting with the /etc/rc.conf.local or /usr/local/etc/rc.d script not executing at boot I figured out that in order to enable wol_magic on a quad intel nic on my hardware pfsense box I had to add the following to the Services->Shellcmd->Command: ifconfig igb1 wol_magic (replace igb1 with the NIC interface that you want to use to wake up the hardware pfsense box - hint it should be the LAN one) , Shellcmd Type: shellcmd

Shellcmd is a package that can be installed using the Package Manager - compared with the /etc/rc.conf.local method this way the command is saved in the config.xml file for backup/restore purposes.

 

 

 

 

Share this post


Link to post
On 5/10/2019 at 12:35 AM, jowi said:

not sure if e.g. the Nest app on the 'normal' wifi will then be able to access the thermostat if that is on another subnet...

Hey jowi.

 

The nest app on "normal" wifi would not be able to communicate with the thermostat on another subnet by default.  You could add firewall rules to allow the 2 clients to talk across subnets if you wanted that.  But by default, pfsense does not allow traffic between different subnets.

 

For what its worth, I recently went down a path similar to the one you seem to be headed towards.  I wanted different subnets for IoT stuff, and for guests when they visit, and maybe for game consoles and set-top boxes to bypass my VPN, etc.  It started to get kinda crazy with all the different NIC ports and wifi access points required.  Also, using different wifi access points for different subnets would probably be messy, with the APs conflicting with each other if on the same band/channel.  But keeping the APs on different bands/channels seemed like a pain...   

 

The solution I found was to use VLANs for different subnets, and an ubiquiti unifi switch + ubiquiti unifi AP for wifi.  Using VLANs means less physical ports required for the NIC that pfsense uses.  On the wifi side, the unifi AP can handle different wifi SSIDs for each VLAN, all in a single AP.  For the switch, any old VLAN capable switch could work, but once committed to the unifi AP it made sense to use the unifi switch.  In addition, both the unifi switch and unifi AP can be managed from a unifi controller, and that can be run as a docker on unraid. 

 

Hope that helps. 

Share this post


Link to post

By any chance does ANYONE have an earlier version of pfSense, than 2.4.4... preferrably 2.3.*?  I am receiving all sorts of REALLY weird installation options which look NOTHING like the videos.... they are asking about terminal mode options (whether I want to use:

ansi

vt100

xterm

cons25w

 

with none of this leading me to to installation...all sorts of crap!  

 

Please advise.

 

 

pfSense terminal mode weird.png

Share this post


Link to post

@SpaceInvaderOne Your videos are fantastic.

 

Thanks to you I have a running Uraid server doing everthing I need apart from Pfsense.

 

I have watched all of the videos but am stuck with a problem.

 

I have passed through my 4 port Intel nic but pfsense only sees one port.

 

Any help would be appreciated.

Share this post


Link to post
On 11/25/2019 at 11:47 PM, Beaker69 said:

@SpaceInvaderOne Your videos are fantastic.

 

Thanks to you I have a running Uraid server doing everthing I need apart from Pfsense.

 

I have watched all of the videos but am stuck with a problem.

 

I have passed through my 4 port Intel nic but pfsense only sees one port.

 

Any help would be appreciated.

Hi,

 

I was about to raise the very same question... Did you manage to fix this yet?

I have a dual gbit Intel nic, which I know works under BSD.

I have followed the tutorial and also see only 1 port.

 

One port has a physical address ending with :46, the other one is :47.

 

When I passthrough both the machine/pfsense sees only :47.

When I passthrough :46 it sees :46 (as expected), and when I passthrough :47 is sees :47 (as expected)..

 

Anyone any clues?

I have fiddled around with PCIe ACS overrides, but to no avail.

Share this post


Link to post

I have not managed to get it sorted yet, but you could try this.

 

 

If you have any luck post your results.

 

Good luck

Share this post


Link to post

I have managed to pass through my nics by editing the hostdev address type like described here:

Maybe you could try this.

Share this post


Link to post

@user2352 and @Beaker69 I managed to fix it with this. Thanks so much!

In my case I had to change both the bus and function for the 2nd nic and then it worked.. :)

 

Happy to see this working, now I can continue 🙂 

Share this post


Link to post
1 hour ago, emod said:

How do you set up VPN for some interfaces?

Hi, happy to write a short tutorial for you, but to make sure I understand correctly.. You mean within pfsense right?

Share this post


Link to post
On 1/14/2020 at 8:22 PM, MaTi said:

Hi, happy to write a short tutorial for you, but to make sure I understand correctly.. You mean within pfsense right?

If it's not too late, I would definitely be interested in a short tutorial of this!

Share this post


Link to post

I Finally got the stub working and I can select the 4 NIC intel card in the VM - But for some reason I only see one interface during the install - I wanted to setup pfsense before I get my new fiber connection but I just get this one interface? And no other for LAN

 

the 4 Nics are visible and selected in the VM?

image.png.36aca24788058c4614d63ad38cbe2691.png

what am I missing?

image.thumb.png.d34ac8a2cc522f7586a05dcc2326abd2.png

 

But I can't see the 4 NIC's during install only one and always link down

image.thumb.png.75518490ac317dcab9c007dd53c2052e.png

 

 

I have tried to do this with both seabios and OVMF

Also stubbed them tried both with vfio-pci.ids and pci-stub.ids?

image.png.f15e32eb248ad5ed98a783ed1a45a56a.png

 

image.png.77f6a0cd8fe2f80f108bbe02cae70ee0.png

 

Can't think of anything new to try?

 

Ok went through every setting again and by chance changed the Machine type to Q35 but not any version went back to the old Q35-2.11 and not the Q35-4.2

And got this... :-)

image.thumb.png.a342f352b856cffe1153c3d755af217b.png

 

 

Edited by casperse

Share this post


Link to post

Does anyone know if there is a Video tutorial for OpenVPN and PIA in Pfsense?

The introduction video talked about installing Openvpn on pfsense but I cant find the video?

Thanks! 🙂

 

Share this post


Link to post

I too am thoroughly frustrated with this!!

 

Folks..... Can I ask you to have a look at my details and see what I have missed please?

 

Like others, I have the 4 Nic Intel card, which pfSense only passes the first NIC through when left to its normal setup....

 

Following lots of editing tips from this thread, and others, I've arrived here with two NICs in play..... but for the life of me, I cant get the second pair up.......

 

So, I stubbed the NICs...

 

1991777566_NicIOMMUgroups.thumb.JPG.297e6786021dc20a5b6202628323db3f.JPG

 

824078583_nicstub.JPG.f6c28509c8d8351c973428e5a5cafc1a.JPG

 

Succesfully arrived in the VM;

 

54384604_Nicstubsuccess.JPG.f714efa07426af0efd6ab9e8dfad9c18.JPG

 

Edited the XML;

342709513_nicallocationhighlight.jpg.d9d324c081fa9cd392c74e6b5630df27.jpg

 

End result;

 

725166885_niconlytwo.thumb.JPG.d4cbde18313d7596bb6196c52cecfdc7.JPG

 

I've also tried various combinations of bus/slot/function for that second pair..... all to no avail.......... :(

 

If I was charging myself by the hour for the time spent on this...... I'd be very, very broke!

Share this post


Link to post
9 minutes ago, pm1961 said:

I too am thoroughly frustrated with this!!

 

Folks..... Can I ask you to have a look at my details and see what I have missed please?

 

Like others, I have the 4 Nic Intel card, which pfSense only passes the first NIC through when left to its normal setup....

 

Following lots of editing tips from this thread, and others, I've arrived here with two NICs in play..... but for the life of me, I cant get the second pair up.......

 

So, I stubbed the NICs...

 

1991777566_NicIOMMUgroups.thumb.JPG.297e6786021dc20a5b6202628323db3f.JPG

 

824078583_nicstub.JPG.f6c28509c8d8351c973428e5a5cafc1a.JPG

 

Succesfully arrived in the VM;

 

54384604_Nicstubsuccess.JPG.f714efa07426af0efd6ab9e8dfad9c18.JPG

 

Edited the XML;

342709513_nicallocationhighlight.jpg.d9d324c081fa9cd392c74e6b5630df27.jpg

 

End result;

 

725166885_niconlytwo.thumb.JPG.d4cbde18313d7596bb6196c52cecfdc7.JPG

 

I've also tried various combinations of bus/slot/function for that second pair..... all to no avail.......... :(

 

If I was charging myself by the hour for the time spent on this...... I'd be very, very broke!

Shouldn't the bus on the hostdevs be 1x00 slot 0x00 and the functions be 0x0 thru 0x3?

Share this post


Link to post
Posted (edited)

Hi David, thanks for the reply.............. I was sure I'd already tried that combo.... but gave it another shot, with the same outcome, unfortunately. :(

Edited by pm1961

Share this post


Link to post
On 3/18/2020 at 1:46 AM, pm1961 said:

I too am thoroughly frustrated with this!!

 

Folks..... Can I ask you to have a look at my details and see what I have missed please?

 

Like others, I have the 4 Nic Intel card, which pfSense only passes the first NIC through when left to its normal setup....

 

Following lots of editing tips from this thread, and others, I've arrived here with two NICs in play..... but for the life of me, I cant get the second pair up.......

 

So, I stubbed the NICs...

 

1991777566_NicIOMMUgroups.thumb.JPG.297e6786021dc20a5b6202628323db3f.JPG

 

824078583_nicstub.JPG.f6c28509c8d8351c973428e5a5cafc1a.JPG

 

Succesfully arrived in the VM;

 

54384604_Nicstubsuccess.JPG.f714efa07426af0efd6ab9e8dfad9c18.JPG

 

Edited the XML;

342709513_nicallocationhighlight.jpg.d9d324c081fa9cd392c74e6b5630df27.jpg

 

End result;

 

725166885_niconlytwo.thumb.JPG.d4cbde18313d7596bb6196c52cecfdc7.JPG

 

I've also tried various combinations of bus/slot/function for that second pair..... all to no avail.......... :(

 

If I was charging myself by the hour for the time spent on this...... I'd be very, very broke!

Have you tried the settings below? Also check that the address types aren't already taken by other devices in the VM.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x1' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x2' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x08' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x3' multifunction='on'/>
    </hostdev>

If this doesn't work have you tried splitting the IOMMU groups? Explanation in this video:

 

 

Share this post


Link to post

@SpaceInvaderOne

I finally got everything working now but I would like to know if you have any special rules for the VM's on Unraid?

 

My Unraid server IP is used and shared by the Docker and the same gateway (subnet) Unraid server IP: 192.168.0.6 like most people have...
I have virtual machines VM's on the Unraid server with their own fixed IP like: 192.168.0.18

 

BUT If I route any traffic through the Pfsense for the server Unraid IP, dockers etc on the 192.168.0.6 it will overrule any traffic coming from my VM having IP: 192.168.0.18 and route everything over the rule set for the Unraid server IP 192.168.0.6 hosting the VM's !!! 😞

 

Do I need to passthrough NIC's to my VM's? in order to separate them from the Unraid server IP?

Share this post


Link to post
Posted (edited)

Hi user2352,

 

Thanks for your input..... I have now got it working using that xml.....  :)

 

I don't believe that it was necesary/possible to break up the IOMMU up any more than unraid had already done. That seemed to be the way in the video too........

 

I was convinced I'd already tried that xml config many times before with no success, so I don't know what I did differently this time.........???

 

I suspect it may have been to do with restart/reboots.... I wasn't very scientific in taking note of what I tried and in what order. I thought it was enough to stop the VM, edit the xml and then restart it... But, for me, that didn't work...........

 

However, some combination of editing and rebooting the entire machine seems to have done the trick....... but I can't prove it......

 

Anyway, all is well now!

 

1643387_workingpfsense.JPG.8cef0453059f316fcb10f71017644e88.JPG

 

ATB,

 

Paul

Edited by pm1961

Share this post


Link to post

Hi,

 

I followed this guide in May last year and have been running great since then.  I have just used the last port on my 4 port nic as a seperate VPN connection and whilst getting help from netgate in setting it up it was noticed my version was very old (2.4.3_1) but the status is always showing as up to date and if I try to update it says I am on the latest version.

 

I have worked through the help section and all end with this error

 

Upgrading pfSense-repo...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Checking integrity...Assertion failed: (strcmp(uid, p->uid) != 0), function pkg_conflicts_check_local_path, file pkg_jobs_conflicts.c, line 386.
Child process pid=10408 terminated abnormally: Abort trap

 

So I have been advised to ditch it and replace it with a fresh installation of 2.4.5

 

Is there a recommended way of doing this? I have obviously taken a backup but looking for any advice as don't want to be without internet whilst in lockdown!

Share this post


Link to post
On 4/24/2018 at 1:12 PM, SpaceInvaderOne said:

Hi @joelones Just set in the bios of the pfsense to enable wake on lan. When the machine is off it will still power the lan port for wake on lan.

I use @Squid excellent user script plugin to send a wol ping using etherwake command

This script runs on array stop

 


etherwake 00:01:3e:4e:5a:b8

 

I also use another script for when the array starts

This uses ssh to login to the pfsense machine and shut it down this way only one pfsense is running at a time

ie 


ssh admin@10.10.20.1 /etc/rc.halt

You will need to generate some ssh key pairs on unRAID and copy the public key to the admin user in pfsense.

 

All of this will be covered in my pfsense videos

I have watched all the videos and forgive me if i have missed it but i can not seem to recall which video and timestamp this was covered in.

Share this post


Link to post

I've been running pfsense in a vm on a headless UNRAID server for some time now, and it works great... until the server has an issue. And then you can't do anything... you can't reach pfsense, you can't ssh or even IPMI into unraid... you don't have network, you don't have internet. The only thing you can do is turn the server off HARD and pray that your disks are ok...

 

And IF you are rebooting, you can't IPMI to enter bios or even see bios etc because you can reach the server only after it has booted and pfsense is started... it's a great way get into pfsense, but i'm gonna go the dedicated pfsense hardware route as soon as i can.

Edited by jowi

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.