Jump to content

pfSense Quad NIC Passthrough Issues - Resolved


Recommended Posts

I switched from an i7-4770k to a 2600X yesterday so I could finally pass through my quad port NIC and consolidate pfSense into my unRAID PC but was "unsuccessful" using Host Passthrough, Machine Q35-4.2, and OVMF (I eventually got it working but had to manually modify the XML); however, it worked without any modification when using Host Passthrough, Machine i440fx-4.2, and SeaBIOS or OVMF (perfect since future GUI changes/saves would destroy custom XML code). 

 

For those who wish to use Q35 I've documented the XML changes below but before I get to this, can somebody clarify some of these settings so I'm using the right ones going forward?

 

CPU Mode - Host Passthrough vs Emulated (QEMU64) ... what's the difference and which should I be using?

Machine - i440fx vs  Q35 - same as above

BIOS - SeaBIOS vs OVMF - same as above

 

My Motherboard BIOS has some settings for Legacy and UEFI ... are these related or impacting unRAID?  In fact, I've the choice of unRAID and UEFI unRAID from my boot menu ... does it matter?

 

 In addition, I was unable to install or boot pfSense until I made the following changes to the "cpu" code based on another thread ... not sure if this is at all related to my pass through issue with Q35?  The 2600X has been out for a while now, why does unRAID add this code?

  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>

changed to ...

  <cpu>
    <topology sockets='1' cores='4' threads='2'/>
  </cpu>

 

 

Regarding the XML changes for Q35, when adding the 4 ports from the quad NIC using unRAID GUI, the following code is generated, and pfSense only shows igb0 interface.  I tried removing the 2nd address code outside of the source tags from each port and ended up with the same result.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>

 

Here's the code changes which allowed pfSense to show valid interfaces as igb0, igb1, igb2, and igb3.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x3'/>
    </hostdev>

 

Edited by bugsysiegals
Link to comment
  • 3 weeks later...
On 3/22/2020 at 1:03 PM, bugsysiegals said:

I switched from an i7-4770k to a 2600X yesterday so I could finally pass through my quad port NIC and consolidate pfSense into my unRAID PC but was "unsuccessful" using Host Passthrough, Machine Q35-4.2, and OVMF (I eventually got it working but had to manually modify the XML); however, it worked without any modification when using Host Passthrough, Machine i440fx-4.2, and SeaBIOS or OVMF (perfect since future GUI changes/saves would destroy custom XML code). 

 

For those who wish to use Q35 I've documented the XML changes below but before I get to this, can somebody clarify some of these settings so I'm using the right ones going forward?

 

CPU Mode - Host Passthrough vs Emulated (QEMU64) ... what's the difference and which should I be using?

Machine - i440fx vs  Q35 - same as above

BIOS - SeaBIOS vs OVMF - same as above

 

My Motherboard BIOS has some settings for Legacy and UEFI ... are these related or impacting unRAID?  In fact, I've the choice of unRAID and UEFI unRAID from my boot menu ... does it matter?

 

 In addition, I was unable to install or boot pfSense until I made the following changes to the "cpu" code based on another thread ... not sure if this is at all related to my pass through issue with Q35?  The 2600X has been out for a while now, why does unRAID add this code?


  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>

changed to ...


  <cpu>
    <topology sockets='1' cores='4' threads='2'/>
  </cpu>

 

 

Regarding the XML changes for Q35, when adding the 4 ports from the quad NIC using unRAID GUI, the following code is generated, and pfSense only shows igb0 interface.  I tried removing the 2nd address code outside of the source tags from each port and ended up with the same result.


    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>

 

Here's the code changes which allowed pfSense to show valid interfaces as igb0, igb1, igb2, and igb3.


    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x3'/>
    </hostdev>

 

can you point out the changes you made in the second part of your code 

Link to comment
  • 3 weeks later...

Rather than separating each NIC port into it's own virtual slot I added multifunction='on' to the first port, eliminated bus address. assigned the same slot address, and matched address function with source address function.

 

For example, here's what I see in Tools > System Devices for my NIC.  You can see these devices share the same slot on the MB, bus 7 and are identified by function 0, 1, 2, and 3.  The slot # will change if you add/remove cards from your MB as I found out yesterday when adding a GPU wondering why pfSense would no longer boot requiring me to update the XML.

[8086:150e] 07:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
[8086:150e] 07:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
[8086:150e] 07:00.2 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
[8086:150e] 07:00.3 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)

Here's a video explaining this in much greater detail ... 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...