NIC Passthrough Help


Recommended Posts

First off, thanks for any help with this.  I am in the process of moving everything from a ESXi host to unRaid and so far am really impressed with the software.  I am now trying to set up a pfSense VM and want to pass through one of my quad NICs to it.  

 

I have tried adding this to the XML but I get an error and the VM won't boot.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
    </hostdev>

 

This is the error I get:

Execution error

 

internal error: process exited while connecting to monitor: 2017-08-30T20:03:15.375733Z qemu-system-x86_64: -device vfio-pci,host=02:00.1,id=hostdev0,bus=pci.0,addr=0x5: vfio: error, group 15 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
2017-08-30T20:03:15.375774Z qemu-system-x86_64: -device vfio-pci,host=02:00.1,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to get group 15
2017-08-30T20:03:15.375799Z qemu-system-x86_64: -device vfio-pci,host=02:00.1,id=hostdev0,bus=pci.0,addr=0x5: Device initialization failed

 

I assume this is because the NIC is tied in to unraid.  I have tried to adding my NIC to pci-stub, but due to all four NICs having the same vendor info (14F1:880) I loose all four NICs on reboot.  I have done a bit of searching and am having a hard time finding a way around this.  Is it possible to only withhold 1 of the 4 NICs?  Any ideas would be fantastic.  My system devices output is below and I have bolded the NIC I am trying to pass through. Let me know if I can add anything else that would be helpful.

 

PCI Devices and IOMMU Groups

IOMMU group 0
	[8086:3406] 00:00.0 Host bridge: Intel Corporation 5520 I/O Hub to ESI Port (rev 13)
IOMMU group 1
	[8086:3408] 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13)
IOMMU group 2
	[8086:340a] 00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13)
IOMMU group 3
	[8086:340b] 00:04.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 4 (rev 13)
IOMMU group 4
	[8086:340c] 00:05.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 5 (rev 13)
IOMMU group 5
	[8086:340d] 00:06.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 6 (rev 13)
IOMMU group 6
	[8086:340e] 00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13)
IOMMU group 7
	[8086:3410] 00:09.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 9 (rev 13)
IOMMU group 8
	[8086:342e] 00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers (rev 13)
	[8086:3422] 00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13)
	[8086:3423] 00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13)
IOMMU group 9
	[8086:3430] 00:16.0 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
	[8086:3431] 00:16.1 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
	[8086:3432] 00:16.2 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
	[8086:3433] 00:16.3 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
	[8086:3429] 00:16.4 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
	[8086:342a] 00:16.5 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
	[8086:342b] 00:16.6 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
	[8086:342c] 00:16.7 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
IOMMU group 10
	[8086:2937] 00:1a.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)
	[8086:2938] 00:1a.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)
	[8086:293c] 00:1a.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)
IOMMU group 11
	[8086:2934] 00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02)
	[8086:2935] 00:1d.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02)
	[8086:293a] 00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02)
IOMMU group 12
	[8086:244e] 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
	[102b:0532] 08:03.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eW WPCM450 (rev 0a)
IOMMU group 13
	[8086:2918] 00:1f.0 ISA bridge: Intel Corporation 82801IB (ICH9) LPC Interface Controller (rev 02)
IOMMU group 14
	[14e4:1639] 01:00.0 Ethernet controller: Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
	[14e4:1639] 01:00.1 Ethernet controller: Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
IOMMU group 15
	[14e4:1639] 02:00.0 Ethernet controller: Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
	[14e4:1639] 02:00.1 Ethernet controller: Broadcom Limited NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
IOMMU group 16
	[1000:0060] 03:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 1078 (rev 04)
IOMMU group 17
	[1000:0072] 05:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
IOMMU group 18
	[14f1:8880] 06:00.0 Multimedia video controller: Conexant Systems, Inc. CX23887/8 PCIe Broadcast Audio and Video Decoder with 3D Comb (rev 0f)
IOMMU group 19
	[8086:2c40] fe:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers (rev 05)
	[8086:2c01] fe:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder (rev 05)
IOMMU group 20
	[8086:2c10] fe:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05)
	[8086:2c11] fe:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0 (rev 05)
	[8086:2c14] fe:02.4 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 1 (rev 05)
	[8086:2c15] fe:02.5 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 1 (rev 05)
IOMMU group 21
	[8086:2c18] fe:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller (rev 05)
	[8086:2c19] fe:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder (rev 05)
	[8086:2c1a] fe:03.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller RAS Registers (rev 05)
	[8086:2c1c] fe:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers (rev 05)
IOMMU group 22
	[8086:2c20] fe:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers (rev 05)
	[8086:2c21] fe:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers (rev 05)
	[8086:2c22] fe:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers (rev 05)
	[8086:2c23] fe:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers (rev 05)
IOMMU group 23
	[8086:2c28] fe:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers (rev 05)
	[8086:2c29] fe:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers (rev 05)
	[8086:2c2a] fe:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers (rev 05)
	[8086:2c2b] fe:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers (rev 05)
IOMMU group 24
	[8086:2c30] fe:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers (rev 05)
	[8086:2c31] fe:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers (rev 05)
	[8086:2c32] fe:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers (rev 05)
	[8086:2c33] fe:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers (rev 05)
IOMMU group 25
	[8086:2c40] ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers (rev 05)
	[8086:2c01] ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder (rev 05)
IOMMU group 26
	[8086:2c10] ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05)
	[8086:2c11] ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0 (rev 05)
	[8086:2c14] ff:02.4 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 1 (rev 05)
	[8086:2c15] ff:02.5 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 1 (rev 05)
IOMMU group 27
	[8086:2c18] ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller (rev 05)
	[8086:2c19] ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder (rev 05)
	[8086:2c1a] ff:03.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller RAS Registers (rev 05)
	[8086:2c1c] ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers (rev 05)
IOMMU group 28
	[8086:2c20] ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers (rev 05)
	[8086:2c21] ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers (rev 05)
	[8086:2c22] ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers (rev 05)
	[8086:2c23] ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers (rev 05)
IOMMU group 29
	[8086:2c28] ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers (rev 05)
	[8086:2c29] ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers (rev 05)
	[8086:2c2a] ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers (rev 05)
	[8086:2c2b] ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers (rev 05)
IOMMU group 30
	[8086:2c30] ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers (rev 05)
	[8086:2c31] ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers (rev 05)
	[8086:2c32] ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers (rev 05)
	[8086:2c33] ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers (rev 05)

CPU Thread Pairings

cpu 0 <===> cpu 8
cpu 1 <===> cpu 9
cpu 2 <===> cpu 10
cpu 3 <===> cpu 11
cpu 4 <===> cpu 12
cpu 5 <===> cpu 13
cpu 6 <===> cpu 14
cpu 7 <===> cpu 15

USB Devices

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
Bus 001 Device 003: ID 0781:5530 SanDisk Corp. Cruzer
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 002: ID 0624:0248 Avocent Corp. Virtual Hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

SCSI Devices

[0:0:0:0]    disk    SanDisk  Cruzer           1100  /dev/sda   8.00GB
[1:0:32:0]   enclosu DP       BACKPLANE        1.07  -               -
[1:2:0:0]    disk    DELL     PERC 6/i         1.22  /dev/sdb    639GB
[2:0:0:0]    enclosu DELL     MD1000           A.04  -               -
[2:0:1:0]    disk    ATA      HITACHI HUA7210S AC4A  /dev/sdc   1.00TB
[2:0:2:0]    disk    ATA      SAMSUNG HD103UJ  1113  /dev/sdd   1.00TB
[2:0:3:0]    disk    ATA      HITACHI HUA7210S AC4A  /dev/sde   1.00TB
[2:0:4:0]    disk    ATA      HITACHI HUA7210S AC4A  /dev/sdf   1.00TB
[2:0:5:0]    disk    ATA      HITACHI HUA7210S AC5A  /dev/sdg   1.00TB
[2:0:6:0]    disk    ATA      HITACHI HUA7210S AC4A  /dev/sdh   1.00TB
[2:0:7:0]    disk    ATA      HITACHI HUA7210S AC4A  /dev/sdi   1.00TB
[2:0:8:0]    disk    ATA      HITACHI HUA7210S AC4A  /dev/sdj   1.00TB
Edited by pottlepaul
Link to comment

You cant pass through individual ports, you'd need to pass through the entire device.  You could probably create some bindings on the ports outside of the VM in unraid, then reference those bindings on a virtual adapter in the VM... although with this being a PFsense Vm, its probably worthwhile to pick up a dedicated NIC from ebay... not expensive and makes this a lot less complicated!

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.