***GUIDE*** Passing Through Network Controllers to unRAID 6 Virtual Machines


jonp

Recommended Posts

  • 2 weeks later...

So.. i just tried this and can report back a few things.

 

Firstly- my hardware and what i want to achieve.

HP  Gen8 Microserver. (1x pciex16 available)

Xeon E3 1220 v2

16Gb ECC DDR3

2x Broadcom nics on board (dual chipset)

3x 4Tb WD reds (1x parity, 2x data)

1x 120Gb Intel SSD Cache

1x Asus GTX 730 2Gb w/ HDMI out

1x Intel Quad Gigabit NIC - available but not preferred.

 

I want to run unraid, and VM a HTPC using the gpu directly. The main thing is I want to like to run pfSense on the NAS as a VM.

Also looking at using dockers for Plex. I would love a good google drive option too.

 

 

Stub - i take it this terminology mean tell the system not to use this device?

 

So when you put in the boot options that manufacturer code is not to be used there can be issues... such as Dual NIC controllers. Mine is a broadcom dual controller on board. If i say to not use the manufacturer i lose all connectivity, so a more granular approach would be awesome.

 

Now the I removed the GPU i actually wanted to use from my one pcie slot and dumped in a Intel quad gigbit card I can stub the intel manufacturer code for this card. Ok, seems like this will work.

 

Nope. brb with actually configs and info.

 

Starting VM

Requested operation is not valid: PCI device 0000:09:00.0 is not assignable

 

i went back and selected the option that was suggested. no luck

 

Intel quad id's

09:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
09:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
0a:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
0a:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)

 

The one in question:

09:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06)
Subsystem: Hewlett-Packard Company NC364T PCI Express Quad Port Gigabit Server Adapter [103c:704b]
Kernel driver in use: e1000e
Kernel modules: e1000e

 

Syslinux config

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append pcie_acs_override=downstream pci-stub.ids=103c:704b initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest

 

 

Hostdev actually re organises i think to group it.

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x0'/>
    </hostdev>

 

Any ideas?

 

I'd love to use one broadcomm for unraid and one for the red interface on pfsense, but i think the way it stubs i need to use a different hw vendor ID?

 

 

I'd love to get the broadcomms working.

03:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe
03:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe

 

03:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f]
Subsystem: Hewlett-Packard Company Device [103c:2133]
Kernel driver in use: tg3
Kernel modules: tg3
03:00.1 Ethernet controller [0200]: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f]
Subsystem: Hewlett-Packard Company Device [103c:2133]
Kernel driver in use: tg3
Kernel modules: tg3

 

Link to comment

So update:

Did some more reading and found this GEM of a post http://lime-technology.com/forum/index.php?topic=34872.msg325436#msg325436

 

Basically, enabled bridging on main network and called it green0 (bonding was disabled)

 

putty in via ssh

login

ifconfig -a

++ here I see eth0 is my main nic with an IP, eth1 has no IP ++

brctl show

++ I can see green0 has both eth0 & eth1 ++

 

brctl delif green0 eth1

 

++ start new bridge ++

brctl addbr red0

brctl stp red0 on

brctl addif red0 eth1

ifconfig eth1 up

ifconfig red0 up

 

Now from here it was as easy as making a new VM and choosing green0 for the network and red0 for the WAN. No PCI stuff.

 

Then after it all worked I added the config to the go file (check the link). I added delif eth1 from green0 just incase it was being added elsewhere.

 

Quick reboot, manual start of the array and everything came up fine!

 

Now I know this isnt hardware level passing through but it actually gets me up and running and now I can still have the gpu in my only pcie slot to enable a htpc type VM with HDMI.

 

This little HP Gen8 microserver is kicking arse!

 

  • Upvote 1
Link to comment
  • 2 weeks later...

Well, i have the virtual nic working.  I can see the internet from the VM.  After doing the steps in the original post thought i now have another nic that shows up in Device Manager, but it has a yellow icon.  I assumed that that means that i need to install drivers for it.  Since i thought that this process for to allow direct access to that nic, i assumed that i would just install it's actual drivers.  But windows doesn't seem to agree.

Link to comment

Success.

 

For anyone reading this in the future i am running a ASRock Extreme7+ motherboard (which has dual integrated network adapters)

 

These are the two integrated adapters:

Intel I219-V

Intel I211-AT

 

I had trouble getting unRaid to even boot when trying to use the I219-V. When the os would boot it couldn't get an ip.  Not sure what that was all about but the I211-AT worked fine.  Anyway, after going through the steps in this thread i was able to pass through the I219-V to a Windows 10VM. 

 

The steps here took me to the point where i had a yellow exclamation in Windows. All i needed to do after that was to install that network cards drivers.  I had trouble at first but i think that maybe i was using the wrong driver, or unzipping the archive from the vendor had issue.  This morning i re-unzipped the file and went into the device manager and pointed directly to the "PRO1000/Winx64" subfolder and the driver installed without a hitch.  I have a new IP from my dhcp server, and with the VirtIO nic now disabled i still have internet access, and i can remote desktop to the windows VM.

 

VT-d + unRaid = magic

 

 

Link to comment

07:00.0 is the ID for the nic, 08:00.0 is for the other but they are both the same manufacturing code of 8086:1533.

 

That's the reason you can't pass through one nic of most dual onboard nics.

Actually I've got 4 of the same nics and I'm passing 3 of them to 3 different Windows VMs.  2 - 32 bit Win7 pros and 1 - 64 bit Win7 pro.

 

 

I do it by not using the syslinux.cfg changes.  I only set up mine with the XML changes.

 

 

Like this:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>

 

 

03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
05:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
06:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

 

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append pci-stub.ids=11ab:6081 initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest

 

 

The PCI stub is for my AOC-SAT2-MV8 I'm passing through to the 64 bit Win7 pro VM.  Also you will notice I left "03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection" alone because it becomes eth0 before any of the VMs are started and all 4 of the nics show up in unRAID.

Link to comment

So you aren't stubbing the nic but you can still allocate it in that way? Is that how I'm reading this? And it's still hardware passthrough?

 

I pass through a single GigE NIC that is the same manufacturer as my on board NIC to a HTPC VM without stubbing it. I'm using a QEMU command though.

Link to comment

So you aren't stubbing the nic but you can still allocate it in that way? Is that how I'm reading this? And it's still hardware passthrough?

Correct.  When I start the VM I can see the nic that is passed through get de-allocated from unRAID so that it can be used in the VM.

 

 

Some of the log entries from when box starts up for the nics:

Jan 29 18:36:53 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/ifconfig eth0 down
Jan 29 18:36:53 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/ifconfig eth0 0.0.0.0 promisc up
Jan 29 18:36:54 Media2 kernel: device eth0 entered promiscuous mode
Jan 29 18:36:54 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/brctl addif br0 eth0
Jan 29 18:36:54 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/ifconfig eth1 down
Jan 29 18:36:54 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/ifconfig eth1 0.0.0.0 promisc up
Jan 29 18:36:54 Media2 kernel: device eth1 entered promiscuous mode
Jan 29 18:36:54 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/brctl addif br0 eth1
Jan 29 18:36:54 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/ifconfig eth2 down
Jan 29 18:36:54 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/ifconfig eth2 0.0.0.0 promisc up
Jan 29 18:36:54 Media2 kernel: device eth2 entered promiscuous mode
Jan 29 18:36:54 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/brctl addif br0 eth2
Jan 29 18:36:54 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/ifconfig eth3 down
Jan 29 18:36:54 Media2 logger: /etc/rc.d/rc.inet1:  /sbin/ifconfig eth3 0.0.0.0 promisc up
Jan 29 18:36:54 Media2 kernel: device eth3 entered promiscuous mode

 

 

Log entries from when a VM starts up.  NOTE: I don't have mine autostart yet.

Jan 29 18:40:22 Media2 kernel: e1000e 0000:04:00.0 eth1: removed PHC
Jan 29 18:40:22 Media2 kernel: e1000e: eth1 NIC Link is Down
Jan 29 18:40:22 Media2 kernel: br0: port 2(eth1) entered disabled state
Jan 29 18:40:22 Media2 avahi-daemon[5596]: Withdrawing workstation service for eth1.
Jan 29 18:40:22 Media2 kernel: br0: port 2(eth1) entered disabled state
Jan 29 18:40:23 Media2 kernel: vfio-pci 0000:0b:01.0: enabling device (0000 -> 0003)
Jan 29 18:40:23 Media2 kernel: vfio-pci 0000:00:1d.0: enabling device (0000 -> 0002)
Jan 29 18:40:23 Media2 kernel: vfio_cap_init: 0000:00:1d.0 hiding cap 0xa
Jan 29 18:40:23 Media2 kernel: vfio-pci 0000:04:00.0: enabling device (0000 -> 0003)

Link to comment
  • 4 weeks later...

So update:

Did some more reading and found this GEM of a post http://lime-technology.com/forum/index.php?topic=34872.msg325436#msg325436

 

Basically, enabled bridging on main network and called it green0 (bonding was disabled)

 

putty in via ssh

login

ifconfig -a

++ here I see eth0 is my main nic with an IP, eth1 has no IP ++

brctl show

++ I can see green0 has both eth0 & eth1 ++

 

brctl delif green0 eth1

 

++ start new bridge ++

brctl addbr red0

brctl stp red0 on

brctl addif red0 eth1

ifconfig eth1 up

ifconfig red0 up

 

Now from here it was as easy as making a new VM and choosing green0 for the network and red0 for the WAN. No PCI stuff.

 

Then after it all worked I added the config to the go file (check the link). I added delif eth1 from green0 just incase it was being added elsewhere.

 

Quick reboot, manual start of the array and everything came up fine!

 

Now I know this isnt hardware level passing through but it actually gets me up and running and now I can still have the gpu in my only pcie slot to enable a htpc type VM with HDMI.

 

This little HP Gen8 microserver is kicking arse!

 

I am about to run through his stuff using a quad-port NIC.  I think you skipped the MAC address & NIC model configuration.. So it likely presented your NICs to pfSense as Realtek (which suck for BSD).  If you use e1000 in his example, I don't think the checksum offloading should be an issue.

Link to comment

Hi all,

 

I'd like to pass my quad intel nic to 4 VM on my unraid box but cant seam to get it to work i keep getting errors.. mainly unassignable...  the 4 Ethernet controllers are what id like to pass through to 4 individual VM.... can someone help me?.. pr

 

here is my PCI Devices list

00:00.0 Host bridge: Intel Corporation Xeon E5 v3/Core i7 DMI2 (rev 02)
00:01.0 PCI bridge: Intel Corporation Xeon E5 v3/Core i7 PCI Express Root Port 1 (rev 02)
00:01.1 PCI bridge: Intel Corporation Xeon E5 v3/Core i7 PCI Express Root Port 1 (rev 02)
00:02.0 PCI bridge: Intel Corporation Xeon E5 v3/Core i7 PCI Express Root Port 2 (rev 02)
00:02.2 PCI bridge: Intel Corporation Xeon E5 v3/Core i7 PCI Express Root Port 2 (rev 02)
00:03.0 PCI bridge: Intel Corporation Xeon E5 v3/Core i7 PCI Express Root Port 3 (rev 02)
00:05.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Address Map, VTd_Misc, System Management (rev 02)
00:05.1 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Hot Plug (rev 02)
00:05.2 System peripheral: Intel Corporation Xeon E5 v3/Core i7 RAS, Control Status and Global Errors (rev 02)
00:05.4 PIC: Intel Corporation Xeon E5 v3/Core i7 I/O APIC (rev 02)
00:11.0 Unassigned class [ff00]: Intel Corporation C610/X99 series chipset SPSR (rev 05)
00:14.0 USB controller: Intel Corporation C610/X99 series chipset USB xHCI Host Controller (rev 05)
00:16.0 Communication controller: Intel Corporation C610/X99 series chipset MEI Controller #1 (rev 05)
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection (2) I218-V (rev 05)
00:1a.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #2 (rev 05)
00:1b.0 Audio device: Intel Corporation C610/X99 series chipset HD Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #1 (rev d5)
00:1c.3 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #4 (rev d5)
00:1c.4 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #5 (rev d5)
00:1c.6 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #7 (rev d5)
00:1d.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation C610/X99 series chipset LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)
00:1f.3 SMBus: Intel Corporation C610/X99 series chipset SMBus Controller (rev 05)
01:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 610] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1)
02:00.0 VGA compatible controller: NVIDIA Corporation Device 17c8 (rev a1)
02:00.1 Audio device: NVIDIA Corporation Device 0fb0 (rev a1)
03:00.0 VGA compatible controller: NVIDIA Corporation Device 17c8 (rev a1)
03:00.1 Audio device: NVIDIA Corporation Device 0fb0 (rev a1)
04:00.0 PCI bridge: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch (rev 0e)
05:02.0 PCI bridge: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch (rev 0e)
05:04.0 PCI bridge: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch (rev 0e)


06:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
06:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
07:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
07:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)


0a:00.0 Network controller: Broadcom Corporation BCM4360 802.11ac Wireless Network Adapter (rev 03)
0b:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
0c:00.0 USB controller: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller
ff:0b.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 R3 QPI Link 0 & 1 Monitoring (rev 02)
ff:0b.1 Performance counters: Intel Corporation Xeon E5 v3/Core i7 R3 QPI Link 0 & 1 Monitoring (rev 02)
ff:0b.2 Performance counters: Intel Corporation Xeon E5 v3/Core i7 R3 QPI Link 0 & 1 Monitoring (rev 02)
ff:0c.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Unicast Registers (rev 02)
ff:0c.1 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Unicast Registers (rev 02)
ff:0c.2 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Unicast Registers (rev 02)
ff:0c.3 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Unicast Registers (rev 02)
ff:0c.4 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Unicast Registers (rev 02)
ff:0c.5 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Unicast Registers (rev 02)
ff:0c.6 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Unicast Registers (rev 02)
ff:0c.7 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Unicast Registers (rev 02)
ff:0f.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Buffered Ring Agent (rev 02)
ff:0f.1 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Buffered Ring Agent (rev 02)
ff:0f.4 System peripheral: Intel Corporation Xeon E5 v3/Core i7 System Address Decoder & Broadcast Registers (rev 02)
ff:0f.5 System peripheral: Intel Corporation Xeon E5 v3/Core i7 System Address Decoder & Broadcast Registers (rev 02)
ff:0f.6 System peripheral: Intel Corporation Xeon E5 v3/Core i7 System Address Decoder & Broadcast Registers (rev 02)
ff:10.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02)
ff:10.1 Performance counters: Intel Corporation Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02)
ff:10.5 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Scratchpad & Semaphore Registers (rev 02)
ff:10.6 Performance counters: Intel Corporation Xeon E5 v3/Core i7 Scratchpad & Semaphore Registers (rev 02)
ff:10.7 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Scratchpad & Semaphore Registers (rev 02)
ff:12.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Home Agent 0 (rev 02)
ff:12.1 Performance counters: Intel Corporation Xeon E5 v3/Core i7 Home Agent 0 (rev 02)
ff:13.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Target Address, Thermal & RAS Registers (rev 02)
ff:13.1 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Target Address, Thermal & RAS Registers (rev 02)
ff:13.2 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder (rev 02)
ff:13.3 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder (rev 02)
ff:13.4 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder (rev 02)
ff:13.5 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder (rev 02)
ff:13.6 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO Channel 0/1 Broadcast (rev 02)
ff:13.7 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO Global Broadcast (rev 02)
ff:14.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 0 Thermal Control (rev 02)
ff:14.1 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 1 Thermal Control (rev 02)
ff:14.2 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 0 ERROR Registers (rev 02)
ff:14.3 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 1 ERROR Registers (rev 02)
ff:14.6 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 (rev 02)
ff:14.7 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 (rev 02)
ff:15.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 2 Thermal Control (rev 02)
ff:15.1 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 3 Thermal Control (rev 02)
ff:15.2 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 2 ERROR Registers (rev 02)
ff:15.3 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 3 ERROR Registers (rev 02)
ff:16.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 1 Target Address, Thermal & RAS Registers (rev 02)
ff:16.6 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO Channel 2/3 Broadcast (rev 02)
ff:16.7 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO Global Broadcast (rev 02)
ff:17.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel 0 Thermal Control (rev 02)
ff:17.4 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 (rev 02)
ff:17.5 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 (rev 02)
ff:17.6 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 (rev 02)
ff:17.7 System peripheral: Intel Corporation Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 (rev 02)
ff:1e.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Power Control Unit (rev 02)
ff:1e.1 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Power Control Unit (rev 02)
ff:1e.2 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Power Control Unit (rev 02)
ff:1e.3 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Power Control Unit (rev 02)
ff:1e.4 System peripheral: Intel Corporation Xeon E5 v3/Core i7 Power Control Unit (rev 02)
ff:1f.0 System peripheral: Intel Corporation Xeon E5 v3/Core i7 VCU (rev 02)
ff:1f.2 System peripheral: Intel Corporation Xeon E5 v3/Core i7 VCU (rev 02)

 

 

 

Link to comment

I see you have done exacly what i require... can i ash How?.. i keep getting errors ...

 

07:00.0 is the ID for the nic, 08:00.0 is for the other but they are both the same manufacturing code of 8086:1533.

 

That's the reason you can't pass through one nic of most dual onboard nics.

Actually I've got 4 of the same nics and I'm passing 3 of them to 3 different Windows VMs.  2 - 32 bit Win7 pros and 1 - 64 bit Win7 pro.

 

 

I do it by not using the syslinux.cfg changes.  I only set up mine with the XML changes.

 

 

Like this:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>

 

 

03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
05:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
06:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

 

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append pci-stub.ids=11ab:6081 initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest

 

 

The PCI stub is for my AOC-SAT2-MV8 I'm passing through to the 64 bit Win7 pro VM.  Also you will notice I left "03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection" alone because it becomes eth0 before any of the VMs are started and all 4 of the nics show up in unRAID.

Link to comment
  • 2 weeks later...

Tried both methods of passing through 4port NIC, but neither were successful

 

1. pci-stub.ids.. in syslinux.cfg but I'm unable to reach the webgui after change

2. added the XML changes by @BobPhoenix to my VM, but receive "Requested operation is not valid: PCI device 0000:04:00.0 is not assignable" when trying to start it

 

Any thoughts?

 

 

Link to comment

Tried both methods of passing through 4port NIC, but neither were successful

 

1. pci-stub.ids.. in syslinux.cfg but I'm unable to reach the webgui after change

2. added the XML changes by @BobPhoenix to my VM, but receive "Requested operation is not valid: PCI device 0000:04:00.0 is not assignable" when trying to start it

 

Any thoughts?

Would need IOMMU groups list as well as the PCI devices to work on this.  I don't remember seeing "PCI device ???? is not assignable" message before
Link to comment
Would need IOMMU groups list as well as the PCI devices to work on this.  I don't remember seeing "PCI device ???? is not assignable" message before

 

00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
00:14.0 USB controller: Intel Corporation 9 Series Chipset Family USB xHCI Controller
00:16.0 Communication controller: Intel Corporation 9 Series Chipset Family ME Interface #1
00:1a.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 9 Series Chipset Family HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 (rev d0)
00:1d.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1
00:1f.0 ISA bridge: Intel Corporation 9 Series Chipset Family Z97 LPC Controller
00:1f.2 SATA controller: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode]
00:1f.3 SMBus: Intel Corporation 9 Series Chipset Family SMBus Controller
01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)
02:00.0 PCI bridge: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch (rev 0e)
03:02.0 PCI bridge: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch (rev 0e)
03:04.0 PCI bridge: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch (rev 0e)
04:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
04:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
05:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
05:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)

 

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/2/devices/0000:00:01.1
/sys/kernel/iommu_groups/3/devices/0000:00:02.0
/sys/kernel/iommu_groups/4/devices/0000:00:03.0
/sys/kernel/iommu_groups/5/devices/0000:00:14.0
/sys/kernel/iommu_groups/6/devices/0000:00:16.0
/sys/kernel/iommu_groups/7/devices/0000:00:1a.0
/sys/kernel/iommu_groups/8/devices/0000:00:1b.0
/sys/kernel/iommu_groups/9/devices/0000:00:1c.0
/sys/kernel/iommu_groups/10/devices/0000:00:1d.0
/sys/kernel/iommu_groups/11/devices/0000:00:1f.0
/sys/kernel/iommu_groups/11/devices/0000:00:1f.2
/sys/kernel/iommu_groups/11/devices/0000:00:1f.3
/sys/kernel/iommu_groups/12/devices/0000:01:00.0
/sys/kernel/iommu_groups/12/devices/0000:01:00.1
/sys/kernel/iommu_groups/13/devices/0000:02:00.0
/sys/kernel/iommu_groups/14/devices/0000:03:02.0
/sys/kernel/iommu_groups/15/devices/0000:03:04.0
/sys/kernel/iommu_groups/16/devices/0000:04:00.0
/sys/kernel/iommu_groups/16/devices/0000:04:00.1
/sys/kernel/iommu_groups/17/devices/0000:05:00.0
/sys/kernel/iommu_groups/17/devices/0000:05:00.1

Link to comment

Would need IOMMU groups list as well as the PCI devices to work on this.  I don't remember seeing "PCI device ??? ? is not assignable" message before

 

00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
00:14.0 USB controller: Intel Corporation 9 Series Chipset Family USB xHCI Controller
00:16.0 Communication controller: Intel Corporation 9 Series Chipset Family ME Interface #1
00:1a.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 9 Series Chipset Family HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 (rev d0)
00:1d.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1
00:1f.0 ISA bridge: Intel Corporation 9 Series Chipset Family Z97 LPC Controller
00:1f.2 SATA controller: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode]
00:1f.3 SMBus: Intel Corporation 9 Series Chipset Family SMBus Controller
01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)
02:00.0 PCI bridge: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch (rev 0e)
03:02.0 PCI bridge: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch (rev 0e)
03:04.0 PCI bridge: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch (rev 0e)
04:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
04:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
05:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)
05:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06)

 

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/2/devices/0000:00:01.1
/sys/kernel/iommu_groups/3/devices/0000:00:02.0
/sys/kernel/iommu_groups/4/devices/0000:00:03.0
/sys/kernel/iommu_groups/5/devices/0000:00:14.0
/sys/kernel/iommu_groups/6/devices/0000:00:16.0
/sys/kernel/iommu_groups/7/devices/0000:00:1a.0
/sys/kernel/iommu_groups/8/devices/0000:00:1b.0
/sys/kernel/iommu_groups/9/devices/0000:00:1c.0
/sys/kernel/iommu_groups/10/devices/0000:00:1d.0
/sys/kernel/iommu_groups/11/devices/0000:00:1f.0
/sys/kernel/iommu_groups/11/devices/0000:00:1f.2
/sys/kernel/iommu_groups/11/devices/0000:00:1f.3
/sys/kernel/iommu_groups/12/devices/0000:01:00.0
/sys/kernel/iommu_groups/12/devices/0000:01:00.1
/sys/kernel/iommu_groups/13/devices/0000:02:00.0
/sys/kernel/iommu_groups/14/devices/0000:03:02.0
/sys/kernel/iommu_groups/15/devices/0000:03:04.0
/sys/kernel/iommu_groups/16/devices/0000:04:00.0
/sys/kernel/iommu_groups/16/devices/0000:04:00.1
/sys/kernel/iommu_groups/17/devices/0000:05:00.0
/sys/kernel/iommu_groups/17/devices/0000:05:00.1

Because of the IOMMU groups above you could pass two controllers through to each VM but not individually.  If you turn on the "PCIe ACS Override:" in VM settings you might be able to separate them but it still may not work.  If you don't turn on the ACS Override you will have better luck but you have to settle for passing both nics in the IOMMU group to the VM together.  So 04:00.0 & 04:00.1 would go to one VM and 05:00.0 & 05:00.1 to another.  I would expect that to work.  I must confess however I have never tried a quad nic card and pass through just my MB built in quad nics.

 

 

 

 

Edit: OH yes you can't just pass a single nic of the pair through at a time they would have to be passed together for it to work at all - if I'm right.

Link to comment
Because of the IOMMU groups above you could pass two controllers through to each VM but not individually.  If you turn on the "PCIe ACS Override:" in VM settings you might be able to separate them but it still may not work.  If you don't turn on the ACS Override you will have better luck but you have to settle for passing both nics in the IOMMU group to the VM together.  So 04:00.0 & 04:00.1 would go to one VM and 05:00.0 & 05:00.1 to another.  I would expect that to work.  I must confess however I have never tried a quad nic card and pass through just my MB built in quad nics.

 

 

Edit: OH yes you can't just pass a single nic of the pair through at a time they would have to be passed together for it to work at all - if I'm right.

 

 

I have PCIe ACS Override on to pass through my GPU I believe.

 

I also tried passing through one, a pair, and all four but still receive the same error.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.