Jump to content
jonp

Simpler / Easier PCI Device Pass Through for NON-GPUs

93 posts in this topic Last Reply

Recommended Posts

so i did:

 

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/>

      </source>

    </hostdev>

 

but when starting the VM i get error :internal error: Device 0000:0e:00.0 is already in use

 

Device 0e:00.0 is my sound card. ive been having issues passing PCI devices through  (USB Controller and PCI Sound card) and passing through USB devices is just as bad outside of the normal key+mouse... DAC wont go through, USB Headset will go through but audio is garbled... Help me please!

 

Share this post


Link to post

PCI Devices
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09)
00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)
00:16.0 Communication controller: Intel Corporation 7 Series/C210 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 (rev c4)
00:1c.2 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 3 (rev c4)
00:1c.3 PCI bridge: Intel Corporation 82801 PCI Bridge (rev c4)
00:1c.4 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 5 (rev c4)
00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation Z77 Express Chipset LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)
00:1f.3 SMBus: Intel Corporation 7 Series/C210 Series Chipset Family SMBus Controller (rev 04)
01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 660 Ti] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)
03:00.0 Ethernet controller: Qualcomm Atheros AR8151 v2.0 Gigabit Ethernet (rev c0)
04:00.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 30)
05:01.0 Multimedia audio controller: C-Media Electronics Inc CMI8788 [Oxygen HD Audio]
06:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
06:00.1 Audio device: NVIDIA Corporation High Definition Audio Controller (rev a1)

 

Added this to XML

 

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x01' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

 

Error:

 

Execution error

internal error: early end of file from monitor: possible problem:
2016-01-17T17:25:30.186112Z qemu-system-x86_64: -device vfio-pci,host=05:01.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: Error: Failed to setup INTx fd: Device or resource busy
2016-01-17T17:25:30.186287Z qemu-system-x86_64: -device vfio-pci,host=05:01.0,id=hostdev0,bus=pci.0,addr=0x6: Device initialization failed
2016-01-17T17:25:30.186298Z qemu-system-x86_64: -device vfio-pci,host=05:01.0,id=hostdev0,bus=pci.0,addr=0x6: Device 'vfio-pci' could not be initialized

 

Any ideas what I did wrong?

Share this post


Link to post

PCI Devices
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09)
00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)
00:16.0 Communication controller: Intel Corporation 7 Series/C210 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 (rev c4)
00:1c.2 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 3 (rev c4)
00:1c.3 PCI bridge: Intel Corporation 82801 PCI Bridge (rev c4)
00:1c.4 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 5 (rev c4)
00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation Z77 Express Chipset LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)
00:1f.3 SMBus: Intel Corporation 7 Series/C210 Series Chipset Family SMBus Controller (rev 04)
01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 660 Ti] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)
03:00.0 Ethernet controller: Qualcomm Atheros AR8151 v2.0 Gigabit Ethernet (rev c0)
04:00.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 30)
05:01.0 Multimedia audio controller: C-Media Electronics Inc CMI8788 [Oxygen HD Audio]
06:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
06:00.1 Audio device: NVIDIA Corporation High Definition Audio Controller (rev a1)

 

Added this to XML

 

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x01' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

 

Error:

 

Execution error

internal error: early end of file from monitor: possible problem:
2016-01-17T17:25:30.186112Z qemu-system-x86_64: -device vfio-pci,host=05:01.0,id=hostdev0,bus=pci.0,addr=0x6: vfio: Error: Failed to setup INTx fd: Device or resource busy
2016-01-17T17:25:30.186287Z qemu-system-x86_64: -device vfio-pci,host=05:01.0,id=hostdev0,bus=pci.0,addr=0x6: Device initialization failed
2016-01-17T17:25:30.186298Z qemu-system-x86_64: -device vfio-pci,host=05:01.0,id=hostdev0,bus=pci.0,addr=0x6: Device 'vfio-pci' could not be initialized

 

Any ideas what I did wrong?

 

Probably nothing you did. There are others with the same problem trying to pass through a Oxygen card. Both PCI and PCIe.

I don't think they found any solutions.

Share this post


Link to post

Hi,

 

Thank you very much, this made me able to pass my LSI Megaraid 2960 Pci-E controller to one of my VM`s, was finally made that i decided to move my private Virtualisation-/Storage-/Gaming-/All-in-One rig from Vsphere 6 to Unraid :)

 

Cheers Mate!

 

Share this post


Link to post

Hi,

 

Thank you very much, this made me able to pass my LSI Megaraid 2960 Pci-E controller to one of my VM`s, was finally made that i decided to move my private Virtualisation-/Storage-/Gaming-/All-in-One rig from Vsphere 6 to Unraid :)

 

Cheers Mate!

 

Hi

 

How did you get to boot from the LSI controller? I added my NVME controller like so...

 

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </hostdev>

 

But it tries to boot from the CDROM not the PCI device I just added. Any ideas?

Share this post


Link to post

Hi jonp,

 

Can I make do with pc-i440fx-2.3 machine type or must it be 'pc'? I'm not able to convert a q35 machine to pc - getting some sort of pci 000:00.00 error.

 

So I've tried to create a pc-i440fx-2.3 machine for ubuntu, but still getting errors when trying to pass through the below PCI-E tuner card.

 

Any thoughts on how I could pass through a DigitalNow Quad tuner which is appearing in the PCI devices list as:

05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa)

06:00.0 USB controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 62)

06:00.1 USB controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 62)

06:00.2 USB controller: VIA Technologies, Inc. USB 2.0 (rev 65)

 

I've enabled PCIe ACS Overrides, because previously these devices were appearing with a bunch of other ones in group 9, now they're separated into IOMMU group 16:

/sys/kernel/iommu_groups/16/devices/0000:05:00.0

/sys/kernel/iommu_groups/16/devices/0000:06:00.0

/sys/kernel/iommu_groups/16/devices/0000:06:00.1

/sys/kernel/iommu_groups/16/devices/0000:06:00.2

 

Here's my XML:

<domain type='kvm'>

  <name>Ubuntu</name>

  <uuid>f153e902-4e72-aab4-3996-cf39214fbfab</uuid>

  <metadata>

    <vmtemplate name="Custom" icon="ubuntu.png" os="ubuntu"/>

  </metadata>

  <memory unit='KiB'>4194304</memory>

  <currentMemory unit='KiB'>4194304</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>4</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='0'/>

    <vcpupin vcpu='1' cpuset='1'/>

    <vcpupin vcpu='2' cpuset='2'/>

    <vcpupin vcpu='3' cpuset='3'/>

  </cputune>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>

  </os>

  <features>

    <acpi/>

    <apic/>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='4' threads='1'/>

  </cpu>

  <clock offset='utc'>

    <timer name='rtc' tickpolicy='catchup'/>

    <timer name='pit' tickpolicy='delay'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/bin/qemu-system-x86_64</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/cache/VMs/Ubuntu.img'/>

      <target dev='hdb' bus='virtio'/>

      <boot order='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/Software/ISOs/Ubuntu/ubuntu-14.04.3-desktop-amd64.iso'/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <controller type='usb' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'/>

    <controller type='ide' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </controller>

    <filesystem type='mount' accessmode='passthrough'>

      <source dir='/mnt/user'/>

      <target dir='Shares'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </filesystem>

    <interface type='bridge'>

      <mac address='52:54:00:ad:5b:49'/>

      <source bridge='br0'/>

      <model type='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <channel type='unix'>

      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Ubuntu.org.qemu.guest_agent.0'/>

      <target type='virtio' name='org.qemu.guest_agent.0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <input type='tablet' bus='usb'/>

    <input type='mouse' bus='ps2'/>

    <input type='keyboard' bus='ps2'/>

    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'>

      <listen type='address' address='0.0.0.0'/>

    </graphics>

    <video>

      <model type='vmvga' vram='16384' heads='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </video>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x06' slot='0x00' function='0x2'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>

    </hostdev>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>

    </memballoon>

  </devices>

</domain>

 

Share this post


Link to post

I am having issues with this. The keyboard and mouse attached to the controller dont work after I pass an entire controller. Here is my xml what am I doing wrong?

 


    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0f' slot='0x00' function='0x0'/>
      </source>
    </hostdev>

 

here is other info that may be needed

 


root@andromeda:~# lspci |grep USB
00:14.0 USB controller: Intel Corporation C610/X99 series chipset USB xHCI Host Controller (rev 05)
00:1a.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #2 (rev 05)
00:1d.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #1 (rev 05)
0e:00.0 USB controller: ASMedia Technology Inc. Device 1242
0f:00.0 USB controller: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller <------the controller in question (I think)


Bus 001 Device 003: ID 174c:2074 ASMedia Technology Inc.
Bus 001 Device 002: ID 0b05:180a ASUSTek Computer, Inc.
Bus 001 Device 005: ID 1a2c:0e24 China Resource Semico Co., Ltd
Bus 001 Device 008: ID 258a:0001
Bus 001 Device 007: ID 093a:2521 Pixart Imaging, Inc. Optical Mouse
Bus 001 Device 004: ID 174c:2074 ASMedia Technology Inc.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 002 Device 004: ID 18a5:0250 Verbatim, Ltd
Bus 002 Device 003: ID 174c:3074 ASMedia Technology Inc.
Bus 002 Device 002: ID 174c:3074 ASMedia Technology Inc.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 003 Device 004: ID 093a:2521 Pixart Imaging, Inc. Optical Mouse
Bus 003 Device 003: ID 258a:0001
Bus 003 Device 002: ID 05e3:0608 Genesys Logic, Inc. Hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub


../../../devices/pci0000:00/0000:00:1c.4/0000:0e:00.0/usb3

Share this post


Link to post

it worked out of the box on the other vm. I think I may have the address wrong. but now running into another bug with the flash drive

Share this post


Link to post

it worked out of the box on the other vm. I think I may have the address wrong. but now running into another bug with the flash drive

The address looks correct from the info you posted. I don't have any other suggestions I'm afraid.

Share this post


Link to post

I'm trying to add two identical TV tuners,

 

If I paste this into the editor,

 

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/>

      </source>

      <source>

        <address domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/>

      </source>

    </hostdev>

 

it turns into this when I update.

 

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </hostdev>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>

    </memballoon>

 

Slots 6 and 8 must be the identical tuners. I can see one of the tuners in the Device Manager and it works.

 

What is the syntax for adding the two devices

 

Thanks

 

Share this post


Link to post

The syntax is the same for two cards as for one. So you need two separate hostdev tags like below.

 

  <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/>
      </source>
    </hostdev>

  <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/>
      </source>
    </hostdev>

Share this post


Link to post

Hi Getting device cannot start in the VM windows 10 machine?  Any ideas?  10GBe nic not supported by unraid so was thinking of passing them to a VM till driver get added to unraid.

 

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
</hostdev>

 

03:00.0 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
03:00.1 Ethernet controller: Intel Corporation Ethernet Connection X552/X557-AT 10GBASE-T
04:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
05:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

 

Screen.jpg.ce60f17925067c6c5cd94119925f4e9c.jpg

Screen2.jpg.6cafcd774b3dd5ed60c3e8299ca184b1.jpg

Share this post


Link to post

Ok was the machine type changed back i440fx from Q35 and nic and driver now work in the win10VM and a Server2012 VM

Share this post


Link to post

Hi Guys!

Trying to add a network card to my pfsense vm.

 

3Com Corporation 3c940              37:09.0 0200: 10b7:1700

Realtek RTL8111/8168/8411          03:00.0 0200: 10ec:8168

 

Added pci-stub.ids=10ec:8168,10b7:1700 to my syslinux.cfg

rebooted

 

Added the following line to my .xml-file

(Im only testing ONE card at the moment for a proof of concept)

 

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x37' slot='0x09' function='0x0'/>

      </source>

    </hostdev>

 

That was changed to:

(2 last lines automatically added)

 

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x37' slot='0x09' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x0'/>

    </hostdev>

 

However i receive the following error when trying to start the vm:

 

virsh start pfSense160325

error: Failed to start domain pfSense160325

error: internal error: early end of file from monitor: possible problem:

2016-03-25T20:01:38.358902Z qemu-system-x86_64: -device vfio-pci,host=37:09.0,id=hostdev0,bus=pci.2,addr=0x6: vfio: failed to set iommu for container: Operation not permitted

2016-03-25T20:01:38.358943Z qemu-system-x86_64: -device vfio-pci,host=37:09.0,id=hostdev0,bus=pci.2,addr=0x6: vfio: failed to setup container for group 13

2016-03-25T20:01:38.358957Z qemu-system-x86_64: -device vfio-pci,host=37:09.0,id=hostdev0,bus=pci.2,addr=0x6: vfio: failed to get group 13

2016-03-25T20:01:38.358969Z qemu-system-x86_64: -device vfio-pci,host=37:09.0,id=hostdev0,bus=pci.2,addr=0x6: Device initialization failed

2016-03-25T20:01:38.358982Z qemu-system-x86_64: -device vfio-pci,host=37:09.0,id=hostdev0,bus=pci.2,addr=0x6: Device 'vfio-pci' could not be initialized

 

Any clues?!

Share this post


Link to post

Hello Everybody! I need your help please!

 

So I try the aforementioned solution but it doesn’t seem to “fully” work with my usb card:

https://www.startech.com/ca/Cards-Adapters/USB-3.0/Cards/PCI-Express-USB-3-Card-4-Dedicated-Channels-4-Port~PEXUSB3S44V#

This card has 4 USB 3.0 with 4 controllers (1 each). I bought it specially for my unRaid VMs ! Indeed, all USB controllers (and PCI bridge - not sure it’s a good thing here) appear in different IOMMU group.

I’ve been trying to give 2 of my Windows 10 VM two USB each but while the pass through devices appear in each VM device manager, they do no work: I get a exclamation mark and it says device could not be started/do not operate (I’ve installed the genuine drivers from the card website).

 

I’ve also tried passing the PCI bridge behind the corresponding controller but I get the error in unRAID saying device does not exist.

 

Thanks to lspci and memory addressing I was able to pinpoint which USB stands behind which PCI bridge (letters).

 

06:00.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:2608] (Main physical PCI ?)

07:01.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:2608] (A)

07:02.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:2608] (B)

07:03.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:2608] ©

07:04.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:2608] (D)

08:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller [1912:0015] (rev 02) (A)

09:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller [1912:0015] (rev 02) (B)

0a:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller [1912:0015] (rev 02) ©

0b:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller [1912:0015] (rev 02) (D)

 

/sys/kernel/iommu_groups/37/devices/0000:06:00.0

/sys/kernel/iommu_groups/38/devices/0000:07:01.0

/sys/kernel/iommu_groups/39/devices/0000:07:02.0

/sys/kernel/iommu_groups/40/devices/0000:07:03.0

/sys/kernel/iommu_groups/41/devices/0000:07:04.0

/sys/kernel/iommu_groups/42/devices/0000:08:00.0

/sys/kernel/iommu_groups/43/devices/0000:09:00.0

/sys/kernel/iommu_groups/44/devices/0000:0a:00.0

/sys/kernel/iommu_groups/45/devices/0000:0b:00.0

 

Hypothesis #1 : should PCI bridge and USB controllers be part of the same iommu group ?

Hypothesis #2 : should I stub the card to hide it from UnRaid at boot ?

 

So here goes my headache… all yours now….you have 2 hours !  ;D

Capture.PNG.0780ba59699283f859cf8e668157a228.PNG

Share this post


Link to post

I want to pass through the firewire on my mobo to VMs, either Ubuntu or Windows, on my unRAID server.  My CPU is an i5-2500K which does not support Vt-d.  My mobo is an Asus P8H67M-EVO.

 

Here is the output of an lspci -tv:

-[0000:00]-+-00.0  Intel Corporation 2nd Generation Core Processor Family DRAM Controller
           +-01.0-[01]--
           +-02.0  Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller
           +-16.0  Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1
           +-1a.0  Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2
           +-1b.0  Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller
           +-1c.0-[02]--
           +-1c.3-[03]----00.0  VIA Technologies, Inc. VT6315 Series Firewire Controller
           +-1c.4-[04]----00.0  ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
           +-1c.5-[05]----00.0  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
           +-1c.6-[06-07]----00.0-[07]--
           +-1c.7-[08]----00.0  Marvell Technology Group Ltd. 88SE6121 SATA II / PATA Controller
           +-1d.0  Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1
           +-1f.0  Intel Corporation H67 Express Chipset Family LPC Controller
           +-1f.2  Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller
           \-1f.3  Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller

And here is the result of an lsusb:

Bus 002 Device 011: ID 0403:f850 Future Technology Devices International, Ltd USB-UIRT (Universal Infrared Receiver+Transmitter)
Bus 002 Device 009: ID 2040:4903 Hauppauge HS PVR
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 054c:0243 Sony Corp. MicroVault Flash Drive
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The Sony USB is the drive holding my unRAID install.

 

Is it possible for me to passthrough the firewire which is on 1c.3?  If so how would I do that?

 

 

Share this post


Link to post

I want to pass through the firewire on my mobo to VMs, either Ubuntu or Windows, on my unRAID server.  My CPU is an i5-2500K which does not support Vt-d.  My mobo is an Asus P8H67M-EVO.

 

Here is the output of an lspci -tv:

-[0000:00]-+-00.0  Intel Corporation 2nd Generation Core Processor Family DRAM Controller
           +-01.0-[01]--
           +-02.0  Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller
           +-16.0  Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1
           +-1a.0  Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2
           +-1b.0  Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller
           +-1c.0-[02]--
           +-1c.3-[03]----00.0  VIA Technologies, Inc. VT6315 Series Firewire Controller
           +-1c.4-[04]----00.0  ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
           +-1c.5-[05]----00.0  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
           +-1c.6-[06-07]----00.0-[07]--
           +-1c.7-[08]----00.0  Marvell Technology Group Ltd. 88SE6121 SATA II / PATA Controller
           +-1d.0  Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1
           +-1f.0  Intel Corporation H67 Express Chipset Family LPC Controller
           +-1f.2  Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller
           \-1f.3  Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller

And here is the result of an lsusb:

Bus 002 Device 011: ID 0403:f850 Future Technology Devices International, Ltd USB-UIRT (Universal Infrared Receiver+Transmitter)
Bus 002 Device 009: ID 2040:4903 Hauppauge HS PVR
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 054c:0243 Sony Corp. MicroVault Flash Drive
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The Sony USB is the drive holding my unRAID install.

 

Is it possible for me to passthrough the firewire which is on 1c.3?  If so how would I do that?

 

You need vt-d to pass through devices, so it's not possible.

Share this post


Link to post

I want to pass the Intel onboard raid controller to a windows 10 vm. 

 

My hardware specs are: 

Intel xeon ES v4 E5-2630
64GB ECC
Asrock X99 extreme4 
GTX 750 (  N750-2GD5/OCV1 )
VT-d enabled

 

I have enabled "PCIe ACS Override".  The output of the IOMMU group after enabling the override and rebooting is:

IOMMU group 23
	[8086:8d47] 00:1f.0 ISA bridge: Intel Corporation C610/X99 series chipset LPC Controller (rev 05)
	[8086:2822] 00:1f.2 RAID bus controller: Intel Corporation SATA Controller [RAID mode] (rev 05)
	[8086:8d22] 00:1f.3 SMBus: Intel Corporation C610/X99 series chipset SMBus Controller (rev 05)

I have tried to passthrough the device with: 

   <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
      </source>
    </hostdev>

While trying to launch the vm I get the following error:

internal error: qemu unexpectedly closed the monitor: 2017-02-28T22:30:08.326161Z qemu-system-x86_64: -device vfio-pci,host=00:1f.2,id=hostdev4,bus=pci.0,addr=0x8: vfio: error, group 23 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
2017-02-28T22:30:08.326194Z qemu-system-x86_64: -device vfio-pci,host=00:1f.2,id=hostdev4,bus=pci.0,addr=0x8: vfio: failed to get group 23
2017-02-28T22:30:08.326212Z qemu-system-x86_64: -device vfio-pci,host=00:1f.2,id=hostdev4,bus=pci.0,addr=0x8: Device initialization failed

I'm moving from Esxi. While using Esxi I managed to passthrough the raid controller. With unRAID in managed to passthrough the gpu, but struggling with the raid controller.

Do I need to detach the Raid controller from unRAID, because I can see the attached drives?

Any hints are appreciated.

Share this post


Link to post

Update:

 

I tried to set the pci stub of the Raid controller, which wasn't working. Then I tried to set all pci stubs of the group 23. Now I can see my Raid 5 of the intel raid controller within the VM.

I also tried to disable the "PCIe ACS Override" afterwards. Which had no effect at all, still working.

 

My syslinux.cfg now looks like like this: ( unrelevant parts removed )


label unRAID OS
  menu default
  kernel /bzimage
  append pci-stub.ids=8086:2822,8086:8d47,8086:8d22 initrd=/bzroot

 

The partition is encrypted with truecrypt/veracrypt, I can mount it without any issues and use it. The raid functionality seems to be given also. I accidentally didn't connect one drive properly and the raid controller re-build the array after reconnecting the missing drive.

 

The problem which I had also with Esxi persists. The Intel Storage Manager tool tells me the drive is incompatible, even everything seems to work. I think it's the raid oprom which isn't loaded within the VM. Is there any possibility to load the raid controller oprom within the VM?

 

I tried to change the settings in the Bios (CSM) to load the Raid oprom: BIOS/UEFI/Never. But that seem to not have any effect at all.

 

The log of the vm shows: 

 

2017-03-01T00:24:32.517154Z qemu-system-x86_64: vfio: Cannot reset device 0000:00:1f.2, no available reset mechanism.

 

Thanks!

 

Edited by duketwo

Share this post


Link to post

Works beautifully!  Even for a novice like me! Always wanted my VM on a discrete ethernet card, now I have in less than 20 minutes :)

 

The PCI device now even shows up in the GUI to passthrough - very clever!

 

Thanks JonP!!!

Edited by JonUKRed

Share this post


Link to post

I have two Innatech USB cards that I pass through to my two gaming VMs, used for Virtual Reality.  The two cards are:

Inateck PCI-E to USB 3.0 5-Port PCI Express Card and 15-Pin Power Connector

https://www.amazon.com/Inateck-Express-Connector-Controller-Internal/dp/B00FPIMJEW

and

Inateck Superspeed 7 Ports PCI-E to USB 3.0 Expansion Card

https://www.amazon.com/dp/B00FPIMICA/ref=psdc_284717_t2_B00FPIMJEW

 

These two cards are mapped at 0000:4e:00.0 and 0000:54:00.0.  The 4e card maps to PCI root at 0000:00:02.0 and the 54 card maps to 0000:00:1c.6.  The 4e goes through a PEX8747 at 0000:4b:00.0

 

I followed the instructions here and successfully pass the cards to the VMs.  They seem to work mostly ok--with occasional flakiness. The main flakiness is associated with my Vive touch controllers, that are attached to the 4e card.  The Vive touch controllers communicate to the Vive headset via a wireless USB chip.  There is a hub in the Vive headset.  My symptoms on the Vive are loss of one touch controller at a time, seemingly at random, though it feels like always when I need it most!  It happens on average about once every 2-5 minutes.  

 

When I launch the two VMs (Muspelheim with Oculus and 54 USB card, Vanaheim with Vive and 4e USB card), I get a similar warning in the virtio logs:

qemu-system-x86_64: -device vfio-pci,host=4e:00.0,addr=00.0: Failed to mmap 0000:4e:00.0 BAR 2. Performance may be slow

I am writing that warning by memory--sorry, I didn't copy the log when I went to work today.

 

My theory about this failure is that there the KVM remap via IOMMU didn't work because Unraid has claimed the devices somehow.  This gets some non-IOMMU or host assisted lookup that only occurs when unraid services the interrupt.  This leads to occasional timeouts to the controllers, which are already at a pretty long chain of USB/PCIE hops:

Xeon -> PCIe port -> PCIe bridge -> USB3.0 Hub (card) -> USB 3.0 hub (Vive)->Wireless USB (touch controller)

 

What can I do to gather more information to debug the issue here?

 

oasis_system.png

oasis_pcie.png

Share this post


Link to post

Here's the real message:

2017-03-21 02:59:32.014+0000: starting up libvirt version: 2.4.0, qemu version: 2.7.1, hostname: Yggdrasil
LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name guest=Vanaheim,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-Vanaheim/master-key.aes -machine pc-i440fx-2.7,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/b3afaf7c-3828-85e2-eb36-d5043a7b6175_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 16384 -realtime mlock=off -smp 8,sockets=1,cores=4,threads=2 -uuid b3afaf7c-3828-85e2-eb36-d5043a7b6175 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-1-Vanaheim/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device nec-usb-xhci,id=usb,bus=pci.0,ad54:00.0,id=hostdev2,bus=pci.0,addr=0x8 -device vfio-pci,host=06:00.0,id=hostdev3,bus=pci.0,addr=0xa -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x9 -msg timestamp=on
Domain id=1 is tainted: high-privileges
Domain id=1 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)
2017-03-21T02:59:34.425641Z qemu-system-x86_64: -device vfio-pci,host=54:00.0,id=hostdev2,bus=pci.0,addr=0x8: Failed to mmap 0000:54:00.0 BAR 2. Performance may be slow

Share this post


Link to post

I have also found that passing through my Alpine Ridge based USB controller (on the mobo), I can avoid this mmap error and pass through the USB devices.  However, I still get drops, just not as many.  I think the issue is that there are occasionally stutters in latency.  It is worse if the MMIO map isn't there.  Any latency longer that .5ms will cause timeouts that lead to dropping the link.  Here's an example from the vrserver.txt log from SteamVR:

Mon Mar 20 2017 20:05:43.621 - lighthouse: LHR-FFA49F41 C: SendOnPoseChange took 0.657625ms
Mon Mar 20 2017 20:05:43.621 - lighthouse: LHR-0D3CA9A9 H: SendOnPoseChange took 0.666565ms
Mon Mar 20 2017 20:05:47.883 - lighthouse: LHR-FFA49F41 C: IMU went off scale.
Mon Mar 20 2017 20:05:54.578 - lighthouse: LHR-F766BF45 C: SendOnPoseChange took 0.578845ms
Mon Mar 20 2017 20:05:54.578 - lighthouse: LHR-0D3CA9A9 H: SendOnPoseChange took 0.641143ms
Mon Mar 20 2017 20:05:54.578 - lighthouse: LHR-FFA49F41 C: SendOnPoseChange took 0.551746ms
Mon Mar 20 2017 20:06:31.147 - lighthouse: LHR-FFA49F41 C: SendOnPoseChange took 0.503695ms

Mon Mar 20 2017 20:07:20.464 - Capturing input focus for vrcompositor (8376). Old focus was 0. Focus stack size 0
Mon Mar 20 2017 20:07:20.464 - Sending InputFocusCaptured old=0 new=8376
Mon Mar 20 2017 20:07:21.832 - Releasing input focus for vrcompositor (8376) because of message. new focus is 0.
Mon Mar 20 2017 20:07:21.832 - Sending VREvent_InputFocusReleased old=8376 new=0
Mon Mar 20 2017 20:07:30.209 - 2 - entering standby
Mon Mar 20 2017 20:07:30.209 - lighthouse: Device LHR-FFA49F41 powering off upon entering standby.
Mon Mar 20 2017 20:07:30.890 - lighthouse: 8B14732891: Wireless controller disconnected
Mon Mar 20 2017 20:07:30.890 - lighthouse: LHR-FFA49F41: Disconnected from receiver 8B14732891
Mon Mar 20 2017 20:07:30.890 - lighthouse: LHR-FFA49F41 C: Dropped 1294 back-facing hits during the previous tracking session

 

Link to the rest of the VR logs:

https://drive.google.com/file/d/0B7ky_0GX-JUBUjBnTTAzUDBJUjg/view?usp=sharing

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.