Hi, I'd appreciate any help on this one.

 

I'm trying to pass my bt878-based PCI card to a Debian VM. I've followed some of the excellent guides here, but I can't seem to get it to work with both devices (functions) enabled.

 

What currently works is this (which only adds the first function of the device:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/>
    </hostdev>

 

Here's the info on the devices:

IOMMU group 10
05:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 03)
06:00.0 Multimedia video controller [0400]: Brooktree Corporation Bt878 Video Capture [109e:036e] (rev 11)
06:00.1 Multimedia controller [0480]: Brooktree Corporation Bt878 Audio Capture [109e:0878] (rev 11)

 

So, 06:00.0 & 06:00.1 should be passed.

 

If it's somehow easier to pass through the entire IOMMU group, that's completely fine as well.

It's all or nothing in the group. You'll need an entry for each device in the group. I've never passed a pci bridge and dont know if you could/would/should...

 

 

To get the 06:00.0 & 06:00.1 devices in their own group, you can try moving the card to another slot, or enable PCIe ACS Override in the vm manager to see if it will separate it out.

Yeah, the entire group is fine. I don't think I can separate the card to it's own group, I think it will always be in the PCI bridge's group.

 

How do I add an entire group though? Currently, I can only add one device from the group.

if it doesn't show up in the vm template editor, you'll have to edit the xml manually

 

so change this

 

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/>
    </hostdev>

 

 

to

 

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
      </source>
    </hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
    </hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
    </hostdev>

 

vm addresses removed and it would repopulate it accordingly.

 

again, don't know about passing the bridge to the vm... so... tell us how it goes! fingers crossed for you....

 

 

Unfortunately, that results in the following error when trying to run:

internal error: qemu unexpectedly closed the monitor: 2017-02-14T00:41:22.857869Z qemu-system-x86_64: -device vfio-pci,host=05:00.0,id=hostdev0,bus=pci.2,addr=0x4: vfio: error getting device 0000:05:00.0 from group 10: No such device
Verify all devices in group 10 are bound to vfio-<bus> or pci-stub and not already in use
2017-02-14T00:41:22.857919Z qemu-system-x86_64: -device vfio-pci,host=05:00.0,id=hostdev0,bus=pci.2,addr=0x4: vfio: failed to get device 0000:05:00.0
2017-02-14T00:41:22.871041Z qemu-system-x86_64: -device vfio-pci,host=05:00.0,id=hostdev0,bus=pci.2,addr=0x4: Device initialization failed

 

When I remove the PCI bridge, I get this instead:

internal error: qemu unexpectedly closed the monitor: 2017-02-14T00:42:37.708492Z qemu-system-x86_64: -device vfio-pci,host=06:00.1,id=hostdev1,bus=pci.2,addr=0x7: vfio: Error: Failed to setup INTx fd: Device or resource busy
2017-02-14T00:42:37.745806Z qemu-system-x86_64: -device vfio-pci,host=06:00.1,id=hostdev1,bus=pci.2,addr=0x7: Device initialization failed

 

I've also tried adding it as a multifunctional device using:

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x1' multifunction='on'/>
    </hostdev>

 

But that again results in this:

internal error: process exited while connecting to monitor: 2017-02-14T00:44:38.569622Z qemu-system-x86_64: -device vfio-pci,host=06:00.1,id=hostdev1,bus=pci.2,multifunction=on,addr=0x6.0x1: vfio: Error: Failed to setup INTx fd: Device or resource busy
2017-02-14T00:44:38.605890Z qemu-system-x86_64: -device vfio-pci,host=06:00.1,id=hostdev1,bus=pci.2,multifunction=on,addr=0x6.0x1: Device initialization failed

did you try enabling acs override to try and force the bridge out? (require reboot)

I haven't tried enabling ACS override yet, since there seem to be a lot of warnings about this possibly causing corruption in certain configurations.

 

However, despite my best google-fu, I cannot seem to find the post actually describing the issues.

 

From what I understand though, ACS override is used to allow devices in the same IOMMU group to be used in multiple VM's?

The intended goal is to help separate devices out that are in groups, allowing successful passthrough.

 

You don't have any other slots you could move the card to and see if it moves into its own group? It's safer!

 

 

there is also a way to do limited acs override by device id that is possible..... i've had to do it to separate an audio component off a video card.

You don't have any other slots you could move the card to and see if it moves into its own group? It's safer!

 

Yeah, this weird board (P10S-X) has like 5 PCI slots. I'll try that tomorrow!

Currently rebooting to try ACS override anyways.

 

EDIT: Group is still the same, getting the same error also.

Since the only other device is the PCIe to PCI bridge, I think all PCI slots will be in this group anyway,

you can try to use acs override on the card only by adding the following to syslinux.cfg after turning off acs override and rebooting:

 

append pcie_acs_override=id:109e:036e,109e:0878 initrd=/bzroot

Nope, still the same device initialization failed error unfortunately.

Both 'devices' work separately, but never at the same time.

 

EDIT: Actually, when I only add the second function of the device, it doesn't seem to show up in the guest OS using lspci...

can you just light it all on fire and get something else??

That'd be too easy now wouldn't it .

 

Anyway, I think it might be an issue with shared IRQs, since both functions of the device of course use the same IRQ. Using them separately works just fine, but as soon as both are added I get that "Failed to setup INTx fd: Device or resource busy" error.

 

One weird thing is that according to this link: http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM

VT-d spec specifies that all conventional PCI devices behind a PCIe-to PCI/PCI-X bridge or conventional PCI bridge can only be collectively assigned to the same guest. PCIe devices do not have this restriction.

 

I have the opposite problem I can only assign one device (function) behind the bridge to the guest at a time.

Just to note, the last sentence in that quote:

 

PCIe devices do not have this restriction.

Yeah, but the problem I'm having, is that I cannot collectively assign my device(s) which are behind a PCIe to PCI bridge. I can only use a single function at a time.

 

Another thing that I can think of that might cause this problem, is that the virtualization engine somehow fails in resetting one of the functions of the device, after already having reset the other one. Not sure how to fix this if that's the case.

Another thing that I can think of that might cause this problem, is that the virtualization engine somehow fails in resetting one of the functions of the device, after already having reset the other one. Not sure how to fix this if that's the case.

 

 

vvvvvvvvv

 

can you just light it all on fire and get something else??