Omnicrash Posted February 13, 2017 Share Posted February 13, 2017 Hi, I'd appreciate any help on this one. I'm trying to pass my bt878-based PCI card to a Debian VM. I've followed some of the excellent guides here, but I can't seem to get it to work with both devices (functions) enabled. What currently works is this (which only adds the first function of the device: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </hostdev> Here's the info on the devices: IOMMU group 10 05:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 03) 06:00.0 Multimedia video controller [0400]: Brooktree Corporation Bt878 Video Capture [109e:036e] (rev 11) 06:00.1 Multimedia controller [0480]: Brooktree Corporation Bt878 Audio Capture [109e:0878] (rev 11) So, 06:00.0 & 06:00.1 should be passed. If it's somehow easier to pass through the entire IOMMU group, that's completely fine as well. Quote Link to comment
1812 Posted February 13, 2017 Share Posted February 13, 2017 It's all or nothing in the group. You'll need an entry for each device in the group. I've never passed a pci bridge and dont know if you could/would/should... To get the 06:00.0 & 06:00.1 devices in their own group, you can try moving the card to another slot, or enable PCIe ACS Override in the vm manager to see if it will separate it out. Quote Link to comment
Omnicrash Posted February 14, 2017 Author Share Posted February 14, 2017 Yeah, the entire group is fine. I don't think I can separate the card to it's own group, I think it will always be in the PCI bridge's group. How do I add an entire group though? Currently, I can only add one device from the group. Quote Link to comment
1812 Posted February 14, 2017 Share Posted February 14, 2017 if it doesn't show up in the vm template editor, you'll have to edit the xml manually so change this <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </hostdev> to <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/> </source> </hostdev> vm addresses removed and it would repopulate it accordingly. again, don't know about passing the bridge to the vm... so... tell us how it goes! fingers crossed for you.... Quote Link to comment
Omnicrash Posted February 14, 2017 Author Share Posted February 14, 2017 Unfortunately, that results in the following error when trying to run: internal error: qemu unexpectedly closed the monitor: 2017-02-14T00:41:22.857869Z qemu-system-x86_64: -device vfio-pci,host=05:00.0,id=hostdev0,bus=pci.2,addr=0x4: vfio: error getting device 0000:05:00.0 from group 10: No such device Verify all devices in group 10 are bound to vfio-<bus> or pci-stub and not already in use 2017-02-14T00:41:22.857919Z qemu-system-x86_64: -device vfio-pci,host=05:00.0,id=hostdev0,bus=pci.2,addr=0x4: vfio: failed to get device 0000:05:00.0 2017-02-14T00:41:22.871041Z qemu-system-x86_64: -device vfio-pci,host=05:00.0,id=hostdev0,bus=pci.2,addr=0x4: Device initialization failed When I remove the PCI bridge, I get this instead: internal error: qemu unexpectedly closed the monitor: 2017-02-14T00:42:37.708492Z qemu-system-x86_64: -device vfio-pci,host=06:00.1,id=hostdev1,bus=pci.2,addr=0x7: vfio: Error: Failed to setup INTx fd: Device or resource busy 2017-02-14T00:42:37.745806Z qemu-system-x86_64: -device vfio-pci,host=06:00.1,id=hostdev1,bus=pci.2,addr=0x7: Device initialization failed I've also tried adding it as a multifunctional device using: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x02' slot='0x06' function='0x1' multifunction='on'/> </hostdev> But that again results in this: internal error: process exited while connecting to monitor: 2017-02-14T00:44:38.569622Z qemu-system-x86_64: -device vfio-pci,host=06:00.1,id=hostdev1,bus=pci.2,multifunction=on,addr=0x6.0x1: vfio: Error: Failed to setup INTx fd: Device or resource busy 2017-02-14T00:44:38.605890Z qemu-system-x86_64: -device vfio-pci,host=06:00.1,id=hostdev1,bus=pci.2,multifunction=on,addr=0x6.0x1: Device initialization failed Quote Link to comment
1812 Posted February 14, 2017 Share Posted February 14, 2017 did you try enabling acs override to try and force the bridge out? (require reboot) Quote Link to comment
Omnicrash Posted February 14, 2017 Author Share Posted February 14, 2017 I haven't tried enabling ACS override yet, since there seem to be a lot of warnings about this possibly causing corruption in certain configurations. However, despite my best google-fu, I cannot seem to find the post actually describing the issues. From what I understand though, ACS override is used to allow devices in the same IOMMU group to be used in multiple VM's? Quote Link to comment
1812 Posted February 14, 2017 Share Posted February 14, 2017 The intended goal is to help separate devices out that are in groups, allowing successful passthrough. You don't have any other slots you could move the card to and see if it moves into its own group? It's safer! Quote Link to comment
1812 Posted February 14, 2017 Share Posted February 14, 2017 there is also a way to do limited acs override by device id that is possible..... i've had to do it to separate an audio component off a video card. Quote Link to comment
Omnicrash Posted February 14, 2017 Author Share Posted February 14, 2017 You don't have any other slots you could move the card to and see if it moves into its own group? It's safer! Yeah, this weird board (P10S-X) has like 5 PCI slots. I'll try that tomorrow! Currently rebooting to try ACS override anyways. EDIT: Group is still the same, getting the same error also. Since the only other device is the PCIe to PCI bridge, I think all PCI slots will be in this group anyway, Quote Link to comment
1812 Posted February 14, 2017 Share Posted February 14, 2017 you can try to use acs override on the card only by adding the following to syslinux.cfg after turning off acs override and rebooting: append pcie_acs_override=id:109e:036e,109e:0878 initrd=/bzroot Quote Link to comment
Omnicrash Posted February 14, 2017 Author Share Posted February 14, 2017 Nope, still the same device initialization failed error unfortunately. Both 'devices' work separately, but never at the same time. EDIT: Actually, when I only add the second function of the device, it doesn't seem to show up in the guest OS using lspci... Quote Link to comment
1812 Posted February 14, 2017 Share Posted February 14, 2017 can you just light it all on fire and get something else?? Quote Link to comment
Omnicrash Posted February 14, 2017 Author Share Posted February 14, 2017 That'd be too easy now wouldn't it . Anyway, I think it might be an issue with shared IRQs, since both functions of the device of course use the same IRQ. Using them separately works just fine, but as soon as both are added I get that "Failed to setup INTx fd: Device or resource busy" error. One weird thing is that according to this link: http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM VT-d spec specifies that all conventional PCI devices behind a PCIe-to PCI/PCI-X bridge or conventional PCI bridge can only be collectively assigned to the same guest. PCIe devices do not have this restriction. I have the opposite problem I can only assign one device (function) behind the bridge to the guest at a time. Quote Link to comment
jonp Posted February 14, 2017 Share Posted February 14, 2017 Just to note, the last sentence in that quote: PCIe devices do not have this restriction. Quote Link to comment
Omnicrash Posted February 14, 2017 Author Share Posted February 14, 2017 Yeah, but the problem I'm having, is that I cannot collectively assign my device(s) which are behind a PCIe to PCI bridge. I can only use a single function at a time. Another thing that I can think of that might cause this problem, is that the virtualization engine somehow fails in resetting one of the functions of the device, after already having reset the other one. Not sure how to fix this if that's the case. Quote Link to comment
1812 Posted February 15, 2017 Share Posted February 15, 2017 Another thing that I can think of that might cause this problem, is that the virtualization engine somehow fails in resetting one of the functions of the device, after already having reset the other one. Not sure how to fix this if that's the case. vvvvvvvvv can you just light it all on fire and get something else?? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.