psychofuzzie

I didn't get the error again after restarting my VM. I even added all my devices back in one go, and it worked. Thank you again for your help!

Went ahead and did this, same result: couldn't find device. Copied from USB Devices: USB Devices Bus 009 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 014 Device 002: ID 1b1c:0c03 Corsair Bus 014 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 013 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 010 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 016 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 015 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 008 Device 002: ID 0781:5571 SanDisk Corp. Cruzer Fit Bus 008 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 012 Device 002: ID 046d:c52b Logitech, Inc. Unifying Receiver Bus 012 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 011 Device 002: ID 0e6f:0213 Logic3 Bus 011 Device 003: ID 046d:c22d Logitech, Inc. G510 Gaming Keyboard Bus 011 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub I went ahead and removed all usb devices, which made the dang thing work! I've got full hot-swap capabilities on my USB 3.0 Ports. I'm going to attempt to add everything back one at a time and see if I can keep it, but honestly the whole reason I wanted this was just to be able to connect my externals so I could restore my stuff. I'm on 6.1.6. Thanks again for your help, I really do appreciate it. If/when I'm more inclined to utilize a usb port full time I'll come back and report my results.

You only pass through the last device number. You also have to enable the ACS override in the VM manager settings to separate the devices in their own groups. Thanks for the tip, I see that in other posts in this thread now. I've enabled ACS Override, so now the IOMMU groups look like this for the respective USB controller: /sys/kernel/iommu_groups/19/devices/0000:00:15.3 /sys/kernel/iommu_groups/29/devices/0000:0c:00.0 Using that information, I've changed the qemu section to this: <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=0c.00,bus=root.1,addr=00.0'/> </qemu:commandline> which throws this error: So I tried using hostdev. Here's that code: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> </source> </hostdev> Oddly enough, it throws this: and added this before the </hostdev> tag: <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> I rebooted my box, thinking that would clear it, but I still get the same error. I don't think I've missed anything else, but I just can't seem to hack it. Do I need to do the hot fix as described in this thread? http://lime-technology.com/forum/index.php?topic=44010.0 Thanks again for helping this noob. In the qemu:arg you forgot .0 in the host address. The error you receive on your second try have nothing to do with the hostdev tag you added. It's a USB device you have passed through that is not connected anymore. Did you have that USB device in one of the ports of the controller you are trying to pass through? Yes, I had two devices plugged into them. I made this change: <qemu:arg value='vfio-pci,host=0c:00.0,bus=root.1,addr=00.0'/> both with a colon and a period to see, and now get 'path is not accessible '/dev/bus/usb/006/003' no such file or directory'. I used midnight commander to see that there is an item '-003' in that directory, but then it just disappears. I've reseated the 3.0 cable on my mobo, and every time I reboot it's there for about 10 minutes, or until I try to launch my VM. Would it make any difference that before I turned on ACS they were in the same IOMMU group as my NIC? everything's separated now of course, I'm just thinking aloud.

You only pass through the last device number. You also have to enable the ACS override in the VM manager settings to separate the devices in their own groups. Thanks for the tip, I see that in other posts in this thread now. I've enabled ACS Override, so now the IOMMU groups look like this for the respective USB controller: /sys/kernel/iommu_groups/19/devices/0000:00:15.3 /sys/kernel/iommu_groups/29/devices/0000:0c:00.0 Using that information, I've changed the qemu section to this: <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=0c.00,bus=root.1,addr=00.0'/> </qemu:commandline> which throws this error: So I tried using hostdev. Here's that code: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> </source> </hostdev> Oddly enough, it throws this: and added this before the </hostdev> tag: <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> I rebooted my box, thinking that would clear it, but I still get the same error. I don't think I've missed anything else, but I just can't seem to hack it. Do I need to do the hot fix as described in this thread? http://lime-technology.com/forum/index.php?topic=44010.0 Thanks again for helping this noob.

IMMEDIATELY after posting this I saw a hot fix for such an issue. Please hold, I'll try that first.

Howdy folks, somewhat of a noob here, was curious if you could help. I'm trying to get my two 3.0 ports on the front of my rig to work, but I don't think I fully comprehend the directions here. I've seen bits and pieces about binding devices to a VFIO driver, but can't make heads or tails of it. What I've done so far in relation to the 12 steps listed at the top of this thread: 1. using KVM directly connected to rig, not SSH 2. found 13 controllers listed 3. Broke out the notepad 4. unRAID is on Bus 008 Device 003: SanDisk Corp. Cruzer Fit 5. Found the two ports I want on Bus 006 Device 003 and Bus 006 Device 004, both WD Externals plugged in 6. output from readlink /sys/bus/usb/devices/usb6 is ../../../devices/pci0000:00/0000:00:15.3/0000:0c:00.0/usb6 7. Where I get a bit confused, I'm assuming that 0000:00:15.3 and 0000:0c:00.0 are both devices though 8. Went to said place 9. List of all devices in Group 16, what I want in red: /sys/kernel/iommu_groups/16/devices/0000:00:15.0 /sys/kernel/iommu_groups/16/devices/0000:00:15.1 /sys/kernel/iommu_groups/16/devices/0000:00:15.2 /sys/kernel/iommu_groups/16/devices/0000:00:15.3 /sys/kernel/iommu_groups/16/devices/0000:0a:00.0 /sys/kernel/iommu_groups/16/devices/0000:0b:00.0 /sys/kernel/iommu_groups/16/devices/0000:0c:00.0 10. I'll skip, I'm on Version 6.1.6 11. I tried both. Here's my XML before changes: <domain type='kvm'> <name>Onyx-PC</name> <uuid>9e9576cb-1876-d455-2a0f-af749bff9088</uuid> <description>Windows Daily Driver</description> <metadata> <vmtemplate name="Custom" icon="windows.png" os="windows"/> </metadata> <memory unit='KiB'>12582912</memory> <currentMemory unit='KiB'>12582912</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>4</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='3'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='4' threads='1'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/vDisks/Onyx-PC/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/Array vDisks/Onyx-PC/vdisk2.img'/> <target dev='hdd' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/ISOs/Windows 7/Windows 7 SP1 Ultimate (64 Bit).iso'/> <target dev='hda' bus='ide'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/ISOs/VirtIOs/virtio-win-0.1.102.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:3e:9e:81'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Onyx-PC.org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='vmvga' vram='16384' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x1b1c'/> <product id='0x0c03'/> </source> </hostdev> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x0e6f'/> <product id='0x0213'/> </source> </hostdev> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x046d'/> <product id='0xc22d'/> </source> </hostdev> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x046d'/> <product id='0xc52b'/> </source> </hostdev> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x1058'/> <product id='0x0827'/> </source> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </memballoon> </devices> </domain> 11a. I added after </devices> and before </domain>: <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=00:15.3,bus=root.1,addr=00.0'/> </qemu:commandline> note I deleted the 'e' after pci in the 3rd line. I saw this variance in a few different posts. I get this error: internal error: early end of file from monitor: possible problem: 2016-03-26T21:38:17.331133Z qemu-system-x86_64: -device vfio-pci,host=00:15.3,bus=root.1,addr=00.0: vfio: error opening /dev/vfio/16: No such file or directory 2016-03-26T21:38:17.331156Z qemu-system-x86_64: -device vfio-pci,host=00:15.3,bus=root.1,addr=00.0: vfio: failed to get group 16 2016-03-26T21:38:17.331166Z qemu-system-x86_64: -device vfio-pci,host=00:15.3,bus=root.1,addr=00.0: Device initialization failed 2016-03-26T21:38:17.331175Z qemu-system-x86_64: -device vfio-pci,host=00:15.3,bus=root.1,addr=00.0: Device 'vfio-pci' could not be initialized I've tried many different combinations of things too, such as host=0c:00.0, etc, and just cant get it to fire up. 11b. I pasted the following before the line containing <memballoon model='virtio'>: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x00' slot='0x15' function='0x3'/> </source> </hostdev> and get this error message: Failed to bind PCI device '0000:00:15.3' to vfio-pci: No such device Obviously, I'm missing something. I've scoured everything that I would think would help, but could use some assistance. Thanks so much for looking at this for me!