October 26, 201510 yr I'm trying to pass through 2 ports on a 4 port NIC card. I've followed the instructions closely and can't seem to get past the message: Error: Requested operation is not valid: PCI device 0000:05:00.0 is not assignable I'm trying to pass through the NIC because of an issue running KMTTG under a Windows 8.1 VM. KMTTG is a tool that downloads video files from a Tivo and then re-encodes them. This process has worked very well under my old ESXi/Unraid config. However under Unraid6/KVM processing hangs up when using a share on Unraid as the target for the files. If I point the target directories to the c: drive on the VM it never fails. If I point to a share on the host Unraid6 server or another physical unraid server it hangs/fails about 50% of the time. The problem appears related to a network or samba issue but I don't see how to confirm. My next troubleshooting attempt is to passtrhough a NIC and take the KVM network out of the picture. lspci output and iommu group information: root@Unraid6:~# lspci -knn ... 03:00.0 PCI bridge [0604]: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch [111d:8018] (rev 0e) Kernel driver in use: pcieport 04:02.0 PCI bridge [0604]: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch [111d:8018] (rev 0e) Kernel driver in use: pcieport 04:04.0 PCI bridge [0604]: Integrated Device Technology, Inc. [iDT] PES12N3A PCI Express Switch [111d:8018] (rev 0e) Kernel driver in use: pcieport 05:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06) Subsystem: Hewlett-Packard Company NC364T PCI Express Quad Port Gigabit Server Adapter [103c:704b] Kernel driver in use: e1000e Kernel modules: e1000e 05:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06) Subsystem: Hewlett-Packard Company NC364T PCI Express Quad Port Gigabit Server Adapter [103c:704b] Kernel driver in use: e1000e Kernel modules: e1000e 06:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06) Subsystem: Hewlett-Packard Company NC364T PCI Express Quad Port Gigabit Server Adapter [103c:704b] Kernel driver in use: e1000e Kernel modules: e1000e 06:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06) Subsystem: Hewlett-Packard Company NC364T PCI Express Quad Port Gigabit Server Adapter [103c:704b] Kernel driver in use: e1000e Kernel modules: e1000e ... root@Unraid6:~# find /sys/kernel/iommu_groups/ -type l ... /sys/kernel/iommu_groups/21/devices/0000:02:00.0 /sys/kernel/iommu_groups/22/devices/0000:03:00.0 /sys/kernel/iommu_groups/23/devices/0000:04:02.0 /sys/kernel/iommu_groups/24/devices/0000:04:04.0 /sys/kernel/iommu_groups/25/devices/0000:05:00.0 /sys/kernel/iommu_groups/25/devices/0000:05:00.1 /sys/kernel/iommu_groups/26/devices/0000:06:00.0 /sys/kernel/iommu_groups/26/devices/0000:06:00.1 ... I've tried editing the append section in the syslinux.cfg file to keep 8086:10bc and 103c:704b from binding and each combination of the two. default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append pci-stub.ids=8086:153b,103c:704b amd_iommu=on iommu=pt iommu=1 vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest append pci-stub.ids=8086:153b,103c:704b amd_iommu=on iommu=pt iommu=1 vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot I've even tried adding this to the go file /usr/local/sbin/vfio-bind 0000:05:00.0 0000:05:00.1 0000:04:02.0 0000:06:00.0 0000:06:00.1 0000:04:04.0 Here is the xml for the VM config file. <domain type='kvm'> <name>W8-KMTTG</name> <uuid>50ad9020-9607-d8a1-d559-e716479f3ebb</uuid> <metadata> <vmtemplate name="Custom" icon="windows.png" os="windows"/> </metadata> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='3'/> <vcpupin vcpu='4' cpuset='4'/> <vcpupin vcpu='5' cpuset='5'/> <vcpupin vcpu='6' cpuset='6'/> <vcpupin vcpu='7' cpuset='7'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> </hyperv> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='8' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/cache/vms/W8-KMTTG/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/cache/iso/en_windows_8.1_with_update_x64_dvd_4065090.iso'/> <target dev='hda' bus='ide'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/iso/virtio-win-0.1.96.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:72:d1:4d'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/W8-KMTTG.org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='vmvga' vram='16384' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </memballoon> </devices> </domain> Does anyone have any ideas?
October 26, 201510 yr Both 05:00.0 and 05:00.1 is in the same iommu group, so you have to pass through both. Is the iommu group listing before or after you enabled PCIe ACS Override?
October 26, 201510 yr Author I've tried passing through both 05:00.0 and 05:00.1 without success. I've also tried 04:02.0 and 04:04.2 and nothing changed. The iommu group listing is from after setting up the following in the syslinux.cfg file. append pci-stub.ids=8086:153b,103c:704b amd_iommu=on iommu=pt iommu=1 vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot the one thing I didn't quite get intially was the values to put in pci-stub.ids. Initially I only put 8086:10bc but later added 103c:704b based on running lspci -knn which showed two ids that might be an issue. 05:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06) Subsystem: Hewlett-Packard Company NC364T PCI Express Quad Port Gigabit Server Adapter [103c:704b]
October 26, 201510 yr You do have a system that supports PCI pass through? What is your motherboard/CPU? What does it say about iommu if you click the info button in the unraid GUI?
October 26, 201510 yr Author IOMMU is supported and already working with Video and USB pass through on other VMs. The motherboard and CPU are GA-990FXA-UD7 / FX8350
October 26, 201510 yr It might be that the card doesn't want to be passed through for some reasons. Do you have any other slots you can put it in to try? Does it work to pass through the whole card? Do you know if the card has been passed through by anyone else? Nothing related to your problem, but you don't need both iommu=1 and iommu_amd=on in your syslinux.cfg. I don't think you need any of them.
December 6, 201510 yr Not sure if you found a solution to this but, I had the same issue however I was trying to pass all four ports. Here is a link to the thread with the solution that worked for me. http://lime-technology.com/forum/index.php?topic=44147.0
Archived
This topic is now archived and is closed to further replies.