rix Posted January 10, 2016 Share Posted January 10, 2016 These are the devices I want to pass through: 00:1b.0 Audio device: Intel Corporation 9 Series Chipset Family HD Audio Controller 01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 660 Ti] (rev a1) 05:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02) 06:00.0 USB controller: Fresco Logic Device 1100 (rev ff) Setting up the VM through the WebUI correctly set up one of my PCI buses (for the GPU) and enabled me to use my onboard audio device. These are all of my System devices: PCI Devices 00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06) 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06) 00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06) 00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06) 00:14.0 USB controller: Intel Corporation 9 Series Chipset Family USB xHCI Controller 00:16.0 Communication controller: Intel Corporation 9 Series Chipset Family ME Interface #1 00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-V 00:1a.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2 00:1b.0 Audio device: Intel Corporation 9 Series Chipset Family HD Audio Controller 00:1c.0 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 (rev d0) 00:1c.3 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 4 (rev d0) 00:1c.4 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 5 (rev d0) 00:1c.7 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 8 (rev d0) 00:1d.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1 00:1f.0 ISA bridge: Intel Corporation 9 Series Chipset Family H97 Controller 00:1f.2 SATA controller: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode] 00:1f.3 SMBus: Intel Corporation 9 Series Chipset Family SMBus Controller 01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 660 Ti] (rev a1) 01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1) 03:00.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 41) 05:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02) 06:00.0 USB controller: Fresco Logic Device 1100 (rev ff) What I have gathered from multipe guides here is that IOMMU Groups disable me from properly passing through single PCIe devices. Both the SATA and the USB controller are in group 9 of which I assume passing through all others (All PCI bridges) would not harm my Unraid setup. IOMMU Groups /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:01:00.0 /sys/kernel/iommu_groups/1/devices/0000:01:00.1 /sys/kernel/iommu_groups/2/devices/0000:00:02.0 /sys/kernel/iommu_groups/3/devices/0000:00:03.0 /sys/kernel/iommu_groups/4/devices/0000:00:14.0 /sys/kernel/iommu_groups/5/devices/0000:00:16.0 /sys/kernel/iommu_groups/6/devices/0000:00:19.0 /sys/kernel/iommu_groups/7/devices/0000:00:1a.0 /sys/kernel/iommu_groups/8/devices/0000:00:1b.0 /sys/kernel/iommu_groups/9/devices/0000:00:1c.0 /sys/kernel/iommu_groups/9/devices/0000:00:1c.3 /sys/kernel/iommu_groups/9/devices/0000:00:1c.4 /sys/kernel/iommu_groups/9/devices/0000:00:1c.7 /sys/kernel/iommu_groups/9/devices/0000:03:00.0 /sys/kernel/iommu_groups/9/devices/0000:05:00.0 /sys/kernel/iommu_groups/9/devices/0000:06:00.0 /sys/kernel/iommu_groups/10/devices/0000:00:1d.0 /sys/kernel/iommu_groups/11/devices/0000:00:1f.0 /sys/kernel/iommu_groups/11/devices/0000:00:1f.2 /sys/kernel/iommu_groups/11/devices/0000:00:1f.3 And this the currently relevant qemu section from my Windows 10 VM. I have manually added the SATA/USB controller, but have no Idea how to pass through the rest of group 9: <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=00:1b.0,bus=root.1,addr=01.0'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=05:00.0,bus=root.1,addr=02.0'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=06:00.0,bus=root.1,addr=03.0'/> </qemu:commandline> The VM starts with or without an enable ACS override, but only the override enables me to use the SATA/USB cards. With the override, however, my VM gets unstable. Could you help me setting up the qemu args for the whole IOMMU group (9) or explain in any other way how I can pass through all four devices (audio, gpu, sata, usb)? Quote Link to comment
saarg Posted January 10, 2016 Share Posted January 10, 2016 How does your Iommu groups look like after enabling the ACS override? Quote Link to comment
rix Posted January 11, 2016 Author Share Posted January 11, 2016 Thanks for the quick reply. I will check that later. For now, wouldn't it be easier to pass through: /sys/kernel/iommu_groups/9/devices/0000:00:1c.0 /sys/kernel/iommu_groups/9/devices/0000:00:1c.3 /sys/kernel/iommu_groups/9/devices/0000:00:1c.4 /sys/kernel/iommu_groups/9/devices/0000:00:1c.7 /sys/kernel/iommu_groups/9/devices/0000:03:00.0 /sys/kernel/iommu_groups/9/devices/0000:05:00.0 /sys/kernel/iommu_groups/9/devices/0000:06:00.0 entirely? I just have no Idea how to do the 00:1c.X devices. (the PCI bridges, that the webui set up as a hub for my NVIDIA card) Quote Link to comment
saarg Posted January 11, 2016 Share Posted January 11, 2016 You can't pass through PCI Bridges as far as I know. Quote Link to comment
lonix Posted January 12, 2016 Share Posted January 12, 2016 Mods please move this Quote Link to comment
rix Posted January 12, 2016 Author Share Posted January 12, 2016 Mods please move this Sorry, I intended to open this in the KVM section How does your Iommu groups look like after enabling the ACS override? IOMMU Groups Warning: Your system has booted with the PCIe ACS Override setting enabled. The below list doesn't not reflect the way IOMMU would naturally group devices. To see natural IOMMU groups for your hardware, go to the VM Settings page and set the PCIe ACS Override setting to No. /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/2/devices/0000:00:02.0 /sys/kernel/iommu_groups/3/devices/0000:00:03.0 /sys/kernel/iommu_groups/4/devices/0000:00:14.0 /sys/kernel/iommu_groups/5/devices/0000:00:16.0 /sys/kernel/iommu_groups/6/devices/0000:00:19.0 /sys/kernel/iommu_groups/7/devices/0000:00:1a.0 /sys/kernel/iommu_groups/8/devices/0000:00:1b.0 /sys/kernel/iommu_groups/9/devices/0000:00:1c.0 /sys/kernel/iommu_groups/10/devices/0000:00:1c.3 /sys/kernel/iommu_groups/11/devices/0000:00:1c.4 /sys/kernel/iommu_groups/12/devices/0000:00:1c.7 /sys/kernel/iommu_groups/13/devices/0000:00:1d.0 /sys/kernel/iommu_groups/14/devices/0000:00:1f.0 /sys/kernel/iommu_groups/14/devices/0000:00:1f.2 /sys/kernel/iommu_groups/14/devices/0000:00:1f.3 /sys/kernel/iommu_groups/15/devices/0000:01:00.0 /sys/kernel/iommu_groups/15/devices/0000:01:00.1 /sys/kernel/iommu_groups/16/devices/0000:03:00.0 /sys/kernel/iommu_groups/17/devices/0000:05:00.0 /sys/kernel/iommu_groups/18/devices/0000:06:00.0 The VM just booted fine with the above settings and the override enabled. Did I set up my pass through devices correctly? And would there be any way passing through the PCIe controllers without enabling the override? Quote Link to comment
saarg Posted January 12, 2016 Share Posted January 12, 2016 There is no way of passing through what you want without the ACS override. There haven't been any cases I have seen here on the forum caused by enabling the ACS override, so it should be safe enough to enable. Post your XML, and I'll see if you did it correctly. You might want to pass through the audio of the graphics card also, as it might lead to instability not passing it through. Quote Link to comment
rix Posted January 12, 2016 Author Share Posted January 12, 2016 Thanks for the help! <domain type='kvm' id='1' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>Windows 10</name> <uuid>f6b812e4-b3ab-3233-f043-253f85258b57</uuid> <description>rix's Gaming/Desktop</description> <metadata> <vmtemplate name="Custom" icon="windows.png" os="windows"/> </metadata> <memory unit='KiB'>10485760</memory> <currentMemory unit='KiB'>10485760</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>6</vcpu> <cputune> <vcpupin vcpu='0' cpuset='2'/> <vcpupin vcpu='1' cpuset='3'/> <vcpupin vcpu='2' cpuset='4'/> <vcpupin vcpu='3' cpuset='5'/> <vcpupin vcpu='4' cpuset='6'/> <vcpupin vcpu='5' cpuset='7'/> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='6' threads='1'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/cache/vdisks/Windows 10/vdisk1.img'/> <backingStore/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <controller type='usb' index='0'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:09:16:f4'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target port='0'/> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Windows 10.org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x045e'/> <product id='0x0768'/> <address bus='1' device='3'/> </source> <alias name='hostdev0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x1038'/> <product id='0x1361'/> <address bus='1' device='2'/> </source> <alias name='hostdev1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x0a12'/> <product id='0x0001'/> <address bus='1' device='5'/> </source> <alias name='hostdev2'/> </hostdev> <memballoon model='virtio'> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=00:1b.0,bus=root.1,addr=01.0'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=05:00.0,bus=root.1,addr=02.0'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=06:00.0,bus=root.1,addr=03.0'/> </qemu:commandline> </domain> That's what I have set up (and posting my reply from currently). An attached external HDD is recognized correctly, so that seems to be working. Quote Link to comment
saarg Posted January 12, 2016 Share Posted January 12, 2016 If it starts and you have the devices available in Windows it's right I would however pass through the audio device of the graphics card also, as many users have stability problems when not passing it through. This device should be passed through: 01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1) Quote Link to comment
rix Posted January 12, 2016 Author Share Posted January 12, 2016 If it starts and you have the devices available in Windows it's right I would however pass through the audio device of the graphics card also, as many users have stability problems when not passing it through. This device should be passed through: 01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1) alright, so something like: <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=root.1,addr=00.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=00:1b.0,bus=root.1,addr=01.0'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=05:00.0,bus=root.1,addr=02.0'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=06:00.0,bus=root.1,addr=03.0'/> </qemu:commandline> ? Quote Link to comment
saarg Posted January 12, 2016 Share Posted January 12, 2016 That looks good :-) Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.