Hoopster Posted December 31, 2016 Share Posted December 31, 2016 I originally created a Win 10 VM with a different MB, CPU and RAM than I now have in the server after upgrading. Of course, I suspected there would be some issues after the hardware upgrade. I went from a Haswell (i5 4590) to a Skylake (Xeon E3-1245 V5) CPU. I understand the Skylake CPUs do have some issues with iommu groups. Here is the error I receive when trying to start the VM: Execution error internal error: process exited while connecting to monitor: 2016-12-29T19:43:57.132717Z qemu-system-x86_64: -device vfio-pci,host=00:1f.3,id=hostdev0,bus=pci.0,addr=0x9: vfio: error, group 6 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver. 2016-12-29T19:43:57.132734Z qemu-system-x86_64: -device vfio-pci,host=00:1f.3,id=hostdev0,bus=pci.0,addr=0x9: vfio: failed to get group 6 2016-12-29T19:43:57.132741Z qemu-system-x86_64: -device vfio-pci,host=00:1f.3,id=hostdev0,bus=pci.0,addr=0x9: Device initialization failed Here is the XML Code for the VM: <domain type='kvm'> <name>Windows Desktop</name> <uuid>7954b90a-bc51-fdb9-a644-f3584d0a1cce</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>2</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='5'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/7954b90a-bc51-fdb9-a644-f3584d0a1cce_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor id='none'/> </hyperv> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='1' threads='2'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/VMs/Windows Desktop/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/VMs/WinApps/vdisk2.img'/> <target dev='hdd' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/VMFiles/Windows ISOs/Windows10.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/VMFiles/Windows ISOs/virtio-win-0.1.118.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='nec-xhci'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:27:43:5f'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <source mode='connect'/> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x00' slot='0x1f' function='0x3'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> </memballoon> </devices> </domain> Machine is i440fx-2.3 BIOS: OVMF VirtIO version is 1.118 Video is VNC QXL (have not yet installed PCIE video card) I know enough (not too much) about VMs to be dangerous. Where should I begin to resolve this issue? Are the changes to the system too many to simply edit existing VM config? Thanks in advance for any ideas you VM experts may have. Link to comment
1812 Posted December 31, 2016 Share Posted December 31, 2016 did you see what other devices are in iommu group 6? Link to comment
Hoopster Posted January 2, 2017 Author Share Posted January 2, 2017 did you see what other devices are in iommu group 6? If I am reading this ouput properly, the highlighted items are in IOMMU group 6 /sys/kernel/iommu_groups /sys/kernel/iommu_groups/0 /sys/kernel/iommu_groups/0/devices /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1 /sys/kernel/iommu_groups/1/devices /sys/kernel/iommu_groups/1/devices/0000:00:02.0 /sys/kernel/iommu_groups/2 /sys/kernel/iommu_groups/2/devices /sys/kernel/iommu_groups/2/devices/0000:00:14.0 /sys/kernel/iommu_groups/2/devices/0000:00:14.2 /sys/kernel/iommu_groups/3 /sys/kernel/iommu_groups/3/devices /sys/kernel/iommu_groups/3/devices/0000:00:16.0 /sys/kernel/iommu_groups/3/devices/0000:00:16.3 /sys/kernel/iommu_groups/4 /sys/kernel/iommu_groups/4/devices /sys/kernel/iommu_groups/4/devices/0000:00:17.0 /sys/kernel/iommu_groups/5 /sys/kernel/iommu_groups/5/devices /sys/kernel/iommu_groups/5/devices/0000:00:1c.0 /sys/kernel/iommu_groups/5/devices/0000:00:1c.7 /sys/kernel/iommu_groups/5/devices/0000:02:00.0 /sys/kernel/iommu_groups/6 /sys/kernel/iommu_groups/6/devices /sys/kernel/iommu_groups/6/devices/0000:00:1f.0 /sys/kernel/iommu_groups/6/devices/0000:00:1f.2 /sys/kernel/iommu_groups/6/devices/0000:00:1f.3 /sys/kernel/iommu_groups/6/devices/0000:00:1f.4 /sys/kernel/iommu_groups/7 /sys/kernel/iommu_groups/7/devices /sys/kernel/iommu_groups/7/devices/0000:00:1f.6 00:00.0 Host bridge: Intel Corporation Skylake Host Bridge/DRAM Registers (rev 07) 00:02.0 VGA compatible controller: Intel Corporation HD Graphics P530 (rev 06) 00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller (rev 31) 00:14.2 Signal processing controller: Intel Corporation Sunrise Point-H Thermal subsystem (rev 31) 00:16.0 Communication controller: Intel Corporation Sunrise Point-H CSME HECI #1 (rev 31) 00:16.3 Serial controller: Intel Corporation Sunrise Point-H KT Redirection (rev 31) 00:17.0 SATA controller: Intel Corporation Sunrise Point-H SATA controller [AHCI mode] (rev 31) 00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #1 (rev f1) 00:1c.7 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #8 (rev f1) 00:1f.0 ISA bridge: Intel Corporation Sunrise Point-H LPC Controller (rev 31) 00:1f.2 Memory controller: Intel Corporation Sunrise Point-H PMC (rev 31) 00:1f.3 Audio device: Intel Corporation Sunrise Point-H HD Audio (rev 31) 00:1f.4 SMBus: Intel Corporation Sunrise Point-H SMBus (rev 31) 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-LM (rev 31) 02:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) Link to comment
1812 Posted January 2, 2017 Share Posted January 2, 2017 As I understand it, everything in a group has to be passed to the vm. right now you're only passing through 0000:00:1f.3. so the other 3 devices in the group have to go as well. <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x00' slot='0x1f' function='0x3'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> that is just 0000:00:1f.3 if you don't want to send everything to it, you can try to turn on acs override and see if it will split them up. Link to comment
Hoopster Posted January 2, 2017 Author Share Posted January 2, 2017 Because of hardware changes, I am sure the groups and device assignments have changed. In group six, the only thing currently being passed through is the audio which I do want to pass through. Is there any need to pass through the ISA Bridge, memory controller and SMBus? I have set ACS override to yes in VM settings and will try that, but, I am in the middle of copying a bunch of files to the NAS so I cannot reboot right now. Link to comment
1812 Posted January 2, 2017 Share Posted January 2, 2017 your xml shows it is setup to pass the audio to the vm. so if you don't want it, take it out either by deselecting the audio device in the vm manager or deleting the previous code from the xml. i'd do that first before acs override. plus, that doesn't require rebooting. Link to comment
Hoopster Posted January 2, 2017 Author Share Posted January 2, 2017 your xml shows it is setup to pass the audio to the vm. so if you don't want it, take it out either by deselecting the audio device in the vm manager or deleting the previous code from the xml. i'd do that first before acs override. plus, that doesn't require rebooting. I DO want to pass through the audio. It is the rest of the devices in the group that I wonder if I need to pass through. By the way, I enabled ACS override and rebooted (stopped data transfer first) and it made no change to the IOMMU groups. All are as they were before enabling ACS override. Since I cannot seem to get the audio separated from the other items in the group, is it just a matter of editing the XML to pass the other devices as well adding <source> tags for the other devices? Will passing through the ISA bridge, memory controller and SMBus to a VM affect the core unRAID server negatively? Link to comment
1812 Posted January 2, 2017 Share Posted January 2, 2017 your xml shows it is setup to pass the audio to the vm. so if you don't want it, take it out either by deselecting the audio device in the vm manager or deleting the previous code from the xml. i'd do that first before acs override. plus, that doesn't require rebooting. I DO want to pass through the audio. It is the rest of the devices in the group that I wonder if I need to pass through. By the way, I enabled ACS override and rebooted (stopped data transfer first) and it made no change to the IOMMU groups. All are as they were before enabling ACS override. Since I cannot seem to get the audio separated from the other items in the group, is it just a matter of editing the XML to pass the other devices as well adding <source> tags for the other devices? Will passing through the ISA bridge, memory controller and SMBus to a VM affect the core unRAID server negatively? i missread, been a long day... happy new year and all that jazz.... question: if you remove the audio device, does it boot properly? moving on.... try this in the syslinux.cfg instead of normal acs override: append pcie_acs_override=id:DEVICE:ID initrd=/bzroot this will try to pry the single device out of the group. you will add your device id to the part that says DEVICE:ID. to get the device id, look under Tools>System Devices, then search for 0000:00:1f.3 under pci devices. the number in brackets at the end of the description is the device id. Link to comment
Hoopster Posted January 2, 2017 Author Share Posted January 2, 2017 Yes, If I remove the audio device pass through from the VM config, the VM starts up without issue. I will try the device-specific acs override in syslinux.cfg. Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.