Hi Guys,

 

I have setup a Windows 10 Pro VM and have attempted to pass-through one of my two GTX 1080's.

 

The second GPU is in the second PCI-Express x 16 slot running at full speed.

The motherboard is a Asus X99-WS-E, it has a PLX chip so there are 4 x PCI-E slots that are capable of running at the full x16 speed.

 

The GPU is actually successfully passed to the VM, the VM boots to the desktop and I have been able to install the latest Nvidia Graphics Experience util and the latest drivers. (I have tried several versions).

 

The issue is the graphics seems to be corrupted specifically:

 

• white horizontal bars that do not redraw
  Start menu that does not draw the various menu items
  Explorer menu buttons and context menus that are not drawn.
   
  Things I have tried:
   
  Various versions of the Nvidia drivers
  Changing the Nvidia Drivers via the Nvidia Control Panel to Compatibility Mode.
  Turning off Hyper-V Support
  Enable PCIe ACS Override (The GPU's are not sharing any ID's with this enable or disabled)
  Upgrade from UNRAID 6.2.4 to 6.3.1.
  Upgraded VM BIOS from i440fx-2.5 to i440fx-2.7.
  Upgraded virtio drivers.
   
  I have read that some users have dumped their GPU BIOS and mapped this in their XML file, I was under the impression this isn't required for me as I am mapping my second GTX 1080 in the second PCI-E slot.
   
  I am pretty much the point where I am thinking the GTX 1080's are not compatible and my UNRAID experiment is not viable.
   
  Any help you can provide would be greatly appreciated; as UNRAID was the entire point of building this machine.
   
  Update:
   
  I am seeing this the Libvirt.log file:
  2017-02-14 09:35:04.868+0000: 9824: warning : qemuDomainObjTaint:3885 : Domain id=4 name='Windows 10' uuid=f5201df7-37bc-17c4-4ae4-4c9d0defa712 is tainted: high-privileges
  2017-02-14 09:35:04.868+0000: 9824: warning : qemuDomainObjTaint:3885 : Domain id=4 name='Windows 10' uuid=f5201df7-37bc-17c4-4ae4-4c9d0defa712 is tainted: host-cpu
  2017-02-14 09:35:10.873+0000: 9824: error : x86FeatureInData:780 : internal error: unknown CPU feature __kvm_hv_spinlocks
  2017-02-14 09:35:10.873+0000: 9824: error : x86FeatureInData:780 : internal error: unknown CPU feature __kvm_hv_vendor_id
   
  Thanks,
   
  Enver
   
   
  

Enver,

 

I can confirm for you that I have a GTX 1080 in one of our test systems working just fine.  Can you check your motherboard BIOS to see if there is an update?  Also, does this issue occur with the secondary GTX 1080 you have?

Hello,

 

Thanks for the reply.

 

Well its good to know it does work, now to identify why its not working with my setup.

 

By secondary GTX 1080 if you mean the one in the second PCI-E slot; this is the one I am using. I have not tried using the GTX 1080 in the primary slot as it is my understanding I would have to dump the BIOS and create an custom XML file.

 

Motherboard has the very latest BIOS. HVM and IOMMU is also enabled.

 

Just as a sanity check I reread the Wiki page on passing through the GPU and dumped the BIOS of the secondary GTX 1080, added it to the XML file and also added the lines to hide KVM from the Nvidia drivers. It appears as if the VM video deteriorates soon after boot up and eventually crashes.....graphics performance is woeful.

 

Here is my XML file which unfortunately has done nothing to fix the problem:

 

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

  <name>Windows 10</name>

  <uuid>f5201df7-37bc-17c4-4ae4-4c9d0defa712</uuid>

  <metadata>

    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>

  </metadata>

  <memory unit='KiB'>8388608</memory>

  <currentMemory unit='KiB'>8388608</currentMemory>

  <memoryBacking>

    <nosharepages/>

  </memoryBacking>

  <vcpu placement='static'>4</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='12'/>

    <vcpupin vcpu='1' cpuset='13'/>

    <vcpupin vcpu='2' cpuset='14'/>

    <vcpupin vcpu='3' cpuset='15'/>

  </cputune>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type>

    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>

    <nvram>/etc/libvirt/qemu/nvram/f5201df7-37bc-17c4-4ae4-4c9d0defa712_VARS-pure-efi.fd</nvram>

  </os>

  <features>

    <acpi/>

    <apic/>

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

      <vendor_id state='on' value='none'/>

    </hyperv>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='2' threads='2'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/local/sbin/qemu</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/domains/Windows 10/vdisk1.img'/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='block' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source dev='/dev/disk/by-id/scsi-2001b4d2014598729'/>

      <target dev='hdd' bus='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/isos/Win10_1607_EnglishInternational_x64.iso'/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/isos/virtio-win-0.1.126-2.iso'/>

      <target dev='hdb' bus='ide'/>

      <readonly/>

      <address type='drive' controller='0' bus='0' target='0' unit='1'/>

    </disk>

    <controller type='usb' index='0' model='nec-xhci'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'/>

    <controller type='ide' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:a9:2f:55'/>

      <source bridge='br0'/>

      <model type='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <channel type='unix'>

      <target type='virtio' name='org.qemu.guest_agent.0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <input type='mouse' bus='ps2'/>

    <input type='keyboard' bus='ps2'/>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>

      </source>

      <rom file='/mnt/user/domains/Windows 10/evgagtx1080ftw.dump'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x1038'/>

        <product id='0x137e'/>

      </source>

      <address type='usb' bus='0' port='1'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x1b1c'/>

        <product id='0x1b13'/>

      </source>

      <address type='usb' bus='0' port='2'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x27ac'/>

        <product id='0x1002'/>

      </source>

      <address type='usb' bus='0' port='3'/>

    </hostdev>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>

    </memballoon>

  </devices>

  <qemu:commandline>

    <qemu:arg value='-cpu'/>

    <qemu:arg value='host,kvm=off'/>

  </qemu:commandline>

</domain>

Enver,

 

I can confirm for you that I have a GTX 1080 in one of our test systems working just fine.  Can you check your motherboard BIOS to see if there is an update?  Also, does this issue occur with the secondary GTX 1080 you have?

 

Could you tell me what version of UNRAID and QEMU machine type you are using? Are you using a custom XML file and if so what custom attributes have you passed?

Update:

 

I switched to the Q35-2.7 Machine Type and this resolved my issue.

 

I could not get the graphics card to run stable(lots of graphic artefacts) on the following i440fx Machine Types, 2.5, 2.6 and 2.7.

 

I am unsure of the functional differences between the two machine types of if one is more developed/feature packed than the other, so if someone can provide some insight I would appreciate it.

 

Thanks,

 

Enver