Hi,

 

I've just brought two Asus GTX 1070 Founders edition GPUs (in order to create two VMs) and want to replace my old AMD GPU which has been working great until now.

I've installed one of the Nvidia pci cards, but when I try to turn the VM back on, the screen remain black.

 

I've also tried to remove and re-create the VM without any luck. I'm able to turn the VM on in the GUI, but nothing happens on the monitor.

I can't find any errors in UnRaid, so hope anyone here has some suggestions.

 

Version: unRAIDServer-6.2.0-rc3

Enable PCIe ACS Override: Yes

 

My XML:

<domain type='kvm' id='1'>
  <name>Kodi</name>
  <uuid>a908771e-08a4-5e6a-6e28-5644bce7c59d</uuid>
  <description>Kodi and gaming vm connected to TV</description>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>13107200</memory>
  <currentMemory unit='KiB'>13107200</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='2'/>
    <vcpupin vcpu='2' cpuset='5'/>
    <vcpupin vcpu='3' cpuset='6'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/a908771e-08a4-5e6a-6e28-5644bce7c59d_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='2' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Kodi/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Kodi/vdisk2.img'/>
      <backingStore/>
      <target dev='hdd' bus='virtio'/>
      <alias name='virtio-disk3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/os/en_windows_10_multiple_editions_x64_dvd_6846432.iso'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.118-2.iso'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:c5:72:ca'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-Kodi/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc52b'/>
        <address bus='3' device='2'/>
      </source>
      <alias name='hostdev2'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </memballoon>
  </devices>
</domain>

 

Edit: Not sure if this means anything:

Jul 28 14:19:01 Samurai kernel: vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+mem

Jul 28 14:19:01 Samurai kernel: device vnet0 entered promiscuous mode

Jul 28 14:19:01 Samurai kernel: br0: port 2(vnet0) entered forwarding state

Jul 28 14:19:01 Samurai kernel: br0: port 2(vnet0) entered forwarding state

Jul 28 14:19:02 Samurai kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

Jul 28 14:19:03 Samurai kernel: vfio-pci 0000:01:00.0: Invalid ROM contents

Jul 28 14:19:03 Samurai kernel: vfio-pci 0000:01:00.0: Invalid ROM contents

Jul 28 14:19:06 Samurai kernel: usb 3-10: reset full-speed USB device number 2 using xhci_hcd

Jul 28 14:19:07 Samurai kernel: usb 3-10: reset full-speed USB device number 2 using xhci_hcd

Jul 28 14:19:07 Samurai kernel: usb 3-10: reset full-speed USB device number 2 using xhci_hcd

Jul 28 14:19:09 Samurai kernel: kvm: zapping shadow pages for mmio generation wraparound

I think I may have figured it out...

 

For some reason I had to go to the BIOS and change the GPU setting from auto to CPU GPU. Even tough it worked with my AMD GPU, it didn't work with Nvidia...

I think I may have figured it out...

 

For some reason I had to go to the BIOS and change the GPU setting from auto to CPU GPU. Even tough it worked with my AMD GPU, it didn't work with Nvidia...

That is a know issue with nvidia adapters for those of us lurking around here for some time 

I've been playing around with this setup for hours and struggled to get both VMs to work.

While one of the machines work (using CPU core 3 and 7 only), the other VM using core 1,2,5 and 6 did not. While I managed to install Windows 10 on it, it stops right after the setup is complete, only displaying the Windows flag on the screen (boot screen). Once I change it to only use core 5 and 6 it boots... is there any logic that explains this behavior?

 

I've also moved my GPU to other PCI slots and disabled ACS Override since it caused read errors on my SSD cache for some reasons...

I thought I shouldn't use Core 0, since UnRaid favoured it, but is it really core 1 I should worry? Is there another explanation and can I utilize my cores in a better way or just stick using two cores per VM?

 

My system has the following cores:

CPU 0 CPU 1 CPU 2 CPU 3

CPU 4 CPU 5 CPU 6 CPU 7