Ok so this is strange. Somethimes both vm are working for 100 fps but sometimes the 1st vm having like 1 fps and the 2nd is always working. I tried also to change the xml with the specific rom for my cards but is not even start the vm(screen off). Here are the xmls for the machines(my cards are gigabyte g1 gtx980ti)

 

 

With vm1 started :

-------------------------------------------------------------------------------------------------------------------------------------

<domain type='kvm' id='11' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

  <name>gamer1</name>

  <uuid>0216f360-8cd3-cecc-b234-3e8bda3973e9</uuid>

  <metadata>

    <vmtemplate name="Custom" icon="windows.png" os="windows"/>

  </metadata>

  <memory unit='KiB'>6291456</memory>

  <currentMemory unit='KiB'>6291456</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>3</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='1'/>

    <vcpupin vcpu='1' cpuset='2'/>

    <vcpupin vcpu='2' cpuset='3'/>

  </cputune>

  <resource>

    <partition>/machine</partition>

  </resource>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>

  </os>

  <features>

    <acpi/>

    <apic/>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='3' threads='1'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='rtc' tickpolicy='catchup'/>

    <timer name='pit' tickpolicy='delay'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/bin/qemu-system-x86_64</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/vmmboot/gamer1/vdisk1.img'/>

      <backingStore/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <alias name='virtio-disk2'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/vmm/Windows_8_1_Pro_x64_En_Us.iso'/>

      <backingStore/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <alias name='ide0-0-0'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/vmm/virtio-win-0.1.118.iso'/>

      <backingStore/>

      <target dev='hdb' bus='ide'/>

      <readonly/>

      <alias name='ide0-0-1'/>

      <address type='drive' controller='0' bus='0' target='0' unit='1'/>

    </disk>

    <controller type='usb' index='0'>

      <alias name='usb'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'>

      <alias name='pci.0'/>

    </controller>

    <controller type='ide' index='0'>

      <alias name='ide'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <alias name='virtio-serial0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:3d:ca:af'/>

      <source bridge='br0'/>

      <target dev='vnet0'/>

      <model type='virtio'/>

      <alias name='net0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </interface>

    <serial type='pty'>

      <source path='/dev/pts/0'/>

      <target port='0'/>

      <alias name='serial0'/>

    </serial>

    <console type='pty' tty='/dev/pts/0'>

      <source path='/dev/pts/0'/>

      <target type='serial' port='0'/>

      <alias name='serial0'/>

    </console>

    <channel type='unix'>

      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/gamer1.org.qemu.guest_agent.0'/>

      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>

      <alias name='channel0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <hostdev mode='subsystem' type='usb' managed='yes'>

      <source>

        <vendor id='0x046d'/>

        <product id='0xc32b'/>

        <address bus='1' device='9'/>

      </source>

      <alias name='hostdev0'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='yes'>

      <source>

        <vendor id='0x046d'/>

        <product id='0xc07e'/>

        <address bus='1' device='6'/>

      </source>

      <alias name='hostdev1'/>

    </hostdev>

    <memballoon model='virtio'>

      <alias name='balloon0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </memballoon>

  </devices>

  <qemu:commandline>

    <qemu:arg value='-device'/>

    <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/>

    <qemu:arg value='-device'/>

    <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>

    <qemu:arg value='-device'/>

    <qemu:arg value='vfio-pci,host=01:00.1,bus=root.1,addr=00.1'/>

  </qemu:commandline>

</domain>

----------------------------------------------------------------------------------------------------------------------

 

  2nd vm  xml started :

--------------------------------------------------------------------------------------------------------------

<domain type='kvm' id='12' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

  <name>gamer2</name>

  <uuid>5922450e-16ba-fd0b-91cc-72126a17f3a7</uuid>

  <description>2nd</description>

  <metadata>

    <vmtemplate name="Custom" icon="windows.png" os="windows"/>

  </metadata>

  <memory unit='KiB'>6291456</memory>

  <currentMemory unit='KiB'>6291456</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>3</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='5'/>

    <vcpupin vcpu='1' cpuset='6'/>

    <vcpupin vcpu='2' cpuset='7'/>

  </cputune>

  <resource>

    <partition>/machine</partition>

  </resource>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>

  </os>

  <features>

    <acpi/>

    <apic/>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='3' threads='1'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='rtc' tickpolicy='catchup'/>

    <timer name='pit' tickpolicy='delay'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/bin/qemu-system-x86_64</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/vmmboot/gamer2/vdisk1.img'/>

      <backingStore/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <alias name='virtio-disk2'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/vmm/Windows.iso'/>

      <backingStore/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <alias name='ide0-0-0'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/vmm/virtio-win-0.1.118.iso'/>

      <backingStore/>

      <target dev='hdb' bus='ide'/>

      <readonly/>

      <alias name='ide0-0-1'/>

      <address type='drive' controller='0' bus='0' target='0' unit='1'/>

    </disk>

    <controller type='usb' index='0'>

      <alias name='usb'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'>

      <alias name='pci.0'/>

    </controller>

    <controller type='ide' index='0'>

      <alias name='ide'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <alias name='virtio-serial0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:de:7d:dc'/>

      <source bridge='br0'/>

      <target dev='vnet0'/>

      <model type='virtio'/>

      <alias name='net0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </interface>

    <serial type='pty'>

      <source path='/dev/pts/0'/>

      <target port='0'/>

      <alias name='serial0'/>

    </serial>

    <console type='pty' tty='/dev/pts/0'>

      <source path='/dev/pts/0'/>

      <target type='serial' port='0'/>

      <alias name='serial0'/>

    </console>

    <channel type='unix'>

      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/gamer2.org.qemu.guest_agent.0'/>

      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>

      <alias name='channel0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <hostdev mode='subsystem' type='usb' managed='yes'>

      <source>

        <vendor id='0x046d'/>

        <product id='0xc31b'/>

        <address bus='1' device='7'/>

      </source>

      <alias name='hostdev0'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='yes'>

      <source>

        <vendor id='0x046d'/>

        <product id='0xc521'/>

        <address bus='1' device='4'/>

      </source>

      <alias name='hostdev1'/>

    </hostdev>

    <memballoon model='virtio'>

      <alias name='balloon0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </memballoon>

  </devices>

  <qemu:commandline>

    <qemu:arg value='-device'/>

    <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/>

    <qemu:arg value='-device'/>

    <qemu:arg value='vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>

    <qemu:arg value='-device'/>

    <qemu:arg value='vfio-pci,host=02:00.1,bus=root.1,addr=00.1'/>

  </qemu:commandline>

</domain>

-------------------------------------------------------------------------------------------------------------------------------------

 

 

  Sorry but i dont know how to format the writing to apear the codes hightlighted......

 

  My cpu is an i7 6700k ...im using 3 cores for each vm...

 

Also a strange thing is the way unraid see the graphic cards  :

 

----------------------------------------------------------------------------------

00:00.0 Host bridge: Intel Corporation Sky Lake Host Bridge/DRAM Registers (rev 07)

00:01.0 PCI bridge: Intel Corporation Sky Lake PCIe Controller (x16) (rev 07)

00:01.1 PCI bridge: Intel Corporation Sky Lake PCIe Controller (x8) (rev 07)

00:02.0 VGA compatible controller: Intel Corporation Sky Lake Integrated Graphics (rev 06)

00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller (rev 31)

00:14.2 Signal processing controller: Intel Corporation Sunrise Point-H Thermal subsystem (rev 31)

00:16.0 Communication controller: Intel Corporation Sunrise Point-H CSME HECI #1 (rev 31)

00:17.0 SATA controller: Intel Corporation Device a102 (rev 31)

00:1b.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Root Port #17 (rev f1)

00:1b.3 PCI bridge: Intel Corporation Sunrise Point-H PCI Root Port #20 (rev f1)

00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #1 (rev f1)

00:1c.2 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #3 (rev f1)

00:1c.4 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #5 (rev f1)

00:1d.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #9 (rev f1)

00:1d.4 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #13 (rev f1)

00:1f.0 ISA bridge: Intel Corporation Sunrise Point-H LPC Controller (rev 31)

00:1f.2 Memory controller: Intel Corporation Sunrise Point-H PMC (rev 31)

00:1f.3 Audio device: Intel Corporation Sunrise Point-H HD Audio (rev 31)

00:1f.4 SMBus: Intel Corporation Sunrise Point-H SMBus (rev 31)

00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-V (rev 31)

01:00.0 VGA compatible controller: NVIDIA Corporation Device 17c8 (rev a1)

01:00.1 Audio device: NVIDIA Corporation Device 0fb0 (rev a1)

02:00.0 VGA compatible controller: NVIDIA Corporation Device 17c8 (rev a1)

02:00.1 Audio device: NVIDIA Corporation Device 0fb0 (rev a1)

04:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)

06:00.0 Ethernet controller: Qualcomm Atheros Device e0a1 (rev 10)

07:00.0 PCI bridge: Intel Corporation Device 1578

08:00.0 PCI bridge: Intel Corporation Device 1578

08:01.0 PCI bridge: Intel Corporation Device 1578

08:02.0 PCI bridge: Intel Corporation Device 1578

08:04.0 PCI bridge: Intel Corporation Device 1578

0b:00.0 USB controller: Intel Corporation Device 15b6

72:00.0 SATA controller: Samsung Electronics Co Ltd Device a801 (rev 01)

 

----------------------------------------------------------------------------------------------------

Im using the  PCIe ACS Override setting enabled because otherwise graphic cards are in the same group and also on my bios the first display is set up to igfx so unraid get that...

 

 

-------------------------------------------------------------------------------------------------------------------------------------

/sys/kernel/iommu_groups/0/devices/0000:00:00.0

/sys/kernel/iommu_groups/1/devices/0000:00:01.0

/sys/kernel/iommu_groups/2/devices/0000:00:01.1

/sys/kernel/iommu_groups/3/devices/0000:00:02.0

/sys/kernel/iommu_groups/4/devices/0000:00:14.0

/sys/kernel/iommu_groups/4/devices/0000:00:14.2

/sys/kernel/iommu_groups/5/devices/0000:00:16.0

/sys/kernel/iommu_groups/6/devices/0000:00:17.0

/sys/kernel/iommu_groups/7/devices/0000:00:1b.0

/sys/kernel/iommu_groups/8/devices/0000:00:1b.3

/sys/kernel/iommu_groups/9/devices/0000:00:1c.0

/sys/kernel/iommu_groups/10/devices/0000:00:1c.2

/sys/kernel/iommu_groups/11/devices/0000:00:1c.4

/sys/kernel/iommu_groups/12/devices/0000:00:1d.0

/sys/kernel/iommu_groups/13/devices/0000:00:1d.4

/sys/kernel/iommu_groups/14/devices/0000:00:1f.0

/sys/kernel/iommu_groups/14/devices/0000:00:1f.2

/sys/kernel/iommu_groups/14/devices/0000:00:1f.3

/sys/kernel/iommu_groups/14/devices/0000:00:1f.4

/sys/kernel/iommu_groups/14/devices/0000:00:1f.6

/sys/kernel/iommu_groups/15/devices/0000:01:00.0

/sys/kernel/iommu_groups/15/devices/0000:01:00.1

/sys/kernel/iommu_groups/16/devices/0000:02:00.0

/sys/kernel/iommu_groups/16/devices/0000:02:00.1

/sys/kernel/iommu_groups/17/devices/0000:04:00.0

/sys/kernel/iommu_groups/18/devices/0000:06:00.0

/sys/kernel/iommu_groups/19/devices/0000:07:00.0

/sys/kernel/iommu_groups/20/devices/0000:08:00.0

/sys/kernel/iommu_groups/21/devices/0000:08:01.0

/sys/kernel/iommu_groups/22/devices/0000:08:02.0

/sys/kernel/iommu_groups/23/devices/0000:08:04.0

/sys/kernel/iommu_groups/24/devices/0000:0b:00.0

/sys/kernel/iommu_groups/25/devices/0000:72:00.0

 

 

----------------------------------------------------------------------------------------------------------------------

 

 

  So please help me because i dont understand why somethimes is working both(for example last sunday me and my friend played around 6 h on the vm's) and aparently is working only vm2 ....on vm1 the frames is 1 even if he see the graphic card.

Please someone help....i had followed most of the things i have found on this forum..apart with the bios dump thing because i installed SSH but is not starting the plugin for some reason. I have tried both bios..changing cores and so on.

Ok..another strange thing. I have uninstall the driver on the vm with problems....restarted and then try to reinstall the driver. Surprise ....The nvidia installer is telling me that the hardware is incompatible with the driver. Come on ...give me some ideeas please

Ok i had the same chip as you for a time. And used to have 2 vms running however not 2 windows vms running at a time for gaming. One gaming and one osx.

 

looking at your cpu assignments it looks like you are not using cores paired with their hyperthread.

 

If i remember the pairs for a quad core 6700 are like this (you can see from tools, system devices, CPU Thread Pairings)

 

0  <===>  4    ( this is one core the 0 and 4)

1  <===>  5

2  <===>  6

3  <===>  7

 

so looking at your xml  (to hight code click icon in toolbar that looks like  #       

 

 <vcpu placement='static'>3</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='2'/>
    <vcpupin vcpu='2' cpuset='3'/>
  </cputune>

 

So your 3 virtual cores on vm1,  are being taken from 1 hyperthread from 3 different cores.

 

<vcpu placement='static'>3</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='5'/>
    <vcpupin vcpu='1' cpuset='6'/>
    <vcpupin vcpu='2' cpuset='7'/>
  </cputune>

 

your second vm is using the other hyper threads from the same cores, so your hyperthreads are being split between  2 machines which will give you poor performance.

 

You need to assign threaded pairs  so for vm 1 it should be

 

 

<vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='4'/>
    <vcpupin vcpu='2' cpuset='1'/>
<vcpupin vcpu='2' cpuset='5'/>
  </cputune>

 

Always pair the hyper threads together.

 

never split a physical core, always use the whole core both threads.

 

Try this and report back.

 

Ok i had the same chip as you for a time. And used to have 2 vms running however not 2 windows vms running at a time for gaming. One gaming and one osx.

 

looking at your cpu assignments it looks like you are not using cores paired with their hyperthread.

 

If i remember the pairs for a quad core 6700 are like this (you can see from tools, system devices, CPU Thread Pairings)

 

0  <===>  4    ( this is one core the 0 and 4)

1  <===>  5

2  <===>  6

3  <===>  7

 

so looking at your xml  (to hight code click icon in toolbar that looks like  #       

 

 <vcpu placement='static'>3</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='2'/>
    <vcpupin vcpu='2' cpuset='3'/>
  </cputune>

 

So your 3 virtual cores on vm1,  are being taken from 1 hyperthread from 3 different cores.

 

<vcpu placement='static'>3</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='5'/>
    <vcpupin vcpu='1' cpuset='6'/>
    <vcpupin vcpu='2' cpuset='7'/>
  </cputune>

 

your second vm is using the other hyper threads from the same cores, so your hyperthreads are being split between  2 machines which will give you poor performance.

 

You need to assign threaded pairs  so for vm 1 it should be

 

 

<vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='4'/>
    <vcpupin vcpu='2' cpuset='1'/>
<vcpupin vcpu='2' cpuset='5'/>
  </cputune>

 

Always pair the hyper threads together.

 

never split a physical core, always use the whole core both threads.

 

Try this and report back.

 

Thanks for replayng..but i already try what u said...using core 0,4,1,5 for vm1 and other for vm2......and still same thing

If you are running both vms at once as you only have 4 cores, make sure you close all dockers before running the vms. To give all you can to the vms.

 

Then you may want to try emulatorpin or even isolating cores from unraid (but i would try emulatorpin first)

 

Have a read through this post and try some of the ideas discussed here  http://lime-technology.com/forum/index.php?topic=49051.0

 

 

also what version of unraid are you running if not 6.2 maybe try that as there are various improvments to the vms in 6.2

6.1.9

I would upgrade to 6.2 rc2. Then you can use hyperv with nvidea cards aswell.

Read through the steps necessary to upgrade to 6.2 for the vms http://lime-technology.com/forum/index.php?topic=47408.0

I would upgrade to 6.2 rc2. Then you can use hyperv with nvidea cards aswell.

Read through the steps necessary to upgrade to 6.2 for the vms http://lime-technology.com/forum/index.php?topic=47408.0

 

  Ok gridrunner...i have did a fresh new install of unraid...shares etc....

 

It is the same. I have tried multiple environments like:

- running only one vm/both

- one vm is on seabios the other in omvf kvm off/on

- i have alocated cores like 0-4,1-5 and 2-6,3-7 but i didint do that unraid core thing because i cant find it lol(but somehow i dont think this is the issue)

 

Observations :

- no matter how i do the 1st card in pci-e is the problem...If i allocate the other card to the vm with the problem then it works flawless;

- The problem starts when i install the nvidia drivers , from the moment the screen blinks and basically the nvidia driver is installed;

- The card is acting normally ..i mean is a g1 gtx980ti and after the driver is installed the windforce logo is shutting down showing me that the fans stop as normally after the driver is installed.

 

Here are the xml's where the  card 01:00.0 is the problem:

 

<domain type='kvm' id='1'>
  <name>HND1</name>
  <uuid>179761bf-cd98-d658-901f-7282bcca0e7a</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>6291456</memory>
  <currentMemory unit='KiB'>6291456</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
    <vcpupin vcpu='2' cpuset='6'/>
    <vcpupin vcpu='3' cpuset='7'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='2' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/vmdisk/HND1/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOS/en_windows_10_multiple_editions_x64_dvd_6846432.iso'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOS/virtio-win-0.1.118-2.iso'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='nec-xhci'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:3e:9b:c9'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-HND1/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <hostdev mode='subsystem' type='pci' managed='yes' xvga='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc07e'/>
        <address bus='1' device='6'/>
      </source>
      <alias name='hostdev2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc32b'/>
        <address bus='1' device='9'/>
      </source>
      <alias name='hostdev3'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </memballoon>
  </devices>
</domain>

 

  and

<domain type='kvm' id='4'>
  <name>HND2</name>
  <uuid>e0a366ef-8860-6d9d-4d19-a89950386a40</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>6291456</memory>
  <currentMemory unit='KiB'>6291456</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='1'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='5'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/e0a366ef-8860-6d9d-4d19-a89950386a40_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='2' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/vmdisk/HND2/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOS/en_windows_10_multiple_editions_x64_dvd_6846432.iso'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOS/virtio-win-0.1.118-2.iso'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='nec-xhci'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:5c:46:a3'/>
      <source bridge='br0'/>
      <target dev='vnet1'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-HND2/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc31b'/>
        <address bus='1' device='7'/>
      </source>
      <alias name='hostdev2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc521'/>
        <address bus='1' device='4'/>
      </source>
      <alias name='hostdev3'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </memballoon>
  </devices>
</domain>

 

 

If with only one vm running, the problem is always related to the card in the first pcie slot. Then try swapping the two cards. So put the second card in slot one etc.  You can then see the problem follows the card and is hardware related somehow.

 

I would also try passing through the rom file for your gpus see how here https://lime-technology.com/forum/index.php?topic=43644.msg464975#msg464975

 

Other than that I am out of ideas im afraid!  Hopefully someone else with more knowledge will see this and chime in.

 

Best of luck.

 

 

I have tried already to swap the cards...same thing...is related to 1st pci express slot somehow. Being a z170x chipset the mobo is coming with the integrated graphics and as i told you before the bios is set up for IGFX so that unraid grab that and it is like that.

  The bios thing i had tried before on seabios but i didint have any output on the monitor unfortunately. I will try even on this one again but i need to research where are the lines in the xml about the graphics.

Ok..strange  think....I reinstall back 6.1.9 because i have got system crash with the 6.2 and also full network down...lol i was need to unplug the unraid machine from the network in order the router works again...anyway this is not about this thread is for

 

  So..new system 6.1.9 array etc. Installed both vm's on OMVF i440fx-2.3 kvm off. Assigned the cpu like 2,6,3,7 for first vm and 0,4,1,5 for 2nd vm. Everything is working fine as you will see in the pictures bellow apart for a thing wich i was see it even the last time when it was working. When i install the gpu driver on the 2nd vm i also get an app from nvidia wich is show me the gpu activity. Im not selecting anything in plus on nvidia driver installation , absolutely same installation of the same driver as vm1.

 

Now i wait  the sync of array finish and then i will restart the array and the vm's to see if it is ok...or is like the last time..when after restarting the array the vm1 having 1 fps.