Hi,

 

Please see the below for background:

 

Hello All,

 

So before I started to build my current server, which is a dual Xeon E5-2630 V3 machine running on a ASUS Z10pe-d8 WS with 128 GB ECC DDR4 RAM and a single 980ti and the onboard Asmedia sata controller with two optical drives attached passed through to a Windows 10 Pro VM using the UEFI VM BIOS and 4 physical cores/8 threads/16 GB RAM, I tested World of Tanks and Planetside 2 on a Xeon E3-1245 V3 machine with 32 GB of DDR3 ECC RAM and the same 980ti passed through with no issues, two cores/four threads/8GB RAM, had consistent FPS. Same settings, same UNRAID 6.2 config. All disks are running off on LSI 9211-8i IT mode HBAs. But when I tried it on the new server, I notice severe lag out of nowhere in World of Tanks and Random Intervals, FPS drops to 45-35 randomly and the machine seems to lag. 3D Mark scores around 11000-12000 randomly, though the physics test only reaches about 24 fps, which is comparable to an i7-870 I tested at work. Looking at similar configs averaging about 35-40 FPSbaremetal, it boggles my mind how the results can be this low even with the emulation overhead. (makes no sense) CPU usage is at/below 30 percent in game and disk usage is low as well. Network Ping and USB input lag do not seem to be the issues, and I'm 100 percent certain it is not my hardware or CPU bottleneck. I am at my wits end having fought with this thing for two weeks and have lost a good bit of sleep in the process. I'm about ready to pay for Limetech support to try and solve this frustrating issue that is giving me figurative grey hairs, but am posting here as a last resort. Any help/advice would be much appreciated.

 

-adilworth2016

 

Now I have pinned and isolated CPU cores as suggested, and am having extreme performance issues in Planetside 2 and 3D Mark Time Spy as well as other titles, averaging about 20-30 FPS per VM. Any ideas? Thanks in advance.

 

-adilworth2016

Your post has me a bit confused as it is titled 2 Gamers, but you only have 1 single 980ti.  Please explain if this is an issue you are experiencing when two VMs are running or not.

Sorry for the confusion, I actually have two VM's running, each with 4 cores, 8 threads, 16 GB of RAM, and a GTX 980 Ti. Their are two of those video cards. The issue is when both VM's are running, though I have yet to test it with just one VM running.

***bump***

Still having the same issue even with just one VM running. Any thoughts?

Did you allocate the cores correctly? 0-3 + 8-11, 4-7 + 12-15. And don't forget to adjust the CPU topology:

 

<topology sockets='1' cores='4' threads='2'/>

I put this in the VM config file? I'll give it a try and report back. Thanks!

No Good, FPS is still abhorrent...

No Good, FPS is still abhorrent...

 

Post your xml so we can take a look. Also, run a cpu passmark and post the result here, do this on both bare metal and UNRAID so we can compare. I had similar issues with a E5-2660V3 that were resolved by pinning cpu cores in pairs, enabling hyper-v and isolcpus in pairs.

Here is the XML for Windows VM One:

 

<domain type='kvm'>

  <name>Windows PC One</name>

  <uuid>7f9ff4c9-a1a9-e13f-5a97-de2e97e1a50a</uuid>

  <metadata>

    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>

  </metadata>

  <memory unit='KiB'>16777216</memory>

  <currentMemory unit='KiB'>16777216</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>8</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='8'/>

    <vcpupin vcpu='1' cpuset='9'/>

    <vcpupin vcpu='2' cpuset='10'/>

    <vcpupin vcpu='3' cpuset='11'/>

    <vcpupin vcpu='4' cpuset='24'/>

    <vcpupin vcpu='5' cpuset='25'/>

    <vcpupin vcpu='6' cpuset='26'/>

    <vcpupin vcpu='7' cpuset='27'/>

    <emulatorpin cpuset='0-7'/>

  </cputune>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>

    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>

    <nvram>/etc/libvirt/qemu/nvram/7f9ff4c9-a1a9-e13f-5a97-de2e97e1a50a_VARS-pure-efi.fd</nvram>

  </os>

  <features>

    <acpi/>

    <apic/>

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

      <vendor id='none'/>

    </hyperv>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='4' threads='2'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/local/sbin/qemu</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/domains/Windows PC One/vdisk1.img'/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/Array vdisks/Windows PC 1/vdisk2.img'/>

      <target dev='hdd' bus='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/ISOS/Windows.iso'/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/ISOS/virtio-win-0.1.118-2.iso'/>

      <target dev='hdb' bus='ide'/>

      <readonly/>

      <address type='drive' controller='0' bus='0' target='0' unit='1'/>

    </disk>

    <controller type='usb' index='0' model='ich9-ehci1'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci1'>

      <master startport='0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci2'>

      <master startport='2'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci3'>

      <master startport='4'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'/>

    <controller type='ide' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:aa:43:c4'/>

      <source bridge='br0'/>

      <model type='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <channel type='unix'>

      <source mode='connect'/>

      <target type='virtio' name='org.qemu.guest_agent.0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x0461'/>

        <product id='0x0010'/>

      </source>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x046d'/>

        <product id='0xc24a'/>

      </source>

    </hostdev>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>

    </memballoon>

  </devices>

</domain>

 

And here is the XML for Windows VM Two

 

<domain type='kvm'>

  <name>Windows PC Two</name>

  <uuid>117fca6c-ffe2-5466-33dd-82c128a92808</uuid>

  <metadata>

    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>

  </metadata>

  <memory unit='KiB'>16777216</memory>

  <currentMemory unit='KiB'>16777216</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>8</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='12'/>

    <vcpupin vcpu='1' cpuset='13'/>

    <vcpupin vcpu='2' cpuset='14'/>

    <vcpupin vcpu='3' cpuset='15'/>

    <vcpupin vcpu='4' cpuset='28'/>

    <vcpupin vcpu='5' cpuset='29'/>

    <vcpupin vcpu='6' cpuset='30'/>

    <vcpupin vcpu='7' cpuset='31'/>

    <emulatorpin cpuset='0-7'/>

  </cputune>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>

    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>

    <nvram>/etc/libvirt/qemu/nvram/117fca6c-ffe2-5466-33dd-82c128a92808_VARS-pure-efi.fd</nvram>

  </os>

  <features>

    <acpi/>

    <apic/>

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

      <vendor id='none'/>

    </hyperv>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='4' threads='2'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/local/sbin/qemu</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/domains/Windows PC Two/vdisk1.img'/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/Array vdisks/Windows PC 2/vdisk2.img'/>

      <target dev='hdd' bus='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/ISOS/Windows.iso'/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/ISOS/virtio-win-0.1.118-2.iso'/>

      <target dev='hdb' bus='ide'/>

      <readonly/>

      <address type='drive' controller='0' bus='0' target='0' unit='1'/>

    </disk>

    <controller type='usb' index='0' model='ich9-ehci1'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci1'>

      <master startport='0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci2'>

      <master startport='2'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci3'>

      <master startport='4'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'/>

    <controller type='ide' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:ef:45:9b'/>

      <source bridge='br0'/>

      <model type='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <channel type='unix'>

      <source mode='connect'/>

      <target type='virtio' name='org.qemu.guest_agent.0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x1532'/>

        <product id='0x0043'/>

      </source>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x1532'/>

        <product id='0x011a'/>

      </source>

    </hostdev>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>

    </memballoon>

  </devices>

</domain>

 

And here is my syslinux config:

 

default /syslinux/menu.c32

menu title Lime Technology, Inc.

prompt 0

timeout 50

label unRAID OS

  menu default

  kernel /bzimage

  append pci-stub.ids=1b21:0612 isolcpus=8,24,9,25,10,26,11,27,12,28,13,29,14,30,15,31 initrd=/bzroot

label unRAID OS GUI Mode

  kernel /bzimage

  append initrd=/bzroot,/bzroot-gui

label unRAID OS Safe Mode (no plugins, no GUI)

  kernel /bzimage

  append initrd=/bzroot unraidsafemode

label Memtest86+

  kernel /memtest

 

I'll post the passmark results when I can.

 

CPU mark VM results

 

VM 1: 6301

 

VM 2: 5817

 

Does it make a difference that this is a dual socket/CPU system? Maybe the HT pairs Unraid is reporting is incorrect?

I made a mistake.

 

Without knowing the output of:

 

numactl -H

 

...You cannot know how you should arrange your pinned cores. Please post that here, if you need help.

 

Hyper threaded CPUs will still show up as a single node, with the virtual core set listed after the main core set. Your problem likely stems from mixing cores from one CPU and the other within the same VMs.

 

You will need to pick an even number of main and their matching virtual cores from the first NUMA node for your One virtual machine, and the same count and such from node two for your Two virtual machine. Unless, for some reason, you wish to run them both on the same CPU.

I started an SSH session to the server and ran the command you specified. Here is the output.

 

login as: root

[email protected]'s password:

Linux 4.4.23-unRAID.

root@NASferatu:~# numactl -H

available: 2 nodes (0-1)

node 0 cpus: 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23

node 0 size: 32129 MB

node 0 free: 343 MB

node 1 cpus: 8 9 10 11 12 13 14 15 24 25 26 27 28 29 30 31

node 1 size: 32253 MB

node 1 free: 154 MB

node distances:

node  0  1

  0:  10  21

  1:  21  10

 

 

Also what should I do about isolating CPU's?

Still seem to be having issues...Maybe I'm not doing something correctly?

Would it help if I pin the cores that Unraid isn't using but are still isolated for the emulator?

bump

Have found that the interface and nics I was using to connect to my pfsense router do not support the traffic shaping rules I put in place...disabled those just in case ping is related...Also changed some power management settings in the BIOS to lean toward performance vs energy efficiency. Will try it out and post updated xmls, syslinux cfg files, and numa command outputs.

Now I'm getting the sense that this has nothing to do with my network or BIOs config..It's something with UNRAID for sure.

World of Tanks (not very CPU heavy) performs just fine, but in Planetside 2 (CPU heavy) the moment their are lag spikes and FPS drops at random...

Updated syslinux.cfg

 

default /syslinux/menu.c32

menu title Lime Technology, Inc.

prompt 0

timeout 50

label unRAID OS

  menu default

  kernel /bzimage

  append pci-stub.ids=1b21:0612 isolcpus=4,20,5,21,6,22,7,23,12,28,13,29,14,30,15,31 initrd=/bzroot

label unRAID OS GUI Mode

  kernel /bzimage

  append initrd=/bzroot,/bzroot-gui

label unRAID OS Safe Mode (no plugins, no GUI)

  kernel /bzimage

  append initrd=/bzroot unraidsafemode

label Memtest86+

  kernel /memtest

 

Updated Windows VM One XML:

 

<domain type='kvm'>

  <name>Windows PC One</name>

  <uuid>8b802c83-b51b-a1ee-29fd-c5b0619d29d3</uuid>

  <metadata>

    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>

  </metadata>

  <memory unit='KiB'>16777216</memory>

  <currentMemory unit='KiB'>16777216</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>8</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='4'/>

    <vcpupin vcpu='1' cpuset='5'/>

    <vcpupin vcpu='2' cpuset='6'/>

    <vcpupin vcpu='3' cpuset='7'/>

    <vcpupin vcpu='4' cpuset='20'/>

    <vcpupin vcpu='5' cpuset='21'/>

    <vcpupin vcpu='6' cpuset='22'/>

    <vcpupin vcpu='7' cpuset='23'/>

    <emulatorpin cpuset='0-3,16-19'/>

  </cputune>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>

    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>

    <nvram>/etc/libvirt/qemu/nvram/8b802c83-b51b-a1ee-29fd-c5b0619d29d3_VARS-pure-efi.fd</nvram>

  </os>

  <features>

    <acpi/>

    <apic/>

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

      <vendor id='none'/>

    </hyperv>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='4' threads='2'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/local/sbin/qemu</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/domains/Windows PC One/vdisk1.img'/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/Array vdisks/Windows PC 1/vdisk2.img'/>

      <target dev='hdd' bus='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/ISOS/Windows.iso'/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/ISOS/virtio-win-0.1.118-2.iso'/>

      <target dev='hdb' bus='ide'/>

      <readonly/>

      <address type='drive' controller='0' bus='0' target='0' unit='1'/>

    </disk>

    <controller type='usb' index='0' model='ich9-ehci1'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci1'>

      <master startport='0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci2'>

      <master startport='2'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci3'>

      <master startport='4'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'/>

    <controller type='ide' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:3a:f9:24'/>

      <source bridge='br0'/>

      <model type='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <channel type='unix'>

      <source mode='connect'/>

      <target type='virtio' name='org.qemu.guest_agent.0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x0461'/>

        <product id='0x0010'/>

      </source>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x046d'/>

        <product id='0xc24a'/>

      </source>

    </hostdev>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>

    </memballoon>

  </devices>

</domain>

 

Updated Windows VM Two XML:

 

<domain type='kvm'>

  <name>Windows PC Two</name>

  <uuid>5068722f-5763-5d25-eb59-22ea76674ccb</uuid>

  <metadata>

    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>

  </metadata>

  <memory unit='KiB'>16777216</memory>

  <currentMemory unit='KiB'>16777216</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>8</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='12'/>

    <vcpupin vcpu='1' cpuset='13'/>

    <vcpupin vcpu='2' cpuset='14'/>

    <vcpupin vcpu='3' cpuset='15'/>

    <vcpupin vcpu='4' cpuset='28'/>

    <vcpupin vcpu='5' cpuset='29'/>

    <vcpupin vcpu='6' cpuset='30'/>

    <vcpupin vcpu='7' cpuset='31'/>

    <emulatorpin cpuset='8-11,24-27'/>

  </cputune>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type>

    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>

    <nvram>/etc/libvirt/qemu/nvram/5068722f-5763-5d25-eb59-22ea76674ccb_VARS-pure-efi.fd</nvram>

  </os>

  <features>

    <acpi/>

    <apic/>

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

      <vendor id='none'/>

    </hyperv>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='4' threads='2'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/local/sbin/qemu</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/domains/Windows PC Two/vdisk1.img'/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/Array vdisks/Windows PC 2/vdisk2.img'/>

      <target dev='hdd' bus='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/ISOS/Windows.iso'/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/ISOS/virtio-win-0.1.118-2.iso'/>

      <target dev='hdb' bus='ide'/>

      <readonly/>

      <address type='drive' controller='0' bus='0' target='0' unit='1'/>

    </disk>

    <controller type='usb' index='0' model='ich9-ehci1'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci1'>

      <master startport='0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci2'>

      <master startport='2'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>

    </controller>

    <controller type='usb' index='0' model='ich9-uhci3'>

      <master startport='4'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'/>

    <controller type='ide' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:08:55:59'/>

      <source bridge='br0'/>

      <model type='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <channel type='unix'>

      <source mode='connect'/>

      <target type='virtio' name='org.qemu.guest_agent.0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x81' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x81' slot='0x00' function='0x1'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x1532'/>

        <product id='0x0043'/>

      </source>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='no'>

      <source>

        <vendor id='0x1532'/>

        <product id='0x011a'/>

      </source>

    </hostdev>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>

    </memballoon>

  </devices>

</domain>

 

 

 

 

Just quickly skimming through your post, i see you are

 

<emulatorpin cpuset='0-7'/>    why so many cores for emulatorpin. just use one core for that.

 

heres what i would do.

 

1. isolcpu 4 cores from each cpu you have, looking at your cpu pairs in tools, system devices. (but makes sure not to pin core 1 as unRAID prefers that core) then pin 4 isolated cores from cpu 1 to vm 1 and other 4 isolated cpus from core 2 to vm2.

 

2. dont isolate the emulatorpin cores in isolcpu

 

3. use only one core (non isolated and not core 1) from cpu 1  and pin it in xml to vm 1 for emulatorpin of that vm

 

4. 3. use only one core (non isolated ) from cpu 2 and pin it in xml to vm 2 for emulatorpin of that vm

 

Dont use any dockers etc whilst you test the vms.

 

 

 

 

 

Ok will give it a shot when I get home from work and report back. Thanks!

Didn't work. Will post updated XML's and Syslinux.cfg later.

What i think you need to do is just try and get one vm working well first before trying 2.

Try taking out the second gpu and just using one gpu and go from there.

 

Maybe try some other bios revision in the motherboard. See if there is a newer one than what you have. But maybe also try and use an older one as well (but check you dont go back too far so your chip isnt supported but as a v3 you should be okay) I had a motherboard where the newest bioses the vtd didnt work correctly but a few revisions back it did?

 

Also check all your bios settings are optimal.

 

Have you tried using a seabios vm as opposed to ovmf?