Inpired by these "x gamers one tover" youtube videos I got a used HP Z820 with dual Xeon E5-2695 v2 @ 2.40GHz and 32GB of RAM. To make it suitable for gaming I got a RTX 2060 super and some really cheap AMD passive cooled card for unraid.

Guided by SpaceInvderOne's videos I began my gaming VM adventure. First problem was awfull disk io when the vdisk was on the array, so I moved it to an unassigned m.2 drive. Now everything seems to be good on that front.

I'm still having very low fps while gaming and tons of stuttering. I can't make sense of it. GPU usage seems to be aroud 30-40% while gaming, CPU usage is between 20-50% on each core. So there is cleary some power left unutilised.

I've been testing mainly on CS:GO and  insurgency sandstorm. Former being having very much playable fps and the latter having a really hard time maintaining even stable 20fps. Both games have these huge fps drops or micro freezes which makes them unplayable. League of legends has similar issues to cs:go.

It really bothers me that the gamign expirence is much worse than on my now retired and sold i5-2500k gaming build. It was bare metal though. But something is clearly not right here since there's such a huge disconnect between gpu/cpu utilisation and performance.

 

I've been using MSI Afterburer and RTTS to monitor and log performance.

I have the GTX card on the top pci-ex16 slot on the motherboard and the AMD card on bottom most pci-ex16 slot.

Oh and I've tried both Q35 ad i440fx vms. Both have the same issues. I'm currently running i440fx.

Here's the info dump:

Syslinux configuration:

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label Unraid OS
  menu default
  kernel /bzimage
  append vfio-pci.ids=10de:1ada,10de:1adb tpcie_acs_override=downstream,multifunction isolcpus=1-8,25-32 nohz_full=1-8,25-32 rcu_nocbs=1-8,25-32 initrd=/bzroot
label Unraid OS GUI Mode
  kernel /bzimage
  append isolcpus=1-8,25-32 initrd=/bzroot,/bzroot-gui
label Unraid OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Unraid OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

VM XML:
 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='1'>
  <name>alestorm-i440fx</name>
  <uuid>531ca6d8-dd08-f38a-f344-386aca3e943e</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>13107200</memory>
  <currentMemory unit='KiB'>13107200</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>12</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='25'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='26'/>
    <vcpupin vcpu='4' cpuset='3'/>
    <vcpupin vcpu='5' cpuset='27'/>
    <vcpupin vcpu='6' cpuset='4'/>
    <vcpupin vcpu='7' cpuset='28'/>
    <vcpupin vcpu='8' cpuset='5'/>
    <vcpupin vcpu='9' cpuset='29'/>
    <vcpupin vcpu='10' cpuset='6'/>
    <vcpupin vcpu='11' cpuset='30'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-4.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/531ca6d8-dd08-f38a-f344-386aca3e943e_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='6' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/mnt/disks/KINGSTON_SA2000M81000G_50026B72824FAB26/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:12:90:6f'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-alestorm-i440fx/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/mnt/user/isos/MSI.RTX2060Super.8192.190702.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x2'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x05' slot='0x00' function='0x3'/>
      </source>
      <alias name='hostdev3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc332'/>
        <address bus='2' device='5'/>
      </source>
      <alias name='hostdev4'/>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x04d9'/>
        <product id='0xa0d1'/>
        <address bus='2' device='4'/>
      </source>
      <alias name='hostdev5'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x08bb'/>
        <product id='0x2902'/>
        <address bus='2' device='3'/>
      </source>
      <alias name='hostdev6'/>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1532'/>
        <product id='0x0207'/>
        <address bus='3' device='3'/>
      </source>
      <alias name='hostdev7'/>
      <address type='usb' bus='0' port='4'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1852'/>
        <product id='0x7022'/>
        <address bus='3' device='4'/>
      </source>
      <alias name='hostdev8'/>
      <address type='usb' bus='0' port='5'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

topology:


cpu pinning:

Hi, are you streaming your game  or using a monitor?

Monitor and USB passtrough mouse etc.

i do believe that what you are experiencing is the cpu bottleneck. 

The trouble with the v1 and v2 e5's is that while they were supporting the PCIE 3.0, they would not fully utilise the GPU for whatever reason. I use the E5 2867w v2 which are 3.4 GHZ per core, and while this does help (higher clock speed), the best results i had was with using a slower GPU (1050TI) there still is stuttering, but that i believe is caused by the multi cpu setup, as the ram is shared between CPUs, that is causing some aditional bottlenecks, whenever the data is exchanged by the ram assigned to diffrent CPUs.

The performance when you look at the framerate i assume is good, but the experience is not as good as the max framerate.

What i find is for example max framerate of 90-120 fps, with low framerate going down to 1.2 fps, and while the lows are in 1-2% range it may not seem like a lot, its still easy to spot.

Aparently its not as bad in newer CPUs, but you would get the same results running bare metal i bet.

 

So I'm pretty much out of luck with current hardware then?

https://ark.intel.com/content/www/us/en/ark/products/75281/intel-xeon-processor-e5-2695-v2-30m-cache-2-40-ghz.html

 

Says that the CPU has max. 40 pci-e lanes and I'm currently using 2 x16, 1 x4 and 1 x1 so there should be enough bandwith there (37/40) - at least in theory. Unless ethernet, usb etc. take some of the lanes.

And what comes to RAM, in theory all the 12GB of ram that has been assigned for the vm could be handeld by singe cpu. Dunno how unraid / linux kernel or what ever manages the ram though.

Wellp.. I guess it's time to learn how to convert a vm to bare metal machine then. At least my emby and handbrake have tons of power now ":D".

You may try to test your setup with Unigine Valley benchmark.  It is not CPU binded and you GPU load must be close to 100% there.

I have RTX 2060 Super as well on Ryzen 3600 and in this test in VM I'm getting around 100 fps average (streaming from headless VM) vs. 110 fps on bare metal Windows installation. If you have your monitor attached I believe the peformance hit would be even less.

 

Although I'm experiencing different issues with my setup, related to game streaming, because I"m running a headless VM setup.

Whatever streaming software I used (Steam/Moonlight/even RDP) my FPS immediately dropped by 50% and GPU utilisation was hardly 60% in certain benchmarks (3DMark) and games (Gears 5). However when I start investigation I've found the very same performance hit on my bare metal install as well.

 

I partially solved my problem by disabling Windows 10 aero effects and all game mode stuff.