Help! Low FPS on vm unraid gaming server using Threadripper 3960x and Nvidia GeForce Rtx 3080


bidwars

Recommended Posts

Hi I am new to Unraid and trying to figure out what might be causing my poor gaming experience.  Gaming is shuttering and giving less than 10fps COD Black ops and other games.

Hardware:

Motherboard - Asus Rog Zenith II Extreme Alpha TRX40 

Ram - Corsair Vengeance RGB Pro 64 GB

Harddrive - Samsung 970 EVO Plus SSD pcie nvme m.2 2TB

CPU - AMD Ryzen Threadripper 3960x 24 core 

GPU - GigaByte  Aorus Gefore RTX 3080 

Power Supply - Gigabyte GP -P850GM 850 W 

OS - Windows 10 Pro 

 

Unraid  Version: 6.9.0-rc1

 

GPU Passthrough Gigabyte.RTX3080.10240.201006.rom

 

 

XML VM Setup

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='4'>
  <name>DerbyWin10</name>
  <uuid>8b994699-15ec-ed64-45ca-8734365c2f96</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>21495808</memory>
  <currentMemory unit='KiB'>21495808</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>38</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='25'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='26'/>
    <vcpupin vcpu='4' cpuset='3'/>
    <vcpupin vcpu='5' cpuset='27'/>
    <vcpupin vcpu='6' cpuset='4'/>
    <vcpupin vcpu='7' cpuset='28'/>
    <vcpupin vcpu='8' cpuset='5'/>
    <vcpupin vcpu='9' cpuset='29'/>
    <vcpupin vcpu='10' cpuset='6'/>
    <vcpupin vcpu='11' cpuset='30'/>
    <vcpupin vcpu='12' cpuset='7'/>
    <vcpupin vcpu='13' cpuset='31'/>
    <vcpupin vcpu='14' cpuset='8'/>
    <vcpupin vcpu='15' cpuset='32'/>
    <vcpupin vcpu='16' cpuset='9'/>
    <vcpupin vcpu='17' cpuset='33'/>
    <vcpupin vcpu='18' cpuset='10'/>
    <vcpupin vcpu='19' cpuset='34'/>
    <vcpupin vcpu='20' cpuset='11'/>
    <vcpupin vcpu='21' cpuset='35'/>
    <vcpupin vcpu='22' cpuset='12'/>
    <vcpupin vcpu='23' cpuset='36'/>
    <vcpupin vcpu='24' cpuset='13'/>
    <vcpupin vcpu='25' cpuset='37'/>
    <vcpupin vcpu='26' cpuset='14'/>
    <vcpupin vcpu='27' cpuset='38'/>
    <vcpupin vcpu='28' cpuset='15'/>
    <vcpupin vcpu='29' cpuset='39'/>
    <vcpupin vcpu='30' cpuset='16'/>
    <vcpupin vcpu='31' cpuset='40'/>
    <vcpupin vcpu='32' cpuset='17'/>
    <vcpupin vcpu='33' cpuset='41'/>
    <vcpupin vcpu='34' cpuset='18'/>
    <vcpupin vcpu='35' cpuset='42'/>
    <vcpupin vcpu='36' cpuset='19'/>
    <vcpupin vcpu='37' cpuset='43'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-5.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/8b994699-15ec-ed64-45ca-8734365c2f96_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='19' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/disks/main/vms/DerbyWin10/vdisk1.img' index='3'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/disks/main/isos/SW_DVD9_NTRL_Win_10_1903_64BIT_English_HomeProEDUSLS_N_X21-99935.ISO' index='2'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/disks/main/isos/virtio-win-0.1.190-1.iso' index='1'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='sata' index='0'>
      <alias name='sata0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:b1:ef:ab'/>
      <source bridge='br0'/>
      <target dev='vnet2'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/2'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/2'>
      <source path='/dev/pts/2'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-4-DerbyWin10/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1.4'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/mnt/disks/main/isos/Gigabyte.RTX3080.10240.201006.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc52b'/>
        <address bus='9' device='9'/>
      </source>
      <alias name='hostdev2'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0b05'/>
        <product id='0x18bb'/>
        <address bus='7' device='6'/>
      </source>
      <alias name='hostdev3'/>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0b05'/>
        <product id='0x18f3'/>
        <address bus='7' device='5'/>
      </source>
      <alias name='hostdev4'/>
      <address type='usb' bus='0' port='4'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0b05'/>
        <product id='0x1915'/>
        <address bus='9' device='11'/>
      </source>
      <alias name='hostdev5'/>
      <address type='usb' bus='0' port='5'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0b05'/>
        <product id='0x1916'/>
        <address bus='9' device='5'/>
      </source>
      <alias name='hostdev6'/>
      <address type='usb' bus='0' port='6'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1b1c'/>
        <product id='0x0c18'/>
        <address bus='7' device='2'/>
      </source>
      <alias name='hostdev7'/>
      <address type='usb' bus='0' port='7'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x413c'/>
        <product id='0x5649'/>
        <address bus='5' device='2'/>
      </source>
      <alias name='hostdev8'/>
      <address type='usb' bus='0' port='8'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x8087'/>
        <product id='0x0029'/>
        <address bus='7' device='4'/>
      </source>
      <alias name='hostdev9'/>
      <address type='usb' bus='0' port='9'/>
    </hostdev>
    <hub type='usb'>
      <alias name='hub0'/>
      <address type='usb' bus='0' port='1'/>
    </hub>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

Images:

 

cpu-pin.png

 

cpu-specs.png

 

I have the PCI-e as an unassigned Device and another WD Attached USB drive.  Not sure if these are setup properly. 

drive-setup.png

Link to comment

I have a s limilar setup (3970x with 3 2080TIs) and also a spare (2950x with 3070 and 2080TI) and they all run games at very respectable frame rates.  The only real difference I see with your setup is the lack of passed through nvme (I have dedicated ones for each VM) and your clock setting is different.  I use the default xml code (basically just has hpet no hypervclock yes).  From what I've read clock settings can really influence Windows VM behavior.  Maybe something to check?  It seems mine are suboptimal but my results don't reflect that.  

Link to comment

Try to put this section into you VM xml settings:

<features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vpindex state='on'/>
      <synic state='on'/>
      <stimer state='on'/>
      <reset state='on'/>
      <vendor_id state='on' value='1234567890ab'/>
      <frequencies state='on'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
    <ioapic driver='kvm'/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='8' threads='1'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
    <feature policy='require' name='svm'/>
    <feature policy='require' name='apic'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='invtsc'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
    <timer name='tsc' present='yes' mode='native'/>
  </clock>

 

Link to comment
12 minutes ago, zeus83 said:

Try to put this section into you VM xml settings:


<features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vpindex state='on'/>
      <synic state='on'/>
      <stimer state='on'/>
      <reset state='on'/>
      <vendor_id state='on' value='1234567890ab'/>
      <frequencies state='on'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
    <ioapic driver='kvm'/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='8' threads='1'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
    <feature policy='require' name='svm'/>
    <feature policy='require' name='apic'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='invtsc'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
    <timer name='tsc' present='yes' mode='native'/>
  </clock>

 

I am pretty sure I have used most of those settings before (not all) and I don't recall seeing any benefits although I never specifically measured FPS for example.  One thing I do note is the topology...what's up with that?  Certainly doesn't mirror the host CPU.

Link to comment

I ran the LatencyMon program and found that my display driver installed was causing a huge lag.  So I uninstalled the driver and installed the 3080 rtx windows driver.  Now i am getting much better results.  I also tried your cpu settings but they didn't seem to help much.  

Link to comment
  • 9 months later...
On 2/1/2021 at 4:11 PM, sprange said:

I have a s limilar setup (3970x with 3 2080TIs) and also a spare (2950x with 3070 and 2080TI) and they all run games at very respectable frame rates.  The only real difference I see with your setup is the lack of passed through nvme (I have dedicated ones for each VM) and your clock setting is different.  I use the default xml code (basically just has hpet no hypervclock yes).  From what I've read clock settings can really influence Windows VM behavior.  Maybe something to check?  It seems mine are suboptimal but my results don't reflect that.  

 Hi All, 

I am having issues with same setup where server keeps crashing all the time. "The Ryzen" issue where it just locks up. 

Hardware

ROG Zenith II Extreme Alpha

AMD Ryzen Threadripper 3970x 32-Core

192GB DDR4 Corsair Vengeance Pro (256GB Kit, the other 64GB is in my Intel Server)

NVIDIA Quadro RTX 4000

Additional 1GB NIC Realtek RTL8111

ROG RYUJIN 360 AIO Water Cooler 

ASUS Hyper M.2 X16 Gen 4 RAID Controller Card with 4 x 1TB Samsung 980 Pro Gen 4

1TB Samsung 980 Pro on the Board Gen 4

1TB Samsung 980 Gen 3 UnAssigned

1TB Samsung 980 Gen 3 as Parity

 

So the even when the array is not mounted or doing anything it becomes unresponsive and a hard power down to restart fixes it. 

I have disabled C States, I have put in rcu_nocbs=0-63 in the config and nothing has helped so far. I have run this machine under Windows for a week and the hardware is stable, no crashes and get the full performance. So i am guessing its something to do  with my BIOS or UnRAID server settings. 

I am tempted to sell it and get 6 Beast Canyon NUCs. So they can each do my work in pieces. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.