SweetPeachez

Hello, unfortunately I ended up giving up due to not getting close to bare metal performance, this was mainly due cache not being mapped correctly for the 3960x under the Linux kernel that was being used at the time, funny enough is that I'm checking this week to see if this is still an issue and will post if there's any progress.

Hey all, I am having an issue with a new build when under load in my gaming VM Very easy to recreate, all I have to do is boot up the game, play for a bit and usually within 20 min I will get a crash that shuts down the VM and seems like it crashes unraid (am unable to access through webui or through ssh) I have upgraded my bios, and played around with various settings in both bios and unraid with no luck so far C-State has been disabled power supply is set to typical current Idle tried setting rcu_nocbs=0-47 in syslinux config Specs are MSI trx40 pro 10g AMD 3960x 128GB of Corsair 3200MHz RAM 2TB Samsung EVO 970 Plus NVME GTX 980Ti being passed through to Windows 10 VM am attaching the diagnostic zip that I downloaded after reboot of the server after the crash (not sure if because I got it after reboot makes it useful or not) Update: am unable to ping the system after crash tower-diagnostics-20200124-1800.zip

I'll be able to test this at some point this weekend...if y'all are wanting a guinea pig to test stuff on the 3960x please let me know, I'm more than willing to try whatever or test code and such.

No biggie! Pinning all cores and turning isolation off all cores got me 10646/12950 my Cache layout doesnt match bare metal and the settings you provided put windows into an unusable state. So it seems like I will be getting best performance with just banking with even cores on one VM and odd cores on the other? At least until another update comes out?

It's "she/her" by the way ....anyways yeah I'll check out Jerkys latest suggestion in a moment... Most of the time I wont need all 48 cpus, but I do a lot of very heavy data processing (a lot of it in GPU) but I do at times still find I need as many CPU cores I can get as well. For typical daily use the machine will be serving 2-3 Gaming VMs though, so still in my best interest to get everything working correctly. Anyways, I'll get to trying stuff after I grab some lunch.

Just ran my most recent configuration that gave me the highest score with the 2 threads 12 cores settings which scored more about 300 points lower, so gonna change that back and look into trying the settings that Jerky suggested

Per my post right before this one I ran that same XML but with just the odd bank cores and got a pretty large increase that I think brings me within range of bare metal (even though I am not using half the cores) EDIT: going to fix my threads per core in the XML and rerun

WoW! I ran 3dMark time spy on the odd bank just now and got 11309 in the VM where as the bare metal ran at 12950

the XML I had just posted before this post was my XML with the changes that Jerky suggested...In this post I'll post the XML that got me the highest score so far....which was my VM minus Jekys suggestions Here is the XML for the "Best" config, this config gives me the highest score in 3dMark time spy <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm' id='2'> <name>Windows 10_A</name> <uuid>af734937-bee3-267c-5a93-9fa189e66e7d</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>126877696</memory> <currentMemory unit='KiB'>126877696</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>46</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='25'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='26'/> <vcpupin vcpu='4' cpuset='3'/> <vcpupin vcpu='5' cpuset='27'/> <vcpupin vcpu='6' cpuset='4'/> <vcpupin vcpu='7' cpuset='28'/> <vcpupin vcpu='8' cpuset='5'/> <vcpupin vcpu='9' cpuset='29'/> <vcpupin vcpu='10' cpuset='6'/> <vcpupin vcpu='11' cpuset='30'/> <vcpupin vcpu='12' cpuset='7'/> <vcpupin vcpu='13' cpuset='31'/> <vcpupin vcpu='14' cpuset='8'/> <vcpupin vcpu='15' cpuset='32'/> <vcpupin vcpu='16' cpuset='9'/> <vcpupin vcpu='17' cpuset='33'/> <vcpupin vcpu='18' cpuset='10'/> <vcpupin vcpu='19' cpuset='34'/> <vcpupin vcpu='20' cpuset='11'/> <vcpupin vcpu='21' cpuset='35'/> <vcpupin vcpu='22' cpuset='12'/> <vcpupin vcpu='23' cpuset='36'/> <vcpupin vcpu='24' cpuset='13'/> <vcpupin vcpu='25' cpuset='37'/> <vcpupin vcpu='26' cpuset='14'/> <vcpupin vcpu='27' cpuset='38'/> <vcpupin vcpu='28' cpuset='15'/> <vcpupin vcpu='29' cpuset='39'/> <vcpupin vcpu='30' cpuset='16'/> <vcpupin vcpu='31' cpuset='40'/> <vcpupin vcpu='32' cpuset='17'/> <vcpupin vcpu='33' cpuset='41'/> <vcpupin vcpu='34' cpuset='18'/> <vcpupin vcpu='35' cpuset='42'/> <vcpupin vcpu='36' cpuset='19'/> <vcpupin vcpu='37' cpuset='43'/> <vcpupin vcpu='38' cpuset='20'/> <vcpupin vcpu='39' cpuset='44'/> <vcpupin vcpu='40' cpuset='21'/> <vcpupin vcpu='41' cpuset='45'/> <vcpupin vcpu='42' cpuset='22'/> <vcpupin vcpu='43' cpuset='46'/> <vcpupin vcpu='44' cpuset='23'/> <vcpupin vcpu='45' cpuset='47'/> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-4.2'>hvm</type> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='46' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10/vdisk1.img' index='3'/> <backingStore/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10/vdisk2.img' index='2'/> <backingStore/> <target dev='hdd' bus='virtio'/> <alias name='virtio-disk3'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.171.iso' index='1'/> <backingStore/> <target dev='hdb' bus='ide'/> <readonly/> <alias name='ide0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='ide' index='0'> <alias name='ide'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:26:17:8b'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-2-Windows 10_A/org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'> <alias name='input0'/> </input> <input type='keyboard' bus='ps2'> <alias name='input1'/> </input> <hostdev mode='subsystem' type='pci' managed='yes' xvga='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x21' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> <rom file='/mnt/user/vBios/myVBios.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x21' slot='0x00' function='0x1'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x04d9'/> <product id='0x0245'/> <address bus='1' device='4'/> </source> <alias name='hostdev2'/> <address type='usb' bus='0' port='1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x0db0'/> <product id='0x543d'/> <address bus='7' device='2'/> </source> <alias name='hostdev3'/> <address type='usb' bus='0' port='2'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x13fe'/> <product id='0x5500'/> <address bus='2' device='4'/> </source> <alias name='hostdev4'/> <address type='usb' bus='0' port='3'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1462'/> <product id='0x7c60'/> <address bus='7' device='3'/> </source> <alias name='hostdev5'/> <address type='usb' bus='0' port='4'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1b1c'/> <product id='0x1b2a'/> <address bus='1' device='6'/> </source> <alias name='hostdev6'/> <address type='usb' bus='0' port='5'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1b1c'/> <product id='0x1b2e'/> <address bus='1' device='5'/> </source> <alias name='hostdev7'/> <address type='usb' bus='0' port='6'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x264a'/> <product id='0x1fa5'/> <address bus='9' device='8'/> </source> <alias name='hostdev8'/> <address type='usb' bus='0' port='7'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x264a'/> <product id='0x1fa6'/> <address bus='9' device='10'/> </source> <alias name='hostdev9'/> <address type='usb' bus='0' port='8'/> </hostdev> <memballoon model='none'/> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+100</label> <imagelabel>+0:+100</imagelabel> </seclabel> </domain>

Gotcha, I'v had that done since my original post as well.

I'll give this a shot...my intentions after making sure I have the VM tuned correctly is to run 2 VMs that use half resources each (of course saving some resources for unraid as well)

I tried all of your suggestions above, also...I'm on the 3960x. My CPUZ bench in the VM Single thread is 508.8 Multi Thread is 16174.8 My CPUZ bench on bare metal Single thread is 518.8 Multi thread is 16823.0 So this leads me to believe that there may be something wrong with 3dMark benchmarks running in VMs? Also, are you saying that the CPU pinning as displayed in the unraid pinning menu can be incorrect? attaching a screenshot of my pinning my current XML <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm' id='4'> <name>Windows 10</name> <uuid>bfdb9f54-3503-24a2-979a-261040b9f2af</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>126877696</memory> <currentMemory unit='KiB'>126877696</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>46</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='25'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='26'/> <vcpupin vcpu='4' cpuset='3'/> <vcpupin vcpu='5' cpuset='27'/> <vcpupin vcpu='6' cpuset='4'/> <vcpupin vcpu='7' cpuset='28'/> <vcpupin vcpu='8' cpuset='5'/> <vcpupin vcpu='9' cpuset='29'/> <vcpupin vcpu='10' cpuset='6'/> <vcpupin vcpu='11' cpuset='30'/> <vcpupin vcpu='12' cpuset='7'/> <vcpupin vcpu='13' cpuset='31'/> <vcpupin vcpu='14' cpuset='8'/> <vcpupin vcpu='15' cpuset='32'/> <vcpupin vcpu='16' cpuset='9'/> <vcpupin vcpu='17' cpuset='33'/> <vcpupin vcpu='18' cpuset='10'/> <vcpupin vcpu='19' cpuset='34'/> <vcpupin vcpu='20' cpuset='11'/> <vcpupin vcpu='21' cpuset='35'/> <vcpupin vcpu='22' cpuset='12'/> <vcpupin vcpu='23' cpuset='36'/> <vcpupin vcpu='24' cpuset='13'/> <vcpupin vcpu='25' cpuset='37'/> <vcpupin vcpu='26' cpuset='14'/> <vcpupin vcpu='27' cpuset='38'/> <vcpupin vcpu='28' cpuset='15'/> <vcpupin vcpu='29' cpuset='39'/> <vcpupin vcpu='30' cpuset='16'/> <vcpupin vcpu='31' cpuset='40'/> <vcpupin vcpu='32' cpuset='17'/> <vcpupin vcpu='33' cpuset='41'/> <vcpupin vcpu='34' cpuset='18'/> <vcpupin vcpu='35' cpuset='42'/> <vcpupin vcpu='36' cpuset='19'/> <vcpupin vcpu='37' cpuset='43'/> <vcpupin vcpu='38' cpuset='20'/> <vcpupin vcpu='39' cpuset='44'/> <vcpupin vcpu='40' cpuset='21'/> <vcpupin vcpu='41' cpuset='45'/> <vcpupin vcpu='42' cpuset='22'/> <vcpupin vcpu='43' cpuset='46'/> <vcpupin vcpu='44' cpuset='23'/> <vcpupin vcpu='45' cpuset='47'/> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-4.2'>hvm</type> </os> <features> <acpi/> <apic/> <hyperv> <vpindex state='on'/> <synic state='on'/> <stimer state='on'/> <reset state='on'/> <vendor_id state='on' value='KVM Hv'/> <frequencies state='on'/> </hyperv> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='23' threads='2'/> <cache mode='passthrough'/> <feature policy='require' name='topoext'/> <feature policy='disable' name='monitor'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='svm'/> <feature policy='disable' name='x2apic'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='yes'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10/vdisk1.img' index='4'/> <backingStore/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Windows 10/vdisk2.img' index='3'/> <backingStore/> <target dev='hdd' bus='virtio'/> <alias name='virtio-disk3'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Windows10.iso' index='2'/> <backingStore/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <alias name='sata0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.171.iso' index='1'/> <backingStore/> <target dev='hdb' bus='sata'/> <readonly/> <alias name='sata0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='sata' index='0'> <alias name='sata0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:94:be:2f'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-4-Windows 10/org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'> <alias name='input0'/> </input> <input type='keyboard' bus='ps2'> <alias name='input1'/> </input> <hostdev mode='subsystem' type='pci' managed='yes' xvga='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x21' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> <rom file='/mnt/user/vBios/myVBios.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x21' slot='0x00' function='0x1'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x04d9'/> <product id='0x0245'/> <address bus='1' device='4'/> </source> <alias name='hostdev2'/> <address type='usb' bus='0' port='1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x0db0'/> <product id='0x543d'/> <address bus='7' device='2'/> </source> <alias name='hostdev3'/> <address type='usb' bus='0' port='2'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x13fe'/> <product id='0x5500'/> <address bus='2' device='4'/> </source> <alias name='hostdev4'/> <address type='usb' bus='0' port='3'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1462'/> <product id='0x7c60'/> <address bus='7' device='3'/> </source> <alias name='hostdev5'/> <address type='usb' bus='0' port='4'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1b1c'/> <product id='0x1b2a'/> <address bus='1' device='6'/> </source> <alias name='hostdev6'/> <address type='usb' bus='0' port='5'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1b1c'/> <product id='0x1b2e'/> <address bus='1' device='5'/> </source> <alias name='hostdev7'/> <address type='usb' bus='0' port='6'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x264a'/> <product id='0x1fa5'/> <address bus='9' device='5'/> </source> <alias name='hostdev8'/> <address type='usb' bus='0' port='7'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x264a'/> <product id='0x1fa6'/> <address bus='9' device='7'/> </source> <alias name='hostdev9'/> <address type='usb' bus='0' port='8'/> </hostdev> <memballoon model='none'/> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+100</label> <imagelabel>+0:+100</imagelabel> </seclabel> </domain>

Just tried it, with the different machine type / seabios machine and it gave me a decrease to about the level I was seeing before....working on getting you CPU-z benchmarks in a moment....also, I'm sure I should be on the latest stable version of unraid as I just downloaded it this last week. Edit: version 6.8.1

Just made a fresh VM with that machine type and seabios and hyperv on and i got a decent boost...now at 10089 on VM compared to 12950 on bare metal

ok, just tried this both ways and no dice..it put windows in an unusable state (Extremely laggy / unresponsive)