Nooke Posted October 23, 2018 Share Posted October 23, 2018 (edited) hi folks, I'm fiddling around for some weeks now trying to get decent performance on my gaming VM. current specs: CPU: Ryzen Threadripper 1920x @4GHz (CPU Voltage: 1.35) RAM: Corsair Vengeance Pro DDR4 3000MHz CL15-17-17-35 @ 2933MHz CL15-17-17-35 (RAM Voltage: 1.35) -> using 12GB for the VM Mainboard: MSI X399 SLI Plus (latest BIOS) GPU: Zotac GTX 1080 TI Blower Cooling: CPU + MB + GPU water cooled with custom loop. (MORA3) SSD: 1TB Samsung 970 Evo (nvme drive) Unraid: 6.6.3 My current .xml file for the windows 10 VM <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm' id='1'> <name>Windows 10</name> <uuid>0b9950e2-8ef7-640f-5fab-e10486f8077d</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>12582912</memory> <currentMemory unit='KiB'>12582912</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <iothreads>2</iothreads> <cputune> <vcpupin vcpu='0' cpuset='16'/> <vcpupin vcpu='1' cpuset='17'/> <vcpupin vcpu='2' cpuset='18'/> <vcpupin vcpu='3' cpuset='19'/> <vcpupin vcpu='4' cpuset='20'/> <vcpupin vcpu='5' cpuset='21'/> <vcpupin vcpu='6' cpuset='22'/> <vcpupin vcpu='7' cpuset='23'/> <emulatorpin cpuset='12-13'/> <iothreadpin iothread='1' cpuset='14-15'/> <iothreadpin iothread='2' cpuset='14-15'/> </cputune> <numatune> <memory mode='strict' nodeset='1'/> </numatune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-3.0'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/0b9950e2-8ef7-640f-5fab-e10486f8077d_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='MSI'/> </hyperv> <kvm> <hidden state='on'/> </kvm> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='8' threads='1'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> <timer name='hypervclock' present='yes'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/mnt/user/domains/Windows 10/vdisk1.raw'/> <backingStore/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Win10_1803_German_x64.iso'/> <backingStore/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <alias name='sata0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/> <backingStore/> <target dev='hdb' bus='sata'/> <readonly/> <alias name='sata0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='sata' index='0'> <alias name='sata0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:a9:e1:4b'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-Windows 10/org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'> <alias name='input0'/> </input> <input type='keyboard' bus='ps2'> <alias name='input1'/> </input> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x42' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> <rom file='/mnt/user0/isos/1080ti-mod.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x42' slot='0x00' function='0x1'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0a' slot='0x00' function='0x3'/> </source> <alias name='hostdev2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x43' slot='0x00' function='0x3'/> </source> <alias name='hostdev3'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/> </hostdev> <memballoon model='virtio'> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </memballoon> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+100</label> <imagelabel>+0:+100</imagelabel> </seclabel> </domain> IOMMU Groups IOMMU group 0:[1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 1:[1022:1453] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 2:[1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 3:[1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 4:[1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 5:[1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 6:[1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 7:[1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 8:[1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 9:[1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 10:[1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 IOMMU group 11:[1022:1460] 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1461] 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1462] 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1463] 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1464] 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1465] 00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1466] 00:19.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1467] 00:19.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 IOMMU group 12:[1022:43ba] 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller (rev 02) [1022:43b6] 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller (rev 02) [1022:43b1] 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge (rev 02) [1022:43b4] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1b21:2142] 03:00.0 USB controller: ASMedia Technology Inc. ASM2142 USB 3.1 Host Controller [8086:1539] 08:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 13:[1022:145a] 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a IOMMU group 14:[1022:1456] 0a:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor IOMMU group 15:[1022:145c] 0a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller IOMMU group 16:[1022:1455] 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 IOMMU group 17:[1022:7901] 0b:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 18:[1022:1457] 0b:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller IOMMU group 19:[1022:1452] 40:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 20:[1022:1453] 40:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 21:[1022:1452] 40:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 22:[1022:1452] 40:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 23:[1022:1453] 40:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 24:[1022:1452] 40:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 25:[1022:1452] 40:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 26:[1022:1454] 40:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 27:[1022:1452] 40:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge IOMMU group 28:[1022:1454] 40:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 29:[144d:a808] 41:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 IOMMU group 30:[10de:1b06] 42:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1) [10de:10ef] 42:00.1 Audio device: NVIDIA Corporation GP102 HDMI Audio Controller (rev a1) IOMMU group 31:[1022:145a] 43:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a IOMMU group 32:[1022:1456] 43:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor IOMMU group 33:[1022:145c] 43:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller IOMMU group 34:[1022:1455] 44:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455 IOMMU group 35:[1022:7901] 44:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) Log 2018-10-23 09:53:20.641+0000: starting up libvirt version: 4.7.0, qemu version: 3.0.0, kernel: 4.18.15-unRAID, hostname: Friday LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name 'guest=Windows 10,debug-threads=on' -S -object 'secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-Windows 10/master-key.aes' -machine pc-i440fx-3.0,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=MSI,kvm=off -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/0b9950e2-8ef7-640f-5fab-e10486f8077d_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 12288 -realtime mlock=off -smp 8,sockets=1,cores=8,threads=1 -object iothread,id=iothread1 -object iothread,id=iothread2 -uuid 0b9950e2-8ef7-640f-5fab-e10486f8077d -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=23,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_ce virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device vfio-pci,host=42:00.0,id=hostdev0,bus=pci.0,addr=0x6,romfile=/mnt/user0/isos/1080ti-mod.rom -device vfio-pci,host=42:00.1,id=hostdev1,bus=pci.0,addr=0x8 -device vfio-pci,host=0a:00.3,id=hostdev2,bus=pci.0,addr=0xb -device vfio-pci,host=43:00.3,id=hostdev3,bus=pci.0,addr=0xc -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x9 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on 2018-10-23 09:53:20.641+0000: Domain id=1 is tainted: high-privileges 2018-10-23 09:53:20.641+0000: Domain id=1 is tainted: host-cpu 2018-10-23T09:53:20.689994Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/0 (label charserial0) System Overview System Overview unRAID system: unRAID server Pro, version 6.6.3 Model: Custom Motherboard: Micro-Star International Co., Ltd. - X399 SLI PLUS (MS-7B09) Processor: AMD Ryzen Threadripper 1920X 12-Core @ 4 GHz HVM: Enabled IOMMU: Enabled Cache: L1 - Cache = 1152 kB (max. capacity 1152 kB) L2 - Cache = 6144 kB (max. capacity 6144 kB) L3 - Cache = 32768 kB (max. capacity 32768 kB) Memory: 32 GB (max. installable capacity 512 GB) DIMM 1 = 8192 MB, 2933 MT/s DIMM 1 = 8192 MB, 2933 MT/s DIMM 1 = 8192 MB, 2933 MT/s DIMM 1 = 8192 MB, 2933 MT/s Network: bond0: fault-tolerance (active-backup), mtu 1500 eth0: 1000Mb/s, full duplex, mtu 1500 Kernel: Linux 4.18.15-unRAID x86_64 OpenSSL: 1.1.0i P + Q algorithm: 16236 MB/s + 30531 MB/s Uptime: 0 days, 6 hours, 6 minutes, 32 seconds Cinebench working flawless -> results OpenGL = 89.40 Multi-Core Performance = 823 Single-Core Performance = 123 Temps GPU = 30°C idle -> 35-40°C full load CPU = 40°C idle -> 55-60°C full load But in gaming I always get fps drops - sometimes even the whole screen just hangs for 3-7 seconds. GPU and CPU usage looks fine so far. RAM configuration currently at NUMA mode and I pinned the gaming VM to numa node 1 (where the 1080 TI is connected to). Cores 12-15 for emulator/iothread pin and the cores 16-23 for the VM itself. I did try the Q35 instead of I440 without any improvement. I didn't passthrough the nmve SSD cause I want to use it as a cache drive aswell (is there a real performance increase if I passthrough the disk??) LSTOPO output Would appreciate any help. If any more informationen needed, feel free to ask for it cheers Nooke Edited October 23, 2018 by Nooke Quote Link to comment
david279 Posted October 23, 2018 Share Posted October 23, 2018 <topology sockets='1' cores='4' threads='2'/> This would make use of SMT in the VM with qemu 3.0. Quote Link to comment
Nooke Posted October 23, 2018 Author Share Posted October 23, 2018 1 hour ago, david279 said: <topology sockets='1' cores='4' threads='2'/> This would make use of SMT in the VM with qemu 3.0. So is there any performance increase by doing that? Instead of showing 8 cores to the vm show 4 cores with 2 threads each? Quote Link to comment
david279 Posted October 23, 2018 Share Posted October 23, 2018 it made my gaming in my VM better for sure. Im using a ryzen 2700x. Quote Link to comment
Nooke Posted October 23, 2018 Author Share Posted October 23, 2018 6 minutes ago, david279 said: it made my gaming in my VM better for sure. Im using a ryzen 2700x. tried it with some benchmarks but couldn't see any performance loss neither gain. I don't know if this is handled at all by qemu as my VM Guest still shows "8 virtual processors" instead of 4 processors with 8 threads. Anyways I let your suggestion in my xml - maybe it helps Quote Link to comment
bastl Posted October 24, 2018 Share Posted October 24, 2018 (edited) Did you tried to remove the emulator/iothread pinning from the VM? For me it works best, pinning all cores from Node1 directly to the VM without emulator pin on my 1950x. Basically i don't need any manual edit of the xml. It runs for a year now without any hickups. What's the power plan of your Windows VM? Set it to performance if it isn't already. Edit: Also your single core score looks a bit low for 4GHz. I get around 145-150 on the same speed. Edited October 24, 2018 by bastl Quote Link to comment
Nooke Posted October 24, 2018 Author Share Posted October 24, 2018 (edited) 2 hours ago, bastl said: Did you tried to remove the emulator/iothread pinning from the VM? For me it works best, pinning all cores from Node1 directly to the VM without emulator pin on my 1950x. Basically i don't need any manual edit of the xml. It runs for a year now without any hickups. What's the power plan of your Windows VM? Set it to performance if it isn't already. Edit: Also your single core score looks a bit low for 4GHz. I get around 145-150 on the same speed. In the beginning I just set up core 12-23 for the VM and let the unraid gui do the rest but I've got worse performance with this configuration. So I ended up using emulator und iothread pinning and it felt much smoother. Power plan in Windows is set to Performance. I even tried different cache/io modes for the vdisk and set up level 3 cache etc. There are so much options for libvirt but I can't seem to find anything related to the threadripper 1st generation Regarding the Cinebench score - I made some changes to my RAM configuration in BIOS and got 148 single cpu score now I think that's fine if I compare that with baremetal Cinebench on my old rig (i7-4770k) which had 165 score Edited October 24, 2018 by Nooke Quote Link to comment
bastl Posted October 24, 2018 Share Posted October 24, 2018 The only big difference i see is that you use the ACS Override patch and i guess you use the "multifunctional" option to split the IOMMU groups up even further. On older builds i had to do the same thing to get my NVME controller split into it's own group. With the latest BIOS and the newer Kernel in 6.xx versions i don't need the ACS override anymore for my needs. The only thing that is grouped together are the network devices and the USB controller which i don't passthrough anyways. USB mouse and keyboard i can attach without issues with the gui and if i need to attach another USB device like an USB HDD or stick the "Libvirt Hotplug USB" plugin works flawlessly. Quote Link to comment
Nooke Posted October 24, 2018 Author Share Posted October 24, 2018 3 minutes ago, bastl said: The only big difference i see is that you use the ACS Override patch and i guess you use the "multifunctional" option to split the IOMMU groups up even further. On older builds i had to do the same thing to get my NVME controller split into it's own group. With the latest BIOS and the newer Kernel in 6.xx versions i don't need the ACS override anymore for my needs. The only thing that is grouped together are the network devices and the USB controller which i don't passthrough anyways. USB mouse and keyboard i can attach without issues with the gui and if i need to attach another USB device like an USB HDD or stick the "Libvirt Hotplug USB" plugin works flawlessly. yeah I'm passing through 2 USB controllers which are used for the rear USB ports on the motherboard. So I had to use the ACS Override patch to get those 2 in their own groups and not touching the one where my usb flashdrive for Unraid is connected to. I use the "downstream" option as PCIe ACS override and "VFIO allow unsafe interrupts" is set to yes aswell Quote Link to comment
bastl Posted October 24, 2018 Share Posted October 24, 2018 (edited) The "VFIO allow unsafe interrupts" if have set to no. Which version of your BIOS you're on? With the release of TR4 gen2 all the manufacturers released a newer version which improves stability on gen1 as well, fixed a couple issues and had better IOMMU groupings. EDIT: Nevermind, you're on the latest BIOS i noticed to late. Edited October 24, 2018 by bastl Quote Link to comment
Nooke Posted October 24, 2018 Author Share Posted October 24, 2018 I'm running the latest BIOS version provided by MSI 7B09vA6 (release: 2018-08-14) Quote Link to comment
bastl Posted October 24, 2018 Share Posted October 24, 2018 (edited) I also don't use the numatune section. I think someone wrote that it hasn't any effect on how unRaid serves the RAM to the VM from a specific node. It might have changed with one of the last libvirt updates. Not sure. <numatune> <memory mode='strict' nodeset='1'/> </numatune> and i'am still on an older i440fx version <type arch='x86_64' machine='pc-i440fx-2.10'>hvm</type> and the hide KVM option i also don't have in my xml <kvm> <hidden state='on'/> </kvm> Another difference is my xml is missing this part <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> The cause might be you're using the newer machine type for your VM. I will attach my current XML if you like to fidle around with it. WIN10_NVME_UEFI.txt Edited October 24, 2018 by bastl Quote Link to comment
Symon Posted October 27, 2018 Share Posted October 27, 2018 (edited) On 10/24/2018 at 12:24 PM, Nooke said: Regarding the Cinebench score - I made some changes to my RAM configuration in BIOS and got 148 single cpu score now I think that's fine if I compare that with baremetal Cinebench on my old rig (i7-4770k) which had 165 score Can I ask what you changed to get better performance? I'm runnig my 1950 @ 3.875 and get a singlecore score of 127. This certainly doesn't fir together with my Windows performance measurement 😀 Edited October 27, 2018 by Symon Quote Link to comment
bastl Posted October 28, 2018 Share Posted October 28, 2018 @Symon What cores do you have assigned to the VM and what motherboard are you using? Quote Link to comment
Nooke Posted October 28, 2018 Author Share Posted October 28, 2018 Ended up starting with a fresh unraid flash drive as I had some strange unraid messages (probably cause of a Upgrade from Unraid 6.5 to 6.6) I just changed RAM configuration (clock 2933MHz, CL15-17-17-35) and clocked the CPU to 4.1GHz In addition I removed my SSD as a Cache drive from unraid and directly passthrough the NVMe drive to the Windows VM. This made some improvements in AS SSD Benchmark. After some more digging around I changed some CPU topology stuff in the .xml. Running as 5 core / 10 threads right now Cinebench Results: OpenGL: 83.58 CPU (multicore): 987 CPU (singlecore): 135 AIDA64 Cache Benchmark result Well I let LatencyMon running while doing a CineBench and the results are awfully bad. I already changed all interrupts vom IRQ to MSI. Any Ideas? Attached you find my current libvirt XML win10.xml Still having same issues as before - fps drops (Call of Duty Black Ops 4 as example) that lead to kinda "freezes" for about 1-4 seconds. This makes it unusable. Does it make sense to fiddle around with the vCPUs?? Currently I have it as the following <vcpupin vcpu='0' cpuset='14'/> <vcpupin vcpu='1' cpuset='15'/> <vcpupin vcpu='2' cpuset='16'/> <vcpupin vcpu='3' cpuset='17'/> <vcpupin vcpu='4' cpuset='18'/> <vcpupin vcpu='5' cpuset='19'/> <vcpupin vcpu='6' cpuset='20'/> <vcpupin vcpu='7' cpuset='21'/> <vcpupin vcpu='8' cpuset='22'/> <vcpupin vcpu='9' cpuset='23'/> Would make this here more sense? <vcpupin vcpu='0' cpuset='14'/> <vcpupin vcpu='1' cpuset='16'/> <vcpupin vcpu='2' cpuset='18'/> <vcpupin vcpu='3' cpuset='20'/> <vcpupin vcpu='4' cpuset='22'/> <vcpupin vcpu='5' cpuset='15'/> <vcpupin vcpu='6' cpuset='17'/> <vcpupin vcpu='7' cpuset='19'/> <vcpupin vcpu='8' cpuset='21'/> <vcpupin vcpu='9' cpuset='23'/> Appreciate any help in this cheers Nooke Quote Link to comment
bastl Posted October 28, 2018 Share Posted October 28, 2018 6 hours ago, Nooke said: AIDA64 Cache Benchmark result Interresting, Aida shows your core clock at 2300MHz. For me it always shows as 195-205MHz. 2300MHz i think is the clock speed what the "power save" governor limits my CPU to. Can you install the "Tips and Tweaks" app via the Community Applications and check what "CPU Scaling Governor" your system is set to? If it's set to "Power Save" change it to "On Demand" might fix it. Quote Link to comment
Nooke Posted October 28, 2018 Author Share Posted October 28, 2018 1 minute ago, bastl said: Interresting, Aida shows your core clock at 2300MHz. For me it always shows as 195-205MHz. 2300MHz i think is the clock speed what the "power save" governor limits my CPU to. Can you install the "Tips and Tweaks" app via the Community Applications and check what "CPU Scaling Governor" your system is set to? If it's set to "Power Save" change it to "On Demand" might fix it. Governor set to "Performance" right now. Shall I switch to On Demand? I deactivated all turbo/speedstep settings and running all time at 100% frequency (don't care about power consumption) Quote Link to comment
bastl Posted October 28, 2018 Share Posted October 28, 2018 Weird. Open a Terminal to the server and check the frequencies with and without load on the system watch grep \"cpu MHz\" /proc/cpuinfo Could also be a cooling issue. My Enermax Liqtech 360 AIO died after 3 months and as soon as the temperatures spiked up the core clocks went down to around 2300MHz. Quote Link to comment
Nooke Posted October 28, 2018 Author Share Posted October 28, 2018 I have a custom waterloop with an MO-RA3 - so there is no temperature issues GPU is running at 30-45°C. CPU maximum 60°C at high load Quote Link to comment
bastl Posted October 28, 2018 Share Posted October 28, 2018 What kind of load you put on your VM as you mad ethat picture? Cinebench or most other stress tests should show cores 14-23 at 100%. I guess the CPU clock shown in CB is because of your custom XML edits <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>EPYC-IBPB</model> <vendor>AMD</vendor> <topology sockets='1' cores='5' threads='2'/> <cache level='3' mode='emulate'/> <feature policy='require' name='topoext'/> <feature policy='disable' name='monitor'/> <feature policy='require' name='x2apic'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='svm'/> </cpu> setting the topology like you did, for me never made any difference to the defaults unraid sets up. Try to use the following part instead of yours. <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='5' threads='2'/> </cpu> I don't know where you get all the edits from. It should be fine without. "disable svm" looks odd enough and also to emulate the L3 cache. Quote Link to comment
Nooke Posted October 28, 2018 Author Share Posted October 28, 2018 (edited) Regarding the CPU Load, just had the Windows VM at idle (Google Chrome with some videos on) so no full load scenario. Anyways with 100% CPU load (cinebench for example) the frequency for the cores keep the same I had that host-passthrough when I first created the VM. So the default unraid libvirt settings doesn't seem to work for me that's the reason I google'd a bit and did some tuning (libvirt.org and some forum posts regarding threadripper tuning). For example with host-passthrough I couldn't finish a multicore cinebench. Passmark is not working aswell with host-passthrough (insta client freeze). that EPYC-IBPB was just added recently - you can check hosts libvirt capabilities via terminal "virsh capabilities". Thought this could solve my issue but I think I did like almost every xml tuning I've found and the issue with frame drops still persist. Edited October 28, 2018 by Nooke Quote Link to comment
bastl Posted October 28, 2018 Share Posted October 28, 2018 Freezes on Cinebench is definitly a sign for a not stable OC. You might turn down your OC first and get your system stable before goin the next step to finetune your VM. Even on stock 3.4 to 3.7 boost games should run smooth without lags. On 4.1GHz for me on usual tasks it looks stable. No crashes no hickups. 1 out of 5 tests with Cinebench it crashes for me. Games i never tested on that clock speed cause it makes no sense to don't have a stable system on heavy workloads and than expect to be able to play games. Quote Link to comment
Nooke Posted October 28, 2018 Author Share Posted October 28, 2018 2 minutes ago, bastl said: Freezes on Cinebench is definitly a sign for a not stable OC. You might turn down your OC first and get your system stable before goin the next step to finetune your VM. Even on stock 3.4 to 3.7 boost games should run smooth without lags. On 4.1GHz for me on usual tasks it looks stable. No crashes no hickups. 1 out of 5 tests with Cinebench it crashes for me. Games i never tested on that clock speed cause it makes no sense to don't have a stable system on heavy workloads and than expect to be able to play games. It's totally stable with this cpu-model in the XML same as with QEMU64. On bare metal (still have another Samsung 850 EVO here) the OC is working fine aswell. I had the same freezes with stock CPU settings - one of the reasons I overclocked it The OC is totally stable, tested on bare metal several hours and afterwards with qemu emulated cpu (qemu64 or epyc). passmark was crashing the VM when I just started the program and it tried to scan the computer hardware (with host-passthrough) - that wasn't a load issue at all but seems to be a specific cpu feature which is enabled by default with the host-passthrough and not really accessable inside the VM. cheers Nooke Quote Link to comment
bastl Posted October 28, 2018 Share Posted October 28, 2018 (edited) Whats the terminal output of "numactl -H" and "numastat qemu" with a running VM? Edited October 28, 2018 by bastl Quote Link to comment
Nooke Posted October 28, 2018 Author Share Posted October 28, 2018 root@Friday:~# numactl -H available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 node 0 size: 15931 MB node 0 free: 315 MB node 1 cpus: 12 13 14 15 16 17 18 19 20 21 22 23 node 1 size: 16115 MB node 1 free: 156 MB node distances: node 0 1 0: 10 16 1: 16 10 root@Friday:~# numastat qemu Per-node process memory usage (in MBs) for PID 8863 (qemu-system-x86) Node 0 Node 1 Total --------------- --------------- --------------- Huge 0.00 0.00 0.00 Heap 0.00 0.05 0.05 Stack 0.00 0.03 0.03 Private 9.90 14927.99 14937.89 ---------------- --------------- --------------- --------------- Total 9.90 14928.07 14937.97 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.