PapaBless

I have the same problem. It happens rebooting a macOS VM with a RX580 passed-through on the second PCIe slot.

Hi, I've been trying to get near native performance for months now. I'm using Rust to test the performance, since is the game that has the most noticeable changes. On bare metal it runs over 100fps avg, (80 ~ 130fps), on UNRAID it reach 85fps max with an average of ~70fps (using the attached xml config). I expect to perform a little worse because Rust is very CPU hungry and I'm assigning 7 cpus (14 with HT) to the VM instead of 8. With other games like COD: MW is the same perf loss, CS:GO scored around 130 ~ 200fps at max settings. Hardware: - ASUS TUF Z390-PRO Gaming - AORUS GeForce RTX 2080 8G - Sapphire Radeon RX 580 8G - Intel® Core i9-9900K - 64GB 2666Mhz DDR4 - Samsung SSD 970 EVO Plus 500GB - 2TB SSD What I've tried: - Isolate CPUs but the first one for <emulatorpin>. (big performance gain, less stutter) - Passthrough of the gpu with all its functions in one bus. (slight performance gain) - Set kvm64 instead of host-passthrough. No noticeable difference. - Create a raw and qcow vdisk using VirtIO and SCSI on each. No big difference here. - Enable/Disable ACS override (IOMMU). No big difference, but it was more stable (using only the NVIDIA card). - Removing all PCIE devices and using only the NVIDIA card on the first slot, with and without its functions passed-through. - Split windows vdisk and steam game library on two different SSDs, also passthrough a physical ssd. - Assign first cores with hyper-threading enabled. - Disable hyper-threading. - Pass both the RX580 and the NVIDIA with ACS disabled, since they share the same IOMMU group I tested each one of these using different cores and pairings, different amounts of ram and disk sizes, and Windows power configuration. Is there something I'm missing? Boot config: kernel /bzimage append i915.alpha_support=1 pci-stub.ids=14e4:43a0 vfio-pci.ids=8086:a348,10de:10f8,10de:1ad8,1b73:1100,8086:24f3,14e4:43a0 pci=noaer modprobe.blacklist=amdgpu isolcpus=1-7 kvm.ignore_msrs=1 initrd=/bzroot Here is the XML that works best atm: <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10 Gaming (all cores)</name> <uuid>00000000-0000-0000-0000-000000000000</uuid> <description>nvidia, all cores</description> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>33554432</memory> <currentMemory unit='KiB'>33554432</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>7</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='2'/> <vcpupin vcpu='2' cpuset='3'/> <vcpupin vcpu='3' cpuset='4'/> <vcpupin vcpu='4' cpuset='5'/> <vcpupin vcpu='5' cpuset='6'/> <vcpupin vcpu='6' cpuset='7'/> <emulatorpin cpuset='0'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-4.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/00000000-0000-0000-0000-000000000000_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='7' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <!-- Windows (no performance diff between raw/qcow2) --> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='writeback'/> <source file='/mnt/cache/domains/Windows 10 Gaming/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </disk> <!-- Game library --> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/disks/shared/game-library.raw'/> <target dev='hdd' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </disk> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0x15'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/> </controller> <controller type='pci' index='7' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='7' port='0x16'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/> </controller> <controller type='pci' index='8' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='8' port='0x17'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/> </controller> <controller type='pci' index='9' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='9' port='0x8'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <interface type='bridge'> <mac address='xx:xx:xx:xx:xx:xx'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='6'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <!-- NVIDIA 2080 w/multifunction --> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <rom file='/mnt/disks/shared/other/NVIDIA.RTX2080.rom'/> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x1'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x2'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x3'/> </source> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x3'/> </hostdev> <!-- Fresco USB Hub --> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </hostdev> <memballoon model='none'/> </devices> </domain> Thanks!