macmus

System specs - Unraid 7.2.4, kernel 6.12.54 - AMD Threadripper / Matisse platform (ASUS board) - QEMU 9.2.3 / libvirt 11.7.0 - RTX 4060 Gigabyte (AD107, 01:00.0) — passed through to Windows 10 VM - RTX 2080 Ti (TU102, 4d:00.0) — passed through to separate Windows 11 VM (working fine, no issues) - RTX 4060 is the primary/boot VGA on the host — console access must be preserved --- Problem My Windows 10 VM with RTX 4060 passthrough fails to start after the VM has been stopped or crashed, with this error: ``` Execution error internal error: Unknown PCI header type '127' for device '0000:01:00.0' ``` A full Unraid server reboot always recovers it — the VM starts fine on first boot. The problem only happens when trying to restart the VM without rebooting the host. --- IOMMU grouping — clean, no sharing issues ``` IOMMU Group 74 01:00.0 NVIDIA Corporation AD107 [GeForce RTX 4060] [10de:2882] (rev a1) IOMMU Group 74 01:00.1 NVIDIA Corporation AD107 HD Audio Controller [10de:22be] (rev a1) ``` Both functions are isolated in their own group. No ACS override needed. --- Current boot args ``` BOOT_IMAGE=/bzimage vfio-pci.ids=1022:148c,10de:1ad6,10de:1ad7 video=efifb:off isolcpus=1-14,25-38 initrd=/bzroot kvm_amd.nested=1 ``` Note: 10de:2882 and 10de:22be (the RTX 4060) are NOT in vfio-pci.ids. The 2080 Ti functions 10de:1ad6 and 10de:1ad7 are listed but not the GPU itself either — those were added for its USB/UCSI controllers. --- Driver state after VM shutdown ``` ls: cannot access '/sys/bus/pci/devices/0000:01:00.0/driver': No such file or directory ls: cannot access '/sys/bus/pci/devices/0000:01:00.1/driver': No such file or directory ``` No driver is bound to the GPU between VM sessions. On boot it is detected correctly: ``` [ 0.392439] pci 0000:01:00.0: [10de:2882] type 00 class 0x030000 PCIe Legacy Endpoint [ 0.435513] pci 0000:01:00.0: vgaarb: setting as boot VGA device [ 2.086958] vfio_pci: add [1022:148c[ffffffff:ffffffff]] class 0x000000/00000000 ``` The RTX 4060 is being picked up as boot VGA by the host, and is NOT being claimed by vfio-pci at boot (only the 2080 Ti USB controllers are). --- dmesg after VM shutdown — GPU reset failure ``` vfio-pci 0000:01:00.0: vfio_bar_restore: reset recovery - restoring BARs [repeats ~30 times] vfio-pci 0000:01:00.0: timed out waiting for pending transaction; performing function level reset anyway vfio-pci 0000:01:00.1: Unable to change power state from D0 to D3hot, device inaccessible pcieport 0000:00:01.1: broken device, retraining non-functional downstream link at 2.5GT/s pcieport 0000:00:01.1: retraining failed pcieport 0000:00:01.1: Data Link Layer Link Active not set in 100 msec vfio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible vfio-pci 0000:01:00.1: Unable to change power state from D3cold to D0, device inaccessible ``` The PCIe link to the GPU drops entirely after the VM stops. On the next VM start QEMU reads all 0xFF from the config space (device unreachable) and reports header type 127. --- VM XML — relevant passthrough section ```xml <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/isos/vbios/GeForceRTX4060.rom'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/> </hostdev> ``` The custom vBIOS ROM file is only 61KB — which seems far too small for an RTX 4060 (I'd expect 200KB+). Not sure if this is contributing. --- What I think is happening The RTX 4060 AD107 appears to have a reset bug similar to the AMD reset bug — after VM shutdown the FLR fails, the PCIe link drops, and the card is completely inaccessible until a full power cycle. The 2080 Ti on the same system passes through without any such issues. --- Questions 1. Is there a known fix for ths reset BUG? 2. Would adding pcie=noaer to boot args help suppress the PCIe error cascade that drops the link? I don't want to brick my Unraid server, but still be able to access it over console. 3. Is my vBIOS ROM ok? I'm not sure how did i get it, since it was long ago. Should I remove it or switch to <rom bar='off'/>? Where i can download a working 4060 bios ? 4. Can I safely add 10de:2882,10de:22be to vfio-pci.ids without losing host console/display output — given this GPU is also the host's primary display output? Thanks in advance. Happy to run any additional diagnostics.

I found this errors in my server logs folder, i don't remember seeing this before. Do you know what it is and how critial is this ? Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CA.WT1A], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CA.MT1A], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CA.WT2A], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CA.MT2A], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CA.WT3A], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CA.MT3A], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CA.WT4A], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CA.MT4A], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CA.MT5A], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CB.WT1B], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CB.MT1B], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CB.WT2B], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CB.MT2B], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CB.WT3B], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CB.MT3B], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CB.WT4B], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CB.MT4B], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CB.MT5B], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CC.WT1C], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CC.MT1C], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CC.WT2C], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CC.MT2C], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CC.WT3C], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CC.MT3C], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CC.WT4C], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CC.MT4C], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CC.MT5C], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CD.WT1D], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CD.MT1D], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CD.WT2D], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CD.MT2D], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CD.WT3D], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CD.MT3D], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CD.WT4D], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CD.MT4D], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): Failure creating named object [\_SB.I2CD.MT5D], AE_ALREADY_EXISTS (20220331/dswload2-326) Sep 22 09:56:06 Tower kernel: ACPI Error: AE_ALREADY_EXISTS, During name lookup/catalog (20220331/psobject-220) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): AE_AML_PACKAGE_LIMIT, Index (0x000000007) is beyond end of object (length 0x6) (20220331/exoparg2-393) Sep 22 09:56:06 Tower kernel: ACPI Error: Aborting method \_SB.S0D2.D2A0.BYUP.BYD8.XHC1.RHUB.PRT6._PLD due to previous error (AE_AML_PACKAGE_LIMIT) (20220331/psparse-529) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): AE_AML_PACKAGE_LIMIT, Index (0x000000007) is beyond end of object (length 0x6) (20220331/exoparg2-393) Sep 22 09:56:06 Tower kernel: ACPI Error: Aborting method \_SB.S0D2.D2A0.BYUP.BYD8.XHC1.RHUB.PRT6._PLD due to previous error (AE_AML_PACKAGE_LIMIT) (20220331/psparse-529) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): AE_AML_PACKAGE_LIMIT, Index (0x000000007) is beyond end of object (length 0x6) (20220331/exoparg2-393) Sep 22 09:56:06 Tower kernel: ACPI Error: Aborting method \_SB.S0D2.D2A0.BYUP.BYD8.XHC1.RHUB.PRT6._PLD due to previous error (AE_AML_PACKAGE_LIMIT) (20220331/psparse-529) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): AE_AML_PACKAGE_LIMIT, Index (0x000000007) is beyond end of object (length 0x6) (20220331/exoparg2-393) Sep 22 09:56:06 Tower kernel: ACPI Error: Aborting method \_SB.S0D2.D2A0.BYUP.BYD8.XHC1.RHUB.PRT6._PLD due to previous error (AE_AML_PACKAGE_LIMIT) (20220331/psparse-529) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): AE_AML_PACKAGE_LIMIT, Index (0x000000007) is beyond end of object (length 0x6) (20220331/exoparg2-393) Sep 22 09:56:06 Tower kernel: ACPI Error: Aborting method \_SB.S0D2.D2A0.BYUP.BYD8.XHC1.RHUB.PRT6._PLD due to previous error (AE_AML_PACKAGE_LIMIT) (20220331/psparse-529) Sep 22 09:56:06 Tower kernel: ACPI BIOS Error (bug): AE_AML_PACKAGE_LIMIT, Index (0x000000007) is beyond end of object (length 0x6) (20220331/exoparg2-393) Sep 22 09:56:06 Tower kernel: ACPI Error: Aborting method \_SB.S0D2.D2A0.BYUP.BYD8.XHC1.RHUB.PRT6._PLD due to previous error (AE_AML_PACKAGE_LIMIT) (20220331/psparse-529) Sep 22 09:56:20 Tower ntpd[2345]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized Sep 22 09:56:20 Tower ntpd[2345]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized

If i do the backup of pendrive it should be safe to to plug the resoration copy or there is stuff written to array/cache during the upgrade too?

I'm good at troubleshooting, but usually the jump between major releases may cause issues, hence I am preemptively asking. This is my main server on which VMs i do all my daily work, so it's kidda crital :-) but on the other hand i don't want to be left behind with updates to the point that i will need to do a reinstall or some such. Do you think it's better to update to 7.2, once it comes out ?

I must admit I have been slacking with upgrades, but that is because everything is working, and I am constantly using 3-5 VMs, which run 24/7 for various tasks. I know that the longer I wait, the more dangerous the upgrade is going to be, but how safe is it for me to do the upgrade now just by "clicking" the upgrade button? Do I need to know anything beforehand? I'm using multiple Docker images and 2 pass-through VMs from 2 graphics cards.

updated to latest unraid version, still only one VM is getting backed up from the list selected. Does anyone have this issue ?

Did u find any solution ?

I finally did a passthrough of two graphic cards I have GF2080Ti and P2000 to the VMs for streaming. I wanted have encoding capabilities. I'm wondering if it's possible to do something like: so ether of those cards I can pass to multiple VMs?

I bought P2000 and it finally worked on both cards.. for some reason G710 did not like to be passthrough

I have same problem and I noticed Sep 3 12:29:10 Tower kernel: pci 0000:01:00.0: vgaarb: setting as boot VGA device Sep 3 12:29:10 Tower kernel: pci 0000:01:00.0: vgaarb: bridge control possible Sep 3 12:29:10 Tower kernel: pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none Sep 3 12:29:10 Tower kernel: pci 0000:4d:00.0: vgaarb: bridge control possible Sep 3 12:29:10 Tower kernel: pci 0000:4d:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none Sep 3 12:29:10 Tower kernel: vgaarb: loaded I'm using GT710 in my slot

you you give me a hand on my issue ?

2080TI I was able to passthrough which is in 3rd slot. GT710 from primary slot I'm not able to passthrough.

so far I am able to passthrough that card, vm boots, but I'm getting Code 43 this is what I am using: kernel /bzimage append vfio-pci.ids=1022:148c,10de:1ad6,10de:1ad7 video=efifb:off isolcpus=2-11,26-35 initrd=/bzroot kvm_amd.nested=1 tower-diagnostics-20230904-1848.zip <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm' id='12'> <name>WinAP</name> <uuid>c52df59f-c118-5e6a-ba7a-97e4cf49c880</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='14'/> <vcpupin vcpu='1' cpuset='38'/> <vcpupin vcpu='2' cpuset='15'/> <vcpupin vcpu='3' cpuset='39'/> <vcpupin vcpu='4' cpuset='16'/> <vcpupin vcpu='5' cpuset='40'/> <vcpupin vcpu='6' cpuset='17'/> <vcpupin vcpu='7' cpuset='41'/> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-7.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/c52df59f-c118-5e6a-ba7a-97e4cf49c880_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv mode='custom'> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='4' threads='2'/> <cache mode='passthrough'/> <feature policy='require' name='topoext'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/vm_ssd/domains_ssd/WinAP_082023.img' index='2'/> <backingStore/> <target dev='hdc' bus='virtio'/> <serial>vdisk1</serial> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.190-1.iso' index='1'/> <backingStore/> <target dev='hdb' bus='ide'/> <readonly/> <alias name='ide0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='ide' index='0'> <alias name='ide'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <interface type='bridge'> <mac address='52:54:00:8f:32:e8'/> <source bridge='br0'/> <target dev='vnet11'/> <model type='virtio-net'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-12-WinAP/org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <alias name='input0'/> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'> <alias name='input1'/> </input> <input type='keyboard' bus='ps2'> <alias name='input2'/> </input> <audio id='1' type='none'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/> </hostdev> <memballoon model='none'/> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+100</label> <imagelabel>+0:+100</imagelabel> </seclabel> </domain>

Can I do same for primary card ? I have currently two cards in my server GT710 and 2080Ti. The 2080ti I'm able to passthrough using append vfio-pci.ids=1022:148c,10de:1ad6,10de:1ad7 video=efifb:off isolcpus=2-11,26-35 pic-ids are devicesID for 2080 but I am not able to passthrough GT710 ? Should I also list it within VFIO ?

Ok I was able to fix pausing I think .. it seems it was not related to graphic card.. but still .. Did everything like in this clip and did not help .. still getting error code 43 <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm' id='6'> <name>WinAP</name> <uuid>c52df59f-c118-5e6a-ba7a-97e4cf49c880</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='14'/> <vcpupin vcpu='1' cpuset='38'/> <vcpupin vcpu='2' cpuset='15'/> <vcpupin vcpu='3' cpuset='39'/> <vcpupin vcpu='4' cpuset='16'/> <vcpupin vcpu='5' cpuset='40'/> <vcpupin vcpu='6' cpuset='17'/> <vcpupin vcpu='7' cpuset='41'/> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-7.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/c52df59f-c118-5e6a-ba7a-97e4cf49c880_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv mode='custom'> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='4' threads='2'/> <cache mode='passthrough'/> <feature policy='require' name='topoext'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/vm_ssd/domains_ssd/WinAP_082023.img' index='2'/> <backingStore/> <target dev='hdc' bus='virtio'/> <serial>vdisk1</serial> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.190-1.iso' index='1'/> <backingStore/> <target dev='hdb' bus='ide'/> <readonly/> <alias name='ide0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='ide' index='0'> <alias name='ide'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <interface type='bridge'> <mac address='52:54:00:8f:32:e8'/> <source bridge='br0'/> <target dev='vnet5'/> <model type='virtio-net'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-6-WinAP/org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'> <alias name='input0'/> </input> <input type='keyboard' bus='ps2'> <alias name='input1'/> </input> <audio id='1' type='none'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> <rom file='/mnt/user/isos/MSI.GT710.2048.160112.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/> </hostdev> <memballoon model='none'/> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+100</label> <imagelabel>+0:+100</imagelabel> </seclabel> </domain>