Hey all wanted to chime in with my experience...
I had several issues over the years with GPU passthrough, eventually I lately settled on having working 3060Ti in 2nd slot (x8) PCI-E, which works flawlessly.
However, my main goal was to allow two machines running at once, yet the first GPU, whatever I used, was having either reset bug, or Error 43...
Now I've tested just today, and my VM resets and shutsdown-powers up flawlessly - using RX 6800.
Here are all the settings used, hope I remember all:
HW:
R9-5900X
AsRock Phantom Gaming 4S (X570 mobo)
64GB RAM (4 sticks)
UEFI:
Enabled 4G, Resize-Bar
Booting from UEFI unRaid on USB stick
Latest AsRock bios (as of 09/2023 or so).
IOMMU, HVM etc on ofc
unRaid:
Version: 6.12.3
Enabled ACS Override = "both"
VFIO allow unsafe interrupts: = "yes"
Syslinux config:
kernel /bzimage
append video=efifb:off isolcpus=3-11,15-23 pcie_acs_override=downstream,multifunction vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot
Ofc make sure to bind EVERYTHING from the GPU to VFIO - RX 6800 has 4 devices on it: GPU, Sound, USB controller + USB serial bus
(IDs 0d:00:0 -> 0d:00:3 for me...)
VM config:
Win10 - I actually didn't install "from scratch", I had an old image lying around with Win10 working from testing the RTX 3080 in 1st PCI-E slot - so I cannot say if this helped in any way...
Q35v7.1
Bios: OVMF
Hyper-V: YES
USB: 3.0 (qemu XHCI)
Make sure ALL FOUR devices are passed through to VM
I did NOT do any manual edits, I wanna avoid them as much as possible so...
To install the driver, I also used the VNC console, which I've now removed after installing Barrier to the VM.
With this, I'm able to restart and/or shutdown and bring the VM back up, no issues.
Haven't done performance testing yet, I'll compare the two VMs to see if there is any considerable difference (the 2nd VM has the 3060ti in 2nd PCIE slot).