(1) Please consider adding something like this for the default syslinux config:
label Unraid OS without GPUs
kernel /bzimage
append initrd=/bzroot earlymodules=vfio-pci video=efifb:off,vesafb:off gfxpayload=text
WARNING: This will mean you don't have a functioning video device for TTY/terminal access to the physical machine.
Why?
On my machine (specs later)...
the primary GPU cannot be selected in the BIOS settings; it is chosen by the proprietary MSI BIOS;
the primary GPU (being the one that Unraid itself boots to) cannot be used for VMs
vfio-pci BIND "succeeds"
launching the VM gets a black screen
after 2-3 launches, "qemu error.... var/log is full"
syslog almost fills the 128 MB RAM drive, qemu logs are indeed overfull
Dumping VBIOS and using a ROM file does not help
None of the various tips/tricks help
These options contribute to the fix:
earlymodules=vfio-pci
This might? load vfio-pci a bit earlier in the chain, helping it to grab the devices before anything else can
video=efifb:off,vesafb:off
This disables the framebuffer device drivers, which can hook into the GPU connected to the primary display
gfxpayload=text
I don't actually know, but it came with the others in the post I read (which I can't find). I suspect it tells the Linux kernel that the graphics subsystem will only take text, no real graphics, but when I find myself reading Linux kernel documentation, I get scared.
(2) GPU passthrough documentation
Please mention the above.
(3) VM options: add "kvm hidden mode"
This solution fixed my final issue (the "alternative for recent libvirt-qemu", the actual page is about patching the NVIDIA drivers to un-break them, please dear PTBs don't ever make me have to do that):
https://github.com/sk1080/nvidia-kvm-patcher#preferred-alternative-for-recent-libvirt--qemu
It would be really handy if the UNRAID GUI for VM creation had a checkbox for "kvm hidden mode"
<domain>
...
<features>
...
<kvm>
<hidden state='on'/>
</kvm>
...
<hyperv>
...
<vendor_id state='on' value='whatever'/>
</hyperv>
...
</features>
...
</domain>
Key system specs (I am happy to send a full diagnostic download directly but not post it):
MSI Z390 Gaming Pro Carbon AC
Intel i7-9700KF
GPUs, all BINDed to vfio-pci in WebGUI -> Tools -> System Devices
Gigabyte NVIDIA RTX 2070 Super
Gigabyte NVIDIA RTX 2070 Super (yes, another)
Quadro K600
1x SATA HDD (sole drive of the array)
2x NVMe, 2x SATA SSD (each in their own pool)
VMs:
GPU Passthrough (all cards working)
4-8 cores assigned
Hyper-V selected
Custom XML as above for kvm hidden