I am still not sure what is the core issue here, but I have a workaround (not a complete fix though). After upgrade from 6.9.2 to 6.11.5 all of my VMs that had RX550 passed through got:
- black screen
- 1 core stuck at 100%
and nothing was happening.
To have them running again I did the following things:
1. Backup both VM disks and XML files.
2. Deleted VMs and created them again with only one GPU - virtual. I use SeaBIOS in all of them and I switched to Q35 (7.1) for both my Ubuntu and W10 based VMs (I had issues with i440fx).
3. Added the same VM disks to each VM.
4. Run them and via VNC, I deleted all the GPU drivers (AMD Cleanup Utility for Windows and followed removal procedure for Ubuntu here).
5. Reboot and then shutdown.
6. Added RX550 as a second GPU (+audio) and started VM.
7. Installed the newest drivers from the AMD website (followed by step by step installation instructions for Ubuntu). At some point (both W10 and Ubuntu) of driver installation I could see a second display via RX550 GPU - that was really good Reboot.
8. Set the monitor as my main display in display settings and virtual 'monitor' as a secondary and I set the displays layout in a way that the virtual one is in the top-right corner of the main one.
9. Reboot.
10. Backup
After that I always have GPU passthrough running perfectly. This is not ideal, since the second 'screen' is still there are using some of the resources, but I never got it working when using only one GPU (external, passthrough), without the virtual one. When I did that I always ended up with black screen again and when doing Force Shutdown of the VM, something was breaking my VM and I ended up with 'No Bootable Disk' text when starting VM again (I needed to create a fresh VM with the same disks). As I said, this is not ideal, but hopefully would allow some of you (and me) to stay with the latest Unraid version and still have their GPU passthrough in VMs without getting black screen.