GPU Passthrough has not worked for me since 6.8.3

[Tonitram]

Hi all,  Im looking for some help with my VMs.  

I recently upgraded from 6.8.3 to 6.12.3.  

Everything went smoothly except for my VMs.   The reason I stayed on 6.8.3 for so long is because it was the last release that VMs worked for me.  I tried 6.9.x, 6.10.x, 6.11.x  and always have issues with VMs in each of those releases.  Either they wont start, GPU passthrough didnt work, or I cant create new ones etc.  I finally went to 6.12.3 because none of my plugins or dockers would upgrade any longer on 6.8.3.  I had 2 Win10 VMs and 2 Linux VMs (1 arch based and 1 debian based distros).  After the update to 6.12.3  I only have 1 Linux VM still working with GPU passthrough.  The other linux VM and both the Win10 VMs passthrough will not work.  I have tried to create new VMs (linux and windows)  but cant get GPU passthrough to work properly. On new linux VMs the best I can get is using the non proprietary drivers but this has poor performance and if I update the driver I only get a black screen.  On new Win 10 VMs I cant get the VM to properly recognize the GPU and I always end up with the code 43 error which I cant get rid of.   I have been trying various things but nothing ever works.   I tried...

- OVMF, seabios
- every machine type from i440fx 3.1 - 7.1 and Q35 4.0 - 7.1
- tried hyper-V on/off
- in XML I tried adding
</hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
but all this did was make the GPU fans spin at maximum for 30-60 secs and then make the VM bootloop
- tried various virtio driver ISOs
- tried with/without stubbing the GPUs
- I have 2 PCIE USB controllers that I had to stub to pass through and both work
- passed the GPU vbios through and added multifunction line (same vbios Ive used for 6+ yrs) But I tried another and no change
- I have Unraid set to boot in legacy and In my motherboard bios I set it to not use UEFI and to use legacy only. also set the unraid USB to only boot as legacy
and the primary graphics is set to use the onboard vga.
- The only way I can create new VMs (win or linux) now is to start with using VNC to do the install, then switch it to use the GPU.  I will get a picture but its low res and I get the code 43 error.
- tried installing many different nvidia drivers from 465.x - 470.x   No change. Code 43 never goes away in windows and in linux I just get a black screen or it wont even boot.

Ive tried so many things that I cant remember them all right now.  Hopefully someone can help.  I really need to get my VMs working like they did in Unraid 6.8.3  I dont mind if I have to delete my old VMs (except for the 1 that still works) since I have all my important stuff backed up already.

 

My motherboard is a ASRock Rack EP2C602-4L/D16 (bios is up to date)
2 Xeon E5-2665 CPUs
128GB ECC Ram
GPUs are GTX660ti and GT710

I know all my equipment is older but they worked fine with VMs for 6+ yrs and didnt have any issues until I tried upgrading past 6.8.3.  I did connect both GPUs into different PCs just to test and they both still work fine on both windows 10 and linux computers.   The only issue is in VMs.  

I will also include my diagnostics which include all the XMLs of my VMs. The only VM that is currently working with GPU passthrough is the Garuda - KDE VM.

 

Thank you in advance to anyone willing to help, I really do appreciate it!

 

 

genisys-diagnostics-20230821-1750.zip

[Tonitram]

Anyone have any suggestions?   I hate to think that my only options are to either stay on 6.12.3 with broken VM's, or revert to 6.8.3 and have an outdated system with features/docker/security issues.

 

I could really use the help if anyone has any suggestions.  Thanks.

[Tonitram]

Update...

I took my GPUs to a friend who has a proxmox server and he was able to add them to both a Linux and a Win 10 VM.  GPU passthrough worked on both using the latest nvidia drivers.   So  its not an issue with the GPU (which I figured since it was working in 6.8.3).   What changed in Unraid to make GPU passthrough stop working?  Am I missing a setting somewhere?