2 AMD GPU Passthrough - Issue with 2nd Card

[jhendrickson818]

I feel like I've been beating this problem with a rock for 2 weeks and just have made no progress on a solution....

 

Hardware:

• MB: Asus Prime H670-Plus D4
• Intel i5 12400
• GPU 1: RX 6600 (in PCIE Gen 4x16 Slot 1 (2nd slot on MB)
• GPU 2: RX 480 (in PCIE Gen 3x16 Slot (4th slot on MB) Note: The 2nd Gen4x16 slot is inaccessible as it's on the bottom of the MB and this is a 2 slot card.
• Random USB 3 controller card in PCIE slot 1

 

Primary VM is Windows 10 and has GPU1 vfio bound. Also passing through a bound nvme drive as the boot drive and the USB controller card. Everything works fine here. 

 

I'm trying to pass the 2nd GPU through to another VM (a second Window 10 install). The drive is vdisk on my cache drive.

 

- If I bind the GPU and pass it through to the VM it locks up Unraid completely after a minute or two. I've got my log going to my flash drive, but nothing is produced when this happens. Adding the vbios (dumped or the one I had previously used from TechPU when this was my primary GPU) doesn't change anything.

- If I don't bind the GPU and still pass it through I don't typically get the server lock up, but I cannot get the GPU to function properly. With or without vbios added. I've run into a multitude of errors in the vm logs and the Unraid sys logs, as well as multiple codes within windows (Code 31, Code 32, Code 34). 

- I've attempted both the SpaceInvaderOne AMD reset script as well as the AMD vendor reset plugin - neither solve any issues whether the GPU is correctly vfio binded or not. 

 

Other notes:

- When the 2nd GPU is not vfio-binded, the monitor that is connected to is does display the Unraid boot log until I start the VM (booting without GUI, using web gui)

- Both primary and 2nd vm have xml edited to fix the bus mapping on the GPU and add multifunction. 

- I'm not great at looking through all these logs. Any help is appreciated! Currently the 2nd GPU is not vfio binded. 

 

tower-diagnostics-20221125-1822.zip

Edited November 26, 2022 by jhendrickson818
Fixed uploads - uploaded .zip

[dirkinthedark]

Ive had some issues as well.  I would try dumping your video card bios into a file then using that.  

[jhendrickson818]

On 12/8/2022 at 7:52 PM, dirkinthedark said:

Ive had some issues as well.  I would try dumping your video card bios into a file then using that.  

I’ve tried various combinations with the bios added and without. Both GPUs were pretty easy when it came to dumping their bios luckily. 
 

I’m wondering if it could be a setting in my bios. Or that I’m booting UEFI and not legacy. 

[dirkinthedark]

Good luck its tricky