Hi All,

 

Been searching all morning for an answer with not much luck. I HAD AMD and Nvidia GPU for my kids to play on 2 gaming VM's. All was good except for AMD reset bug so I got another Nvidia GPU... figured out that now my 2 Nvidia GPU's the device ID are identical (2 different vendors). Only 1 of 2 VM's will boot... How can I break these up so each VM has their own video card and boot? 

 

Thanks for any and all assistance.

Jason.

Edited July 28, 20214 yr by JRL70

Try enabling "PCIe ACS override" in the advanced VM settings and have another look at the IOMMU groups and if they've split up.

Thank you, yes they are split up. I've added them as Q35-5.1 OVMF or SEABIOS - 1 VM starts (28:00.0) and the other fails (27:00.0). Ironically, the one that starts I don't need to add GPU ROM to... the other I just tried this morning. Still no luck. I'm pretty sure I'm overlooking something pretty simple.

 

edit ** I have the GPU in the first slot of 3 PCI-E slots. (I had the AMD card there and it was fine) I'm wondering if it should be on slot 3?

 

 

 

Happy to do some research if someone can direct me to it. Thank you very much for your help.

 

Edited July 29, 20214 yr by JRL70

So I have been working on this problem for hours.

 

I have an Asus Crosshair Hero VII (wifi) with a 2700x and bios #2008. I ran two 1070ti and used 2 windows 10 vms for over 2 years daily (2 gamers 1 cpu)

Well I just got a new cpu, 3950x, and I had to update my bios, so I now have 4603.

Now both GPU have identical pcie ID names and virtually everything I try ends up crashing unRAID, I have even corrupted my usb a couple times. They are in different IOMMU.

I have been able to get ONE of the cards to pass through to a VM, but if I try to start the second one the system hard crashes.

 

I will continue searching for a way to get my daily system up and running, I really hate to have to roll back bios and go back to an 8 core cpu.

Hard freeze when trying to click the button to pass through the gpus and restart... Have to pull the USB drive again and delete /config/vfio-pci.cfg just to access the server again.

28 minutes ago, tsawind said:

Hard freeze

Hard freeze?It's not...Don't expect to read anything on the monitor after that, because you are attaching to vfio the gpu to which the monitor is connected!

In this situation the server is accessible from external devices, via lan.

 

The issue of identical vendor/product ids is a non issue in recent version of unraid because it takes into account both vendor/product id AND physical address bus:slot.function (which is unique)

Edited October 27, 20214 yr by ghost82

@tsawind  Personally I gave up on my quest and went back to an AMD card with my Nvidia card - the lack of help on this forum is pretty horrible.  As you can see I tried to get help but then my thread went cold... I'm pretty technical and spend most of my time researching that's why my post count is so low. But I couldn't figure my issue out.

 

 

4 hours ago, ghost82 said:

Hard freeze?It's not...Don't expect to read anything on the monitor after that, because you are attaching to vfio the gpu to which the monitor is connected!

In this situation the server is accessible from external devices, via lan.

 

The issue of identical vendor/product ids is a non issue in recent version of unraid because it takes into account both vendor/product id AND physical address bus:slot.function (which is unique)

Correct, hard freeze. I understand that the gpu that unraid is showing up on is supposed to be "handed off" to the VM. unRAID becomes inaccessible from the network as well, so yes a hard freeze. 

45 minutes ago, tsawind said:

so yes a hard freeze

and what does the syslog say about the freeze?

Okay well after a couple of hours of sleep, I have made some progress.  I enabled syslog to save to flash, and I believe what has been happening is that when the old VM would start up it was using probably old usb pass-through and other wrong things in the xml. I have the second gpu VM operating, currently working to get a display out of the main (1st slot) gpu.

 

Boom, Success!!! A New VM and using a custom hacked Graphics ROM Bios. ROM file for the GPU, now the "handoff" is working!

3 hours ago, JRL70 said:

@tsawind  Personally I gave up on my quest and went back to an AMD card with my Nvidia card - the lack of help on this forum is pretty horrible.  As you can see I tried to get help but then my thread went cold... I'm pretty technical and spend most of my time researching that's why my post count is so low. But I couldn't figure my issue out.

 

 

 

 

This is a link to a space invader video talking about your original problem.

From what I understand, NVIDIA has something in their bios to make their gaming video cards become resistant to virtualization or something. When it is the first slot, it need a custom virtual BIOS .ROM file which fixes this or something. I hope I solved your original issue with this response, maybe someone else will read this and give a better explanation!

28 minutes ago, tsawind said:

When it is the first slot, it need a custom virtual BIOS .ROM file which fixes this or something

nvidia roms downloaded from techpowerup, as an example, may have an header which is used when you flash the rom to a bios chip. Kvm is not compatible with this, so we need to remove it, simply with a hex editor.

However dumping our own vbios is a lot better and it doesn't contain any header to be removed.