GPU passthrough in the 1st PCI-E slot


Bozo

Recommended Posts

Hello guys, I'm completely new to unraid and I wanted to host 2 gaming VMs for lan parties, so that there's one tower less to bring.

 

In the end I got it to work following those excellent guides from Spaceinvader, but funnily enough - my GPUs have to be in the 2nd and 3rd PCI-E slots (I set the iGPU as primary for unraid) - When any of the GPUs is in the first slot, that VMs graphics start lagging horribly after installing the NVIDIA driver (with just the basic windows driver it runs smoothly, however I cannot run any games like that).

 

If the GPUs are in the 2nd and 3rd PCI-E slots it runs fine with no problems, ran games like this already. It doesn't matter which of the 2 cards is in which slot, switched them around with exactly the same results - the VM with the GPU in the first PCI-E slot lags horribly, no matter which GPU it is.

 

The problem is, in my computer case the PSU interferes with the GPU in the 3rd slot, so I could only test it with the PSU laying around loosely, which is not acceptable for lan parties.

 

Is there any way to make it run fine in the 1st slot? I'll post any configuration files that would be needed, I just don't know what to post.

 

My setup is:

 

Aorus z390 PRO WIFI

i9 9900k

2x8 3333mhz hyperx predator

ASUS GTX 980 TI STRIX OC

Palit GTX 650 (just for testing, it'll probably be a friends' 1060 for the LAN parties)

 

I sincerely thank in advance for any help or responses :)

Link to comment

Few things to try.

 

Can you try running a vm with just one GPU only in the system the 980ti. Have that GPU in slot one and see if that way it runs ok.

Here is the vbios for your card that downloaded from here. https://www.techpowerup.com/vgabios/173726/asus-gtx980ti-6144-150618

i have already hex edited it. Please try passing this with it.   Asus.GTX980Ti.6144.150618.rom

 

The gtx 650 the card that you have may or may not support uefi. If you can only start machine with seabios for that card ,then the gpu doesnt support uefi.

You can fix that If your card is the 2gig version. i found a uefi vbios herehttps://www.techpowerup.com/vgabios/157807/palit-gtx650-2048-131018 and also have hex edited this one too.  With this passed through (if yours is 2 gig version and not 1gig ) it should allow you to use ovmf bios rather than seabios. Palit.GTX650.2048.131018.rom 

 

Also what does your iommu groups look like?

 

Lastly I have found issues like this can be bios related in the mb. You could try updating the bios ( or even sometimes downgrading the bios) and it may help.

Link to comment

Warm thanks for the reply.

 

So far, both VMs were running without vbioses - Unraid says it's optional, so i tried running the VMs without vbioses and it worked.

The VM with the GTX 980 ti is set up with OVFM.

The VM with the GTX 650 is indeed set up with seabios, as it's the 1gig version - it runs fine as long as the 650 isn't in the top PCI-E slot.

 

Following Your advice I tried running the first VM with only the 980 ti

 

  • 980 ti as the only discrete GPU (igpu for unraid)
    • Without vbios
      • Interestingly I had to reinstall the GPU driver (system acted as if a new GPU was installed), but after the installation exactly the same problem appears - The system is extremely laggy and unresponsive.
    • With provided vbios (selected in the dropdown in the vm config, i'm not sure if that's the proper way to do it)
      • No display
  • 980 ti as the only GPU (igpu disabled)
    • Without vbios
      • No display
    • With provided vbios (selected in the dropdown in the vm config, i'm not sure if that's the proper way to do it)
      • No display

Initially both GPUs were in the same IOMMU group, I selected the "Downstream" setting in PCIe ACS override dropdown and added "pcie_acs_override=id:8086:1905" to the configuration.

 

My IOMMU groups with both GPUs installed and igpu set as primary (980 ti in the top slot, VM laggy; 650 in the 2nd slot, VM working fine)

 

Quote

IOMMU group 0:[8086:3e30] 00:00.0 Host bridge: Intel Corporation 8th Gen Core 8-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S](rev 0a)

IOMMU group 1:[8086:1901] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) (rev 0a)

IOMMU group 2:[8086:1905] 00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) (rev 0a)

IOMMU group 3:[8086:3e98] 00:02.0 VGA compatible controller: Intel Corporation UHD Graphics 630 (Desktop 9 Series)

IOMMU group 4:[8086:a379] 00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10)

IOMMU group 5:[8086:a36d] 00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10)

                        [8086:a36f] 00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)

                        [8086:a370] 00:14.3 Network controller: Intel Corporation Wireless-AC 9560 [Jefferson Peak] (rev 10)

IOMMU group 6:[8086:a360] 00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10)

IOMMU group 7:[8086:a352] 00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)

IOMMU group 8:[8086:a340] 00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0)

IOMMU group 9:[8086:a338] 00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0)

IOMMU group 10:[8086:a330] 00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0)

IOMMU group 11:[8086:a305] 00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev 10)

                        [8086:a348] 00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)

                        [8086:a323] 00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10)

                        [8086:a324] 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller (rev 10)

                        [8086:15bc] 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-V (rev 10)

IOMMU group 12:[10de:17c8] 01:00.0 VGA compatible controller: NVIDIA Corporation GM200 [GeForce GTX 980 Ti] (rev a1)

                        [10de:0fb0] 01:00.1 Audio device: NVIDIA Corporation GM200 High Definition Audio (rev a1)

IOMMU group 13:[10de:0fc6] 02:00.0 VGA compatible controller: NVIDIA Corporation GK107 [GeForce GTX 650] (rev a1)

                        [10de:0e1b] 02:00.1 Audio device: NVIDIA Corporation GK107 HDMI Audio Controller (rev a1)

IOMMU group 14:[144d:a808] 03:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981

 

Sorry for the late reply, it took me quite a bit of time to configure and test all options as stated above.

 

 

Edit: Tried updating the bios, but it seems to have failed - after the update the PC restarted 2 times and then when I checked the bios version it still says F8 (I was trying to update to F10), so i think one of the bios chips (MB has dual bios) might be cooked now and I'm afraid to try that again.

Edited by Bozo
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.