Multiple PCIE pass though on a Hasswell board possible


Recommended Posts

I have following groups on my board. I'm using only single card right now which i pass to my windows VM.

I want to add one more NVIDIA card and use it for my Windows VM, and the current Radeon RX for my MacOS.

 

I see that controller is passed in group 1 , does it mean that another card will be also is same IOMMU1 group ?

What are repercussions of having controller together with graphic card? 

 

IOMMU group 0:    [8086:0c00] 00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
IOMMU group 1:    [8086:0c01] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
[1002:67df] 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev ef)
[1002:aaf0] 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
IOMMU group 2:    [8086:0412] 00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
IOMMU group 3:    [8086:0c0c] 00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
IOMMU group 4:    [8086:8cb1] 00:14.0 USB controller: Intel Corporation 9 Series Chipset Family USB xHCI Controller
IOMMU group 5:    [8086:8cba] 00:16.0 Communication controller: Intel Corporation 9 Series Chipset Family ME Interface #1
IOMMU group 6:    [8086:15a1] 00:19.0 Ethernet controller: Intel Corporation Ethernet Connection (2) I218-V
IOMMU group 7:    [8086:8cad] 00:1a.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2
IOMMU group 8:    [8086:8ca0] 00:1b.0 Audio device: Intel Corporation 9 Series Chipset Family HD Audio Controller
IOMMU group 9:    [8086:8c90] 00:1c.0 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 (rev d0)
IOMMU group 10:    [8086:8c96] 00:1c.3 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 4 (rev d0)
IOMMU group 11:    [8086:8ca6] 00:1d.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1
IOMMU group 12:    [8086:8cc4] 00:1f.0 ISA bridge: Intel Corporation Z97 Chipset LPC Controller
[8086:8c82] 00:1f.2 SATA controller: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode]
[8086:8ca2] 00:1f.3 SMBus: Intel Corporation 9 Series Chipset Family SMBus Controller
IOMMU group 13:    [1b21:0612] 03:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)

 

Thanks a lot for help!

Edited by macmus
Link to comment

so you suggesting mouting a temporary graphic card in secondary slot and check IOMMU groups ?

1. It it possible that both cards will be in same IOMMU group ?

2.Do i need controller in both groups ?

3.Would I have controller in both groups ?

 

Does it mean i cannot use 2 cards .. with both passesthu to separate VMs?

Edited by macmus
Link to comment

If you have a spare PCIe card (does NOT have to be a GPU), add that to the other intended slot and check IOMMU groups of both cards. You asked if it's possible that both cards are in the same group. Of course it is possible.

 

Now if you do NOT have a spare PCIe card, move your current GPU to the other slot and check IOMMU group of the current card. If it's in a different group then chances are if you have 2 cards, they will be in different groups. It's not a guarantee but it gives you a better indication than guessing.

 

Ignore the PCIe controller device.

 

13 hours ago, macmus said:

Does it mean i cannot use 2 cards .. with both passesthu to separate VMs?

If you want a simple yes or no answer, you won't get it unless you can find another user with the exact same config as yours.

  1. There is NO guarantee that you will be able to pass through both cards to separate VMs even if they are in their own IOMMU groups.
    e.g. instability, can't reset the GPU (so can't restart the VM) etc
  2. Having both cards in the same IOMMU group also doesn't mean it's impossible to pass them through to separate VMs.

There are workarounds if they are in the same IOMMU group (or in the same IOMMU groups with other devices) i.e. stubbing the devices in the same group. However,

  • In some cases that's impossible e.g. the device in the same group cannot be stubbed (e.g. the USB controller where the Unraid USB stick is plugged into, LAN port, etc.).
  • In some cases even after stubbing, the emulator still refuses to let you pass the device through.

 

 

Link to comment


IOMMU group 1:    


[8086:0c01] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
[8086:0c05] 00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)
[1002:67df] 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev ef)
[1002:aaf0] 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
[1b21:0612] 02:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)

Slot3:

IOMMU group 0:    [8086:0c00] 00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
IOMMU group 1:    [8086:0c01] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
[1002:67df] 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev ef)
[1002:aaf0] 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
IOMMU group 2:    [8086:0412] 00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
IOMMU group 3:    [8086:0c0c] 00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
IOMMU group 4:    [8086:8cb1] 00:14.0 USB controller: Intel Corporation 9 Series Chipset Family USB xHCI Controller
IOMMU group 5:    [8086:8cba] 00:16.0 Communication controller: Intel Corporation 9 Series Chipset Family ME Interface #1
IOMMU group 6:    [8086:15a1] 00:19.0 Ethernet controller: Intel Corporation Ethernet Connection (2) I218-V
IOMMU group 7:    [8086:8cad] 00:1a.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2
IOMMU group 8:    [8086:8ca0] 00:1b.0 Audio device: Intel Corporation 9 Series Chipset Family HD Audio Controller
IOMMU group 9:    [8086:8c90] 00:1c.0 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 (rev d0)
IOMMU group 10:    [8086:8c96] 00:1c.3 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 4 (rev d0)
IOMMU group 11:    [8086:8c9e] 00:1c.7 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 8 (rev d0)
IOMMU group 12:    [8086:8ca6] 00:1d.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1
IOMMU group 13:    [8086:8cc4] 00:1f.0 ISA bridge: Intel Corporation Z97 Chipset LPC Controller
[8086:8c82] 00:1f.2 SATA controller: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode]
[8086:8ca2] 00:1f.3 SMBus: Intel Corporation 9 Series Chipset Family SMBus Controller
IOMMU group 14:    [1b21:0612] 03:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
IOMMU group 15:    [1b21:0612] 04:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)


IOMMU group 0:    [8086:0c00] 00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
IOMMU group 1:    [8086:0c01] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
[1002:67df] 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev ef)
[1002:aaf0] 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
IOMMU group 2:    [8086:0412] 00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
IOMMU group 3:    [8086:0c0c] 00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
IOMMU group 4:    [8086:8cb1] 00:14.0 USB controller: Intel Corporation 9 Series Chipset Family USB xHCI Controller
IOMMU group 5:    [8086:8cba] 00:16.0 Communication controller: Intel Corporation 9 Series Chipset Family ME Interface #1
IOMMU group 6:    [8086:15a1] 00:19.0 Ethernet controller: Intel Corporation Ethernet Connection (2) I218-V
IOMMU group 7:    [8086:8cad] 00:1a.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2
IOMMU group 8:    [8086:8ca0] 00:1b.0 Audio device: Intel Corporation 9 Series Chipset Family HD Audio Controller
IOMMU group 9:    [8086:8c90] 00:1c.0 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 (rev d0)
IOMMU group 10:    [8086:8c96] 00:1c.3 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 4 (rev d0)
IOMMU group 11:    [8086:8c98] 00:1c.4 PCI bridge: Intel Corporation 9 Series Chipset Family PCI Express Root Port 5 (rev d0)
IOMMU group 12:    [8086:8ca6] 00:1d.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1
IOMMU group 13:    [8086:8cc4] 00:1f.0 ISA bridge: Intel Corporation Z97 Chipset LPC Controller
[8086:8c82] 00:1f.2 SATA controller: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode]
[8086:8ca2] 00:1f.3 SMBus: Intel Corporation 9 Series Chipset Family SMBus Controller
IOMMU group 14:    [1b21:0612] 03:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
IOMMU group 15:    [1b21:0612] 04:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.