June 10, 201610 yr I have a ASUS Z97-A motherboard with 3 PCIe ports. Two of them are PCIe 3 while the last one is PCIe 2. My issue is that when I have two GPU's connected to the two PCIe 3 port this message appears when trying to boot the second VM: However, everything works if one card is moved to the PCIe 2 slot. Unfortunately there isn't a lot of space in the case and I'm trying to figure out a way to make the second PCIe 3 port work. I've tried changing all the GPU-related settings in the BIOS without any success. Is this a known issue with a simple fix or do I have to move the GPU to PCIe 2 again? Edit: I assume that both PCIe slots being presented as "/sys/kernel/iommu_groups/1/" is the real issue here. Is there any setting to split them? I've heard "PCIe ACS Override" could, but with a performance penalty(?). System Devices says the following: 00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06) 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06) 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller [8086:0c05] (rev 06) 00:14.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB xHCI Controller [8086:8cb1] 00:16.0 Communication controller [0780]: Intel Corporation 9 Series Chipset Family ME Interface #1 [8086:8cba] 00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I218-V [8086:15a1] 00:1a.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2 [8086:8cad] 00:1b.0 Audio device [0403]: Intel Corporation 9 Series Chipset Family HD Audio Controller [8086:8ca0] 00:1c.0 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 [8086:8c90] (rev d0) 00:1c.3 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev d0) 00:1d.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1 [8086:8ca6] 00:1f.0 ISA bridge [0601]: Intel Corporation 9 Series Chipset Family Z97 LPC Controller [8086:8cc4] 00:1f.2 SATA controller [0106]: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode] [8086:8c82] 00:1f.3 SMBus [0c05]: Intel Corporation 9 Series Chipset Family SMBus Controller [8086:8ca2] 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Pitcairn XT [Radeon HD 7870 GHz Edition] [1002:6818] 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0] 02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper XT [Radeon HD 5770] [1002:68b8] 02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series] [1002:aa58] 04:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 04) /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.1 /sys/kernel/iommu_groups/1/devices/0000:01:00.0 /sys/kernel/iommu_groups/1/devices/0000:01:00.1 /sys/kernel/iommu_groups/1/devices/0000:02:00.0 /sys/kernel/iommu_groups/1/devices/0000:02:00.1 /sys/kernel/iommu_groups/2/devices/0000:00:14.0 /sys/kernel/iommu_groups/3/devices/0000:00:16.0 /sys/kernel/iommu_groups/4/devices/0000:00:19.0 /sys/kernel/iommu_groups/5/devices/0000:00:1a.0 /sys/kernel/iommu_groups/6/devices/0000:00:1b.0 /sys/kernel/iommu_groups/7/devices/0000:00:1c.0 /sys/kernel/iommu_groups/8/devices/0000:00:1c.3 /sys/kernel/iommu_groups/8/devices/0000:04:00.0 /sys/kernel/iommu_groups/9/devices/0000:00:1d.0 /sys/kernel/iommu_groups/10/devices/0000:00:1f.0 /sys/kernel/iommu_groups/10/devices/0000:00:1f.2 /sys/kernel/iommu_groups/10/devices/0000:00:1f.3
June 11, 201610 yr Enabling the ACS override will most likely fix your problem. There shouldn't be any performance issue enabling it.
June 11, 201610 yr Enabling the ACS override will most likely fix your problem. There shouldn't be any performance issue enabling it. Exactly as saarg says enable acs override in settings, vm manager. Dont forget to reboot server afterwards. Why do you need to do this is because both your gpus are in the same iommu group /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.1 /sys/kernel/iommu_groups/1/devices/0000:01:00.0 /sys/kernel/iommu_groups/1/devices/0000:01:00.1 /sys/kernel/iommu_groups/1/devices/0000:02:00.0 /sys/kernel/iommu_groups/1/devices/0000:02:00.1 Acs override should split the cards into their own iommu group
Archived
This topic is now archived and is closed to further replies.