Unraid 6.6 iommu groups are different, and not viable

[Dstubbs28]

Upgraded to 6.6, I can no longer pass through my USB controller to my VM (which means no hotplugging) as the IOMMU group now stays locked no matter what ACS patch override mode I chose. My NVME drive is now in the same group as my PCIE usb card (1106:3483), which is a huge problem.

 

I had an older bios before the upgrade, updated the bios once I saw the iommu groups are completely borked, no change.

 

Anyone have any ideas how to remedy this?

 

# for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d);do echo "IOMMU group $(basename "$iommu_group")"; for device in $(\ls -1 "$iommu_group"/devices/); do if [[ -e "$iommu_group"/devices/"$device"/reset ]]; then echo -n "[RESET]"; fi; echo -n $'\t';lspci -nns "$device"; done; done
IOMMU group 7
[RESET] 00:1b.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #19 [8086:a342] (rev f0)
[RESET] 00:1b.3 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #20 [8086:a343] (rev f0)
[RESET] 00:1b.4 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #21 [8086:a32c] (rev f0)
[RESET] 02:00.0 USB controller [0c03]: VIA Technologies, Inc. VL805 USB 3.0 Host Controller [1106:3483] (rev 01)
[RESET] 03:00.0 PCI bridge [0604]: Integrated Technology Express, Inc. IT8892E PCIe to PCI Bridge [1283:8892] (rev 71)
[RESET] 05:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961 [144d:a804]
IOMMU group 5
        00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
IOMMU group 3
        00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
IOMMU group 1
        00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
[RESET] 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X] [1002:67df] (rev e7)
        01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] [1002:aaf0]
IOMMU group 8
[RESET] 00:1c.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #5 [8086:a33c] (rev f0)
[RESET] 06:00.0 Ethernet controller [0200]: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) [8086:105e] (rev 06)
[RESET] 06:00.1 Ethernet controller [0200]: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications) [8086:105e] (rev 06)
IOMMU group 6
        00:17.0 SATA controller [0106]: Intel Corporation Cannon Lake PCH SATA AHCI Controller [8086:a352] (rev 10)
IOMMU group 4
        00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
        00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
IOMMU group 2
[RESET] 00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics 630 (Desktop) [8086:3e92]
IOMMU group 10
        00:1f.0 ISA bridge [0601]: Intel Corporation H370 Chipset LPC/eSPI Controller [8086:a304] (rev 10)
        00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
        00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
        00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
        00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-V [8086:15bc] (rev 10)
IOMMU group 0
        00:00.0 Host bridge [0600]: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers [8086:3ec2] (rev 07)
IOMMU group 9
[RESET] 00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 [8086:a330] (rev f0)
[RESET] 07:00.0 Non-Volatile memory controller [0108]: Intel Corporation SSD 600P Series [8086:f1a5] (rev 03)

 

 

dmserver-diagnostics-20180925-2237.zip

Edited September 26, 2018 by Dstubbs28

[Dstubbs28]

anyone?

[JorgeB]

Not much else you can do, downgrade or wait for a new kernel/bios that might fix the issue, best long term solution would be to use hardware that always separates the IOMMU groups without the need for the ACS override patch.

[Dstubbs28]

14 minutes ago, johnnie.black said:

Not much else you can do, downgrade or wait for a new kernel/bios that might fix the issue, best long term solution would be to use hardware that always separates the IOMMU groups without the need for the ACS override patch.

Well thats part of the issue, with ACS on, off, multipurpose or downstream, the iommu groups remain the same.

[JorgeB]

Yes, that's why if you to make sure that doesn't happen buy hardware that always separates groups, e.g. Xeon E5 based motherboards.

[Dstubbs28]

11 hours ago, johnnie.black said:

Yes, that's why if you to make sure that doesn't happen buy hardware that always separates groups, e.g. Xeon E5 based motherboards.

My hardware worked fine before 6.6.  Iommu groups were separated properly. Thats like someone telling a guy with a flat tire fresh out of the tire shop that he should of bought a better car

[JorgeB]

2 minutes ago, Dstubbs28 said:

My hardware worked fine before 6.6.  Iommu groups were separated properly.

It might have worked, it might work again on v6.7, or v6.8, but you're always dependent on the kernel and/or bios used, and if it doesn't work there's not much you can do about it, you're just not using the right hardware for dependable hardware pass-trough.

[pederm]

I faced a similar problem on my Asus x370 board after a BIOS update. According to Spaceinvader Ones excellent video, please ensure that the IOMMU setting in BIOS is changed from Automatic to Enabled. After that when setting pcie_acs_override,  the hardware passed through as before.