derstef Posted October 31, 2019 Share Posted October 31, 2019 (edited) First of all, thank you for sharing the information above. I've running F12i Bios with the same board and my non boot NVME drives arent in any IOMMU Groups at all. (im using all 3 M.2 slots) They show up with lspci but not using the script here: https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Ensuring_that_the_groups_are_valid When i try to start my VM i get "Fehler: Interner Fehler: Process exited prior to exec: libvirt: Fehler : Interner Fehler: Ungültige Einheit 0000:42:00.0 iommu_group Datei /sys/bus/pci/devices/0000:42:00.0/iommu_group ist kein symlink" (sorry my system is localized to german) Have you encountered such situation too? Edited October 31, 2019 by derstef Quote Link to comment
testdasi Posted October 31, 2019 Share Posted October 31, 2019 (edited) 22 minutes ago, derstef said: First of all, thank you for sharing the information above. I've running F12i Bios with the same board and my non boot NVME drives arent in any IOMMU Groups at all. (im using all 3 M.2 slots) They show up with lspci but not using the script here: https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Ensuring_that_the_groups_are_valid When i try to start my VM i get "Fehler: Interner Fehler: Process exited prior to exec: libvirt: Fehler : Interner Fehler: Ungültige Einheit 0000:42:00.0 iommu_group Datei /sys/bus/pci/devices/0000:42:00.0/iommu_group ist kein symlink" (sorry my system is localized to german) Have you encountered such situation too? Tools -> System Devices -> wait a few seconds -> copy-paste what you see in the "PCI Devices and IOMMU Groups" section here. (alternatively Tools -> Diagnostics and attach zip file but that's TMI for what you are asking). Edited October 31, 2019 by testdasi Quote Link to comment
derstef Posted October 31, 2019 Share Posted October 31, 2019 1 hour ago, testdasi said: Tools -> System Devices -> wait a few seconds -> copy-paste what you see in the "PCI Devices and IOMMU Groups" section here. (alternatively Tools -> Diagnostics and attach zip file but that's TMI for what you are asking). Sorry, i am not using unraid. Im using Fedora 31 with ACS patch. Hope thats fine in this forum. Does the distro flavor have any influence to the IOMMU grouping? Here the lspci -kk lspci -kk 00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Root Complex Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Root Complex 00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) I/O Memory Management Unit Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) I/O Memory Management Unit 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge Kernel driver in use: pcieport 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge Kernel driver in use: pcieport 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge Kernel driver in use: pcieport 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B Kernel driver in use: pcieport 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B Kernel driver in use: pcieport 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) Subsystem: Gigabyte Technology Co., Ltd Device 5001 Kernel modules: i2c_piix4, sp5100_tco 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) Subsystem: Gigabyte Technology Co., Ltd Device 5001 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 Kernel driver in use: k10temp Kernel modules: k10temp 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 Kernel driver in use: k10temp Kernel modules: k10temp 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 00:19.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 00:19.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller (rev 02) Subsystem: ASMedia Technology Inc. Device 1142 Kernel driver in use: xhci_hcd 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller (rev 02) DeviceName: Promontory SATA Subsystem: ASMedia Technology Inc. Device 1062 Kernel driver in use: ahci 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge (rev 02) Kernel driver in use: pcieport 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) DeviceName: Onboard LAN Atheros Kernel driver in use: pcieport 02:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) Kernel driver in use: pcieport 02:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) Kernel driver in use: pcieport 02:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) Kernel driver in use: pcieport 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) Kernel driver in use: pcieport 04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) Subsystem: Gigabyte Technology Co., Ltd Device e000 Kernel driver in use: igb Kernel modules: igb 05:00.0 Network controller: Intel Corporation Wireless 8265 / 8275 (rev 78) DeviceName: Onboard LAN Realtek Subsystem: Intel Corporation Dual Band Wireless-AC 8265 Kernel driver in use: iwlwifi Kernel modules: iwlwifi 06:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) Subsystem: Gigabyte Technology Co., Ltd Device e000 Kernel driver in use: igb Kernel modules: igb 08:00.0 Non-Volatile memory controller: Silicon Motion, Inc. Device 2260 (rev 03) Subsystem: Silicon Motion, Inc. Device 2260 Kernel driver in use: nvme Kernel modules: nvme 09:00.0 VGA compatible controller: NVIDIA Corporation TU104 [GeForce RTX 2080] (rev a1) Subsystem: Micro-Star International Co., Ltd. [MSI] Device 3723 Kernel driver in use: vfio-pci Kernel modules: nouveau 09:00.1 Audio device: NVIDIA Corporation Device 10f8 (rev a1) Subsystem: Micro-Star International Co., Ltd. [MSI] Device 3723 Kernel driver in use: vfio-pci Kernel modules: snd_hda_intel 09:00.2 USB controller: NVIDIA Corporation Device 1ad8 (rev a1) Subsystem: Micro-Star International Co., Ltd. [MSI] Device 3723 Kernel driver in use: vfio-pci 09:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1ad9 (rev a1) Subsystem: Micro-Star International Co., Ltd. [MSI] Device 3723 Kernel driver in use: vfio-pci Kernel modules: i2c_nvidia_gpu 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function Subsystem: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function 0a:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor Kernel driver in use: ccp Kernel modules: ccp 0a:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller Subsystem: Gigabyte Technology Co., Ltd Device 5007 Kernel driver in use: xhci_hcd 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function Subsystem: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function 0b:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) DeviceName: DIE0 M.2 SATA Subsystem: Gigabyte Technology Co., Ltd Device b002 Kernel driver in use: ahci 0b:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller DeviceName: Audio Codec ALC1220 Subsystem: Gigabyte Technology Co., Ltd Device a0c2 Kernel driver in use: snd_hda_intel Kernel modules: snd_hda_intel 40:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Root Complex Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Root Complex 40:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) I/O Memory Management Unit Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) I/O Memory Management Unit 40:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 40:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge Kernel driver in use: pcieport 40:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge Kernel driver in use: pcieport 40:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 40:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 40:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge Kernel driver in use: pcieport 40:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 40:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 40:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B Kernel driver in use: pcieport 40:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge 40:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B Kernel driver in use: pcieport 41:00.0 Non-Volatile memory controller: ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive (rev 03) Subsystem: ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive Kernel driver in use: nvme Kernel modules: nvme 42:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951 (rev 01) Subsystem: Samsung Electronics Co Ltd Device a801 Kernel driver in use: nvme Kernel modules: nvme 43:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (rev ff) Subsystem: Sapphire Technology Limited Radeon RX 550 640SP Kernel driver in use: amdgpu Kernel modules: amdgpu 43:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] Subsystem: Sapphire Technology Limited Device aae0 Kernel driver in use: snd_hda_intel Kernel modules: snd_hda_intel 44:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function Subsystem: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function 44:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor Kernel driver in use: ccp Kernel modules: ccp 44:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller Kernel driver in use: xhci_hcd 45:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function Subsystem: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function 45:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) Subsystem: Gigabyte Technology Co., Ltd Device b002 Kernel driver in use: ahci And here the output of the script: IOMMU Group 0: 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452] IOMMU Group 1: 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453] IOMMU Group 10: 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454] IOMMU Group 11: 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 59) 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51) IOMMU Group 12: 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1460] 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1461] 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1462] 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1463] 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1464] 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1465] 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1466] 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1022:1467] IOMMU Group 13: 00:19.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1460] 00:19.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1461] 00:19.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1462] 00:19.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1463] 00:19.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1464] 00:19.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1465] 00:19.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1466] 00:19.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1022:1467] IOMMU Group 14: 01:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller [1022:43ba] (rev 02) IOMMU Group 15: 01:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller [1022:43b6] (rev 02) IOMMU Group 16: 01:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge [1022:43b1] (rev 02) IOMMU Group 17: 02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02) IOMMU Group 18: 02:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02) IOMMU Group 19: 02:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02) IOMMU Group 2: 00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453] IOMMU Group 20: 02:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02) IOMMU Group 21: 02:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02) IOMMU Group 22: 04:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03) IOMMU Group 23: 05:00.0 Network controller [0280]: Intel Corporation Wireless 8265 / 8275 [8086:24fd] (rev 78) IOMMU Group 24: 06:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03) IOMMU Group 25: 08:00.0 Non-Volatile memory controller [0108]: Silicon Motion, Inc. Device [126f:2260] (rev 03) IOMMU Group 26: 09:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU104 [GeForce RTX 2080] [10de:1e82] (rev a1) IOMMU Group 27: 09:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f8] (rev a1) IOMMU Group 28: 09:00.2 USB controller [0c03]: NVIDIA Corporation Device [10de:1ad8] (rev a1) IOMMU Group 29: 09:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device [10de:1ad9] (rev a1) IOMMU Group 3: 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452] IOMMU Group 30: 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function [1022:145a] IOMMU Group 31: 0a:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor [1022:1456] IOMMU Group 32: 0a:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller [1022:145c] IOMMU Group 33: 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function [1022:1455] IOMMU Group 34: 0b:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51) IOMMU Group 35: 0b:00.3 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller [1022:1457] IOMMU Group 4: 00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452] IOMMU Group 5: 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453] IOMMU Group 6: 00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452] IOMMU Group 7: 00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452] IOMMU Group 8: 00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454] IOMMU Group 9: 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452] Quote Link to comment
testdasi Posted November 1, 2019 Share Posted November 1, 2019 15 hours ago, derstef said: Sorry, i am not using unraid. Im using Fedora 31 with ACS patch. Hope thats fine in this forum. Does the distro flavor have any influence to the IOMMU grouping? This forum only focuses on Unraid related queries so it's unlikely you will get any answer relevant to Fedora 31. Not even the same kernel. Quote Link to comment
derstef Posted November 1, 2019 Share Posted November 1, 2019 1 hour ago, testdasi said: This forum only focuses on Unraid related queries so it's unlikely you will get any answer relevant to Fedora 31. Not even the same kernel. To me it looks more Bios or Hardware-related so i hope to get some answer from users of the same hardware and Bios. However i totally understand that i will not get any help by Unraid staff when not using their software. Quote Link to comment
jbartlett Posted November 6, 2019 Author Share Posted November 6, 2019 On 10/31/2019 at 3:59 PM, derstef said: 08:00.0 Non-Volatile memory controller: Silicon Motion, Inc. Device 2260 (rev 03) Subsystem: Silicon Motion, Inc. Device 2260 Kernel driver in use: nvme Kernel modules: nvme I see this one which matches mine. You may have to stub it out so the OS doesn't grab it. Quote Link to comment
JWMutant Posted November 14, 2019 Share Posted November 14, 2019 Will be following this thread with interest as im thinking about buy either this board or the Gigabyte X399 AORUS Xtreme. Quote Link to comment
jbartlett Posted November 14, 2019 Author Share Posted November 14, 2019 (edited) 10 hours ago, JWMutant said: Will be following this thread with interest as im thinking about buy either this board or the Gigabyte X399 AORUS Xtreme. Sorry, I commented about the wrong motherboard. Deleted post. The Designare board is good but the PCIe passthrough proved to be problematic for me without the PCIe override enabled. I haven't tried in the past year though. It's been running rock steady though with a 1950X otherwise. The board I'm working with now and in love with is the ASUS ROG Zenith Extreme Alpha X399. Edited November 14, 2019 by jbartlett Quote Link to comment
JWMutant Posted November 21, 2019 Share Posted November 21, 2019 I know I keep digging up an old thread but it beats starting a new one for no reason. Im looking at getting this board. Do you see me having any issues running a cheap GPU for unraid and good GPU for Win10 VM a SAS card and a Network card all at once? I know there are a few issues with IOMMU groups with alot of X399 boards which is the reason for the NIC and dont really need USB pass through per say as ill be RDP via a thin client. Quote Link to comment
testdasi Posted November 21, 2019 Share Posted November 21, 2019 2 hours ago, JWMutant said: I know I keep digging up an old thread but it beats starting a new one for no reason. Im looking at getting this board. Do you see me having any issues running a cheap GPU for unraid and good GPU for Win10 VM a SAS card and a Network card all at once? I know there are a few issues with IOMMU groups with alot of X399 boards which is the reason for the NIC and dont really need USB pass through per say as ill be RDP via a thin client. I don't see any reason why not. Depending on what cheap GPU you use, you might even still have a spare PCIe slot left for other uses. I can't quite connect your points about IOMMU group issues and needing the NIC. Shameless plug: you can have a look at my rig below with that mobo (with IOMMU groups) Quote Link to comment
jbartlett Posted November 21, 2019 Author Share Posted November 21, 2019 (edited) I don't have a SAS card but I run a cheap GPU along for unraid to connect to and a Quadro P2000 for my broadcasting VM on a different MB (new build). I decided on a GeForce GT 1030 for being both single slot and a larger fan. Larger fans don't have to move as fast to move the same amount of air and thus tend to be quieter. Edited November 21, 2019 by jbartlett Quote Link to comment
JWMutant Posted November 21, 2019 Share Posted November 21, 2019 (edited) The issue with IOMMU is it doesn't separate the dual on board network instead leaves them in the same group making pass through a little hard, hence the extra NIC, my VM needs a dedicated NIC. Edited November 21, 2019 by JWMutant Quote Link to comment
JWMutant Posted November 23, 2019 Share Posted November 23, 2019 If you had to choose between the X399 Designare EX and the ASRock X399 Taichi which would you choose? Quote Link to comment
jbartlett Posted November 23, 2019 Author Share Posted November 23, 2019 1 hour ago, JWMutant said: If you had to choose between the X399 Designare EX and the ASRock X399 Taichi which would you choose? That's tough to say without actually having first hand experience on the Taichi board. Specs can read all nice and lovely but once you get it in hand, the IOMMU configuration is all whacked or the BIOS doesn't fully support HW passthrough, etc. What I did was to Google the MB and KVM/quemu/hypervisor/etc to see what other people have done with it and then decide that way. Still a gamble. Quote Link to comment
meep Posted November 24, 2019 Share Posted November 24, 2019 2 hours ago, JWMutant said: If you had to choose between the X399 Designare EX and the ASRock X399 Taichi which would you choose? I have the Taichi if you want to ping me any questions. I’ve got to know it very well in the past couple of weeks. Quote Link to comment
jpowell8672 Posted November 24, 2019 Share Posted November 24, 2019 9 hours ago, JWMutant said: If you had to choose between the X399 Designare EX and the ASRock X399 Taichi which would you choose? ASRock X399 Taichi Quote Link to comment
derstef Posted December 8, 2019 Share Posted December 8, 2019 On 10/31/2019 at 9:35 PM, derstef said: First of all, thank you for sharing the information above. I've running F12i Bios with the same board and my non boot NVME drives arent in any IOMMU Groups at all. (im using all 3 M.2 slots) They show up with lspci but not using the script here: https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Ensuring_that_the_groups_are_valid When i try to start my VM i get "Fehler: Interner Fehler: Process exited prior to exec: libvirt: Fehler : Interner Fehler: Ungültige Einheit 0000:42:00.0 iommu_group Datei /sys/bus/pci/devices/0000:42:00.0/iommu_group ist kein symlink" (sorry my system is localized to german) Have you encountered such situation too? So after patching my kernel with ACS and the Agesa patches and using downstream, multifunction setting for the ACS the Designare EX is finally usable for me. I still got some trouble with the reset of the NVMe drive (stubbed via vfio) i pass through to windows (Samsung) when the VM is shut down but thats just annoying and no showstopper. For network i don't pass through the NICs but use an USB network adapter for windows which is just fine. The not so important (latency wise) VMs are bridged to the hosts NIC. For sound i use a Corsair wireless headset with usb dongle which can be passed to the windows VM and gets recognized (back) by the host flawlessly when shutting down the windows vm. So in overall id call it a success story for the Designare EX. Im just a little disappointed about the need for ACS, as this wasn't necessary on my former ASRock X370 Killer SLI AM4 board. Quote Link to comment
jbartlett Posted December 9, 2019 Author Share Posted December 9, 2019 20 hours ago, derstef said: I still got some trouble with the reset of the NVMe drive (stubbed via vfio) i pass through to windows (Samsung) when the VM is shut down but thats just annoying and no showstopper. Test it without passing through any other devices. I have a MB that one of the LAN ports may not always survive a VM reboot if I was also passing through a graphics card - stopping passing through the card and it survived over & over. Just something to troubleshoot. Quote Link to comment
testdasi Posted December 9, 2019 Share Posted December 9, 2019 21 hours ago, derstef said: So after patching my kernel with ACS and the Agesa patches and using downstream, multifunction setting for the ACS the Designare EX is finally usable for me. I still got some trouble with the reset of the NVMe drive (stubbed via vfio) i pass through to windows (Samsung) when the VM is shut down but thats just annoying and no showstopper. For network i don't pass through the NICs but use an USB network adapter for windows which is just fine. The not so important (latency wise) VMs are bridged to the hosts NIC. For sound i use a Corsair wireless headset with usb dongle which can be passed to the windows VM and gets recognized (back) by the host flawlessly when shutting down the windows vm. So in overall id call it a success story for the Designare EX. Im just a little disappointed about the need for ACS, as this wasn't necessary on my former ASRock X370 Killer SLI AM4 board. Just notice you mentioned 12i BIOS. The 12i BIOS is bugged (at least on my Designare EX). It doesn't even boot properly with my settings, let alone passing through any device. I am now sticking to 12e. With regards to ACS Override, you are not running enterprise-level security. For home users, 99% of any concern over ACS Override is completely irrelevant. It's a crude example, but it's like of course you don't want to wear underwear to work but at home, who cares if you are in your underwear if that makes your life (at home) easier? Last but not least, you are not using Unraid. Quote Link to comment
skois Posted December 17, 2019 Share Posted December 17, 2019 On 12/9/2019 at 10:22 PM, testdasi said: Just notice you mentioned 12i BIOS. The 12i BIOS is bugged (at least on my Designare EX). It doesn't even boot properly with my settings, let alone passing through any device. I am now sticking to 12e. With regards to ACS Override, you are not running enterprise-level security. For home users, 99% of any concern over ACS Override is completely irrelevant. It's a crude example, but it's like of course you don't want to wear underwear to work but at home, who cares if you are in your underwear if that makes your life (at home) easier? Last but not least, you are not using Unraid. What problems do you have with F12i? Also i can't see on the website the F12e you mention. (I'm not someone that can help you solve anything) I will be upgrading my server today to a x399 Ex Designare + tr2920. Dont know what BIOS does it have preinstalled yet. Quote Link to comment
JWMutant Posted December 17, 2019 Share Posted December 17, 2019 Also i can't see on the website the F12e you mention.Same. I have had a look as well and can’t find it on the site.Sent from my iPhone using Tapatalk Quote Link to comment
testdasi Posted December 17, 2019 Share Posted December 17, 2019 (edited) 4 hours ago, skois said: What problems do you have with F12i? Also i can't see on the website the F12e you mention. (I'm not someone that can help you solve anything) I will be upgrading my server today to a x399 Ex Designare + tr2920. Dont know what BIOS does it have preinstalled yet. These were the problems: Couldn't reliably save config. When config could be saved, it didn't retain beyond 1-2 boot cycles. Exit without saving = can't boot up at all (need to clear CMOS for it to boot back up). Saving profile crashed the BIOS itself (blank screen). Patch note said "PCIe bifurcation" as additional feature, which is misleading. It really just changes the wording of the BIOS setting for PCIe slots. This feature was already available for a while e.g. F12e, just under different name. I think Gigabyte only keeps the latest of each version number so the 12e probably is no longer available. These problems also could very well be my sample's idiosyncracies. If you guys have problems with 12i, just ask. I have 12e saved down (original Gigabyte version). Last but not least: if you need ACS Override, turn it on (and ignore all the security chatter). Only James Bond worries about being hacked through ACS Override on a home server. Edited December 17, 2019 by testdasi Quote Link to comment
JWMutant Posted December 17, 2019 Share Posted December 17, 2019 If you have 12e I wouldnt mind getting a zip if thats ok. Quote Link to comment
testdasi Posted December 17, 2019 Share Posted December 17, 2019 10 hours ago, JWMutant said: If you have 12e I wouldnt mind getting a zip if thats ok. Find it in the post below. Quote Link to comment
JWMutant Posted December 18, 2019 Share Posted December 18, 2019 10 hours ago, testdasi said: Find it in the post below. Thank you very much. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.