rachid596 Posted June 16, 2021 Share Posted June 16, 2021 5 hours ago, k0vw said: Yeah it's the drivers that have changed i think? They're now the USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller that everyone has been having trouble with. The groups themselves haven't changed. I've deleted the VFIO config file from the USB and tried to rebuild it but still having the same issues no matter what. It's killing me haha I don't know what can cause this issues. Me with my ryzen 9 3900x and x570 Board it's the old kernel in 6.8.3 without flr patch but now it's included with kernel 5.x. Have you install last bios ? Send your iommu Quote Link to comment
k0vw Posted June 16, 2021 Share Posted June 16, 2021 1 hour ago, rachid596 said: I don't know what can cause this issues. Me with my ryzen 9 3900x and x570 Board it's the old kernel in 6.8.3 without flr patch but now it's included with kernel 5.x. Have you install last bios ? Send your iommu Yeah I'm on the latest bios for my board. Sorry im not sure if this is the easiest way to do what you asked. IOMMU group 0: [1022:1482] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 1: [1022:1483] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 2: [1022:1482] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 3: [1022:1482] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 4: [1022:1483] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 5: [1022:1483] 00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 6: [1022:1482] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 7: [1022:1482] 00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 8: [1022:1482] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 9: [1022:1484] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 10: [1022:1482] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 11: [1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 12: [1022:1484] 00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 13: [1022:1484] 00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 14: [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 15: [1022:1440] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1441] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1442] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1443] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1444] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1445] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1446] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1447] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 IOMMU group 16: [1022:43d0] 03:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43d0 (rev 01) Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002: ID 0781:5567 SanDisk Corp. Cruzer Blade Bus 001 Device 003: ID 046b:ff01 American Megatrends, Inc. Virtual Hub Bus 001 Device 004: ID 046b:ff20 American Megatrends, Inc. Virtual Cdrom Device Bus 001 Device 005: ID 046b:ff31 American Megatrends, Inc. Virtual HardDisk Device Bus 001 Device 006: ID 046b:ffb0 American Megatrends, Inc. Virtual Ethernet Bus 001 Device 007: ID 046b:ff40 American Megatrends, Inc. Virtual Floppy Device Bus 001 Device 008: ID 046b:ff10 American Megatrends, Inc. Virtual Keyboard and Mouse Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub [1022:43c8] 03:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset SATA Controller (rev 01) [4:0:0:0] disk ATA WDC WDS100T2G0A- 40WD /dev/sdd 1.00TB [5:0:0:0] disk ATA WDC WDS240G2G0A- 0000 /dev/sde 240GB [6:0:0:0] disk ATA ST3000DM007-1WY1 0001 /dev/sdf 3.00TB [7:0:0:0] disk ATA ST3000DM007-1WY1 0001 /dev/sdg 3.00TB [10:0:0:0] disk ATA ST3000DM007-1WY1 0001 /dev/sdh 3.00TB [11:0:0:0] disk ATA ST8000DM004-2CX1 0001 /dev/sdi 8.00TB [1022:43c6] 03:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Bridge (rev 01) [1022:43c7] 20:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1022:43c7] 20:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1022:43c7] 20:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1022:43c7] 20:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1022:43c7] 20:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1022:43c7] 20:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port (rev 01) [1a03:1150] 21:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 04) [1a03:2000] 22:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41) [8086:1533] 23:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) [8086:1533] 24:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) [1b21:0612] 25:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02) [13:0:0:0] disk ATA ST8000DM004-2CX1 0001 /dev/sdj 8.00TB IOMMU group 17: [10de:1c82] 2b:00.0 VGA compatible controller: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] (rev a1) [10de:0fb9] 2b:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1) IOMMU group 18: [10de:1cb3] 2c:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P400] (rev a1) [10de:0fb9] 2c:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1) IOMMU group 19: [1022:148a] 2d:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function IOMMU group 20: [1022:1485] 2e:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP IOMMU group 21: [1022:1486] 2e:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP IOMMU group 22: [1022:149c] 2e:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller This controller is bound to vfio, connected USB devices are not visible. IOMMU group 23: [1022:1487] 2e:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller IOMMU group 24: [1022:7901] 30:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 25: [1022:7901] 31:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) The attached image is the USB that I'm trying to passthrough (20-23). Thank you so much for your help. I really appreciate it. Quote Link to comment
rachid596 Posted June 16, 2021 Share Posted June 16, 2021 Send a screenshot of your iommu it's better. In log do you have any error when you start the vm ? In the log of your VM ? Envoyé de mon HD1913 en utilisant Tapatalk Quote Link to comment
k0vw Posted June 16, 2021 Share Posted June 16, 2021 (edited) 19 minutes ago, rachid596 said: Send a screenshot of your iommu it's better. In log do you have any error when you start the vm ? In the log of your VM ? Envoyé de mon HD1913 en utilisant Tapatalk Please see the attached screenshot. Sorry how would i access the log files? Once i try to start the VM im unable to click the VM icon and go to logs. The sever begins to freeze and crashes. Edited June 16, 2021 by k0vw Quote Link to comment
rachid596 Posted June 16, 2021 Share Posted June 16, 2021 1 minute ago, k0vw said: Please see the attached screenshot. Sorry how would i access the log files? For system log go to tools and system log. For the log of the VM click inthe icon of the vm and logs Quote Link to comment
k0vw Posted June 16, 2021 Share Posted June 16, 2021 1 minute ago, rachid596 said: For system log go to tools and system log. For the log of the VM click inthe icon of the vm and logs Yeah sorry I cant click on the VM logs. The system just begins to freeze and pins all CPU cores to 100%. I just tried to start the VM to see if i could see the logs so this system log is quite short. opti-syslog-20210616-0852.zip Quote Link to comment
rachid596 Posted June 16, 2021 Share Posted June 16, 2021 3 minutes ago, k0vw said: Yeah sorry I cant click on the VM logs. The system just begins to freeze and pins all CPU cores to 100%. I just tried to start the VM to see if i could see the logs so this system log is quite short. do you edit anything here? Quote Link to comment
k0vw Posted June 16, 2021 Share Posted June 16, 2021 1 minute ago, rachid596 said: do you edit anything here? Yes i've added append pcie_no_flr=1022:1485,1022:1486,1022:149c,1022:1487 vfio-pci.ids=1022:1485,1022:1486,1022:149c,1022:1487 Quote Link to comment
rachid596 Posted June 16, 2021 Share Posted June 16, 2021 Just now, k0vw said: Yes i've added append pcie_no_flr=1022:1485,1022:1486,1022:149c,1022:1487 vfio-pci.ids=1022:1485,1022:1486,1022:149c,1022:1487 delete this pcie_no_flr=1022:1485,1022:1486,1022:149c,1022:1487 vfio-pci.ids=1022:1485,1022:1486,1022:149c,1022:1487 and in tick what you want in tools system devices Quote Link to comment
k0vw Posted June 16, 2021 Share Posted June 16, 2021 1 minute ago, rachid596 said: delete this pcie_no_flr=1022:1485,1022:1486,1022:149c,1022:1487 vfio-pci.ids=1022:1485,1022:1486,1022:149c,1022:1487 and in tick what you want in tools system devices Done. The ones ticked in the system devices are still the same. Rebooted the system and tried to start the VM. Same issue. I did go to the logs right away and saw this Jun 16 19:05:44 Opti kernel: clocksource: Switched to clocksource hpet Jun 16 19:05:55 Opti kernel: vfio-pci 0000:2e:00.0: not ready 1023ms after FLR; waiting Jun 16 19:05:58 Opti kernel: vfio-pci 0000:2e:00.0: not ready 2047ms after FLR; waiting Jun 16 19:06:01 Opti kernel: vfio-pci 0000:2e:00.0: not ready 4095ms after FLR; waiting Jun 16 19:06:06 Opti kernel: vfio-pci 0000:2e:00.0: not ready 8191ms after FLR; waiting Had to reboot the server again opti-syslog-20210616-0907.zip Quote Link to comment
rachid596 Posted June 16, 2021 Share Posted June 16, 2021 the problem is group 20. Quote Link to comment
k0vw Posted June 16, 2021 Share Posted June 16, 2021 Just now, rachid596 said: the problem is group 20. Yeah thats the USB Im trying to pass through. Very frustrating. Quote Link to comment
rachid596 Posted June 16, 2021 Share Posted June 16, 2021 Usb is group 22 ? Envoyé de mon HD1913 en utilisant Tapatalk Quote Link to comment
k0vw Posted June 16, 2021 Share Posted June 16, 2021 2 hours ago, rachid596 said: Usb is group 22 ? Envoyé de mon HD1913 en utilisant Tapatalk Yeah that one... So i've found the issue. It was the FLR on 2e.00.0 which is the non essential device. I'd been told that i needed to pass that through for the usb to work and was doing that when i had my 2600 installed. Just passing through the USB works fine. Thanks so much for your help mate. Really appreciate it. Quote Link to comment
rachid596 Posted June 17, 2021 Share Posted June 17, 2021 22 hours ago, k0vw said: Yeah that one... So i've found the issue. It was the FLR on 2e.00.0 which is the non essential device. I'd been told that i needed to pass that through for the usb to work and was doing that when i had my 2600 installed. Just passing through the USB works fine. Thanks so much for your help mate. Really appreciate it. No problem Have a good day enjoy Quote Link to comment
Maor Posted July 16, 2021 Share Posted July 16, 2021 (edited) Hi @rachid596 I have a Gigabyte X570 aorus master + 5900x, I am on bios F34 with agesa combov2 1.2.0.3B, also tried with with bios F33i agesa 1.2.0.1A with same problem. I run unraid 6.9.2, which have kernel 5.10, which should have the necessary patches, unless the patches were dropped? I am trying to passthrough the audio controller, my IOMMU: IOMMU group 0: [1022:1482] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 1: [1022:1483] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 2: [1022:1483] 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 3: [1022:1482] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 4: [1022:1482] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 5: [1022:1483] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 6: [1022:1483] 00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge IOMMU group 7: [1022:1482] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 8: [1022:1482] 00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 9: [1022:1482] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 10: [1022:1484] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 11: [1022:1482] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 12: [1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 13: [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 14: [1022:1440] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1441] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1442] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1443] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1444] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1445] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1446] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1447] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 IOMMU group 15: [1e0f:0009] 01:00.0 Non-Volatile memory controller: KIOXIA Corporation NVMe SSD (rev 01) [N:0:1:1] disk KIOXIA-EXCERIA SSD__1 /dev/nvme0n1 1.00TB IOMMU group 16: [1022:57ad] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream IOMMU group 17: [1022:57a3] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 18: [1022:57a3] 03:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 19: [1022:57a3] 03:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 20: [1022:57a3] 03:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 21: [1022:57a3] 03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 22: [1022:57a3] 03:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 23: [1022:57a3] 03:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge IOMMU group 24: [1022:57a4] 03:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:1485] 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:149c] 0b:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002: ID 048d:8297 Integrated Technology Express, Inc. ITE Device(8595) Bus 001 Device 003: ID 05e3:0608 Genesys Logic, Inc. Hub Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub [1022:149c] 0b:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 003 Device 002: ID 8087:0029 Intel Corp. Bus 003 Device 003: ID 05e3:0608 Genesys Logic, Inc. Hub Bus 003 Device 004: ID 13fe:4300 Kingston Technology Company Inc. USB DISK 2.0 Bus 003 Device 005: ID 046d:c539 Logitech, Inc. USB Receiver Bus 003 Device 006: ID 046a:0010 Cherry GmbH SmartBoard XX44 Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub IOMMU group 25: [1022:57a4] 03:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:7901] 0c:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 26: [1022:57a4] 03:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:7901] 0d:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) [5:0:0:0] disk ATA HGST HUH721212AL T3R0 /dev/sda 12.0TB [9:0:0:0] disk ATA HGST HUH721212AL T3R0 /dev/sdd 12.0TB IOMMU group 27: [1e0f:0009] 04:00.0 Non-Volatile memory controller: KIOXIA Corporation NVMe SSD (rev 01) [N:1:1:1] disk KIOXIA-EXCERIA SSD__1 /dev/nvme1n1 1.00TB IOMMU group 28: [1e0f:0009] 05:00.0 Non-Volatile memory controller: KIOXIA Corporation NVMe SSD (rev 01) [N:2:1:1] disk KIOXIA-EXCERIA SSD__1 /dev/nvme2n1 1.00TB IOMMU group 29: [10de:1b81] 06:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1) [10de:10f0] 06:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1) IOMMU group 30: [8086:2723] 07:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a) IOMMU group 31: [8086:1539] 08:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 32: [10ec:8125] 09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05) IOMMU group 33: [144d:a808] 0a:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [N:3:4:1] disk SAMSUNG MZVLB512HAJQ-000L7__1 /dev/nvme3n1 512GB IOMMU group 34: [10de:1b82] 0e:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070 Ti] (rev a1) [10de:10f0] 0e:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1) IOMMU group 35: [10de:1b81] 0f:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1) [10de:10f0] 0f:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1) IOMMU group 36: [1022:148a] 10:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function IOMMU group 37: [1022:1485] 11:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP IOMMU group 38: [1022:1486] 11:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP IOMMU group 39: [1022:149c] 11:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller This controller is bound to vfio, connected USB devices are not visible. IOMMU group 40: [1022:1487] 11:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller I have the Audio device bound to VFIO: Loading config from /boot/config/vfio-pci.cfg BIND=0000:07:00.0|8086:2723 0000:11:00.0|1022:1485 0000:11:00.1|1022:1486 0000:11:00.3|1022:149c 0000:11:00.4|1022:1487 --- Processing 0000:07:00.0 8086:2723 Vendor:Device 8086:2723 found at 0000:07:00.0 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:07:00.0/iommu_group/devices/0000:07:00.0 Binding... Successfully bound the device 8086:2723 at 0000:07:00.0 to vfio-pci --- Processing 0000:11:00.0 1022:1485 Vendor:Device 1022:1485 found at 0000:11:00.0 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:11:00.0/iommu_group/devices/0000:11:00.0 Binding... Successfully bound the device 1022:1485 at 0000:11:00.0 to vfio-pci --- Processing 0000:11:00.1 1022:1486 Vendor:Device 1022:1486 found at 0000:11:00.1 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:11:00.1/iommu_group/devices/0000:11:00.1 Binding... Successfully bound the device 1022:1486 at 0000:11:00.1 to vfio-pci --- Processing 0000:11:00.3 1022:149c Vendor:Device 1022:149c found at 0000:11:00.3 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:11:00.3/iommu_group/devices/0000:11:00.3 Binding... Unbound 0000:11:00.3 from xhci_hcd Successfully bound the device 1022:149c at 0000:11:00.3 to vfio-pci --- Processing 0000:11:00.4 1022:1487 Vendor:Device 1022:1487 found at 0000:11:00.4 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:11:00.4/iommu_group/devices/0000:11:00.4 Binding... Successfully bound the device 1022:1487 at 0000:11:00.4 to vfio-pci --- vfio-pci binding complete Devices listed in /sys/bus/pci/drivers/vfio-pci: lrwxrwxrwx 1 root root 0 Jul 14 16:22 0000:07:00.0 -> ../../../../devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:03.0/0000:07:00.0 lrwxrwxrwx 1 root root 0 Jul 14 16:22 0000:11:00.0 -> ../../../../devices/pci0000:00/0000:00:08.1/0000:11:00.0 lrwxrwxrwx 1 root root 0 Jul 14 16:22 0000:11:00.1 -> ../../../../devices/pci0000:00/0000:00:08.1/0000:11:00.1 lrwxrwxrwx 1 root root 0 Jul 14 16:22 0000:11:00.3 -> ../../../../devices/pci0000:00/0000:00:08.1/0000:11:00.3 lrwxrwxrwx 1 root root 0 Jul 14 16:22 0000:11:00.4 -> ../../../../devices/pci0000:00/0000:00:08.1/0000:11:00.4 ls -l /dev/vfio/ total 0 crw------- 1 root root 249, 0 Jul 14 16:22 30 crw------- 1 root root 249, 1 Jul 14 16:22 37 crw------- 1 root root 249, 2 Jul 14 16:22 38 crw------- 1 root root 249, 3 Jul 14 16:22 39 crw------- 1 root root 249, 4 Jul 14 16:22 40 crw-rw-rw- 1 root root 10, 196 Jul 14 16:22 vfio my VM setting: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x11' slot='0x00' function='0x4'/> </source> <alias name='hostdev2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> </hostdev> log from my VM: -no-hpet \ -no-shutdown \ -boot strict=on \ -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 \ -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 \ -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 \ -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2 \ -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 \ -blockdev '{"driver":"file","filename":"/mnt/user/domains/Earth LLC/vdisk1.img","node-name":"libvirt-4-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-4-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-4-storage"}' \ -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=libvirt-4-format,id=virtio-disk2,bootindex=1,write-cache=on \ -blockdev '{"driver":"file","filename":"/mnt/user/domains/Earth LLC/vdisk2.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"qcow2","file":"libvirt-3-storage","backing":null}' \ -device virtio-blk-pci,bus=pci.0,addr=0x6,drive=libvirt-3-format,id=virtio-disk3,write-cache=on \ -blockdev '{"driver":"file","filename":"/mnt/user/isos/Windows_1909.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \ -device ide-cd,bus=ide.0,unit=0,drive=libvirt-2-format,id=ide0-0-0,bootindex=2 \ -blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.173-2.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \ -device ide-cd,bus=ide.0,unit=1,drive=libvirt-1-format,id=ide0-0-1 \ -netdev tap,fd=35,id=hostnet0 \ -device virtio-net,netdev=hostnet0,id=net0,mac=52:54:00:ac:47:5b,bus=pci.0,addr=0x3 \ -chardev pty,id=charserial0 \ -device isa-serial,chardev=charserial0,id=serial0 \ -chardev socket,id=charchannel0,fd=36,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \ -device usb-tablet,id=input0,bus=usb.0,port=1 \ -vnc 0.0.0.0:1,websocket=5701 \ -k en-us \ -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 \ -device vfio-pci,host=0000:0e:00.0,id=hostdev0,bus=pci.0,addr=0x8,romfile=/mnt/user/isos/evga1070ti.rom \ -device vfio-pci,host=0000:0e:00.1,id=hostdev1,bus=pci.0,addr=0x9 \ -device vfio-pci,host=0000:11:00.4,id=hostdev2,bus=pci.0,addr=0xa \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2021-07-16 17:40:09.939+0000: Domain id=3 is tainted: high-privileges 2021-07-16 17:40:09.939+0000: Domain id=3 is tainted: host-cpu char device redirected to /dev/pts/1 (label charserial0) 2021-07-16T17:40:16.396552Z qemu-system-x86_64: vfio: Cannot reset device 0000:11:00.4, depends on group 37 which is not owned. 2021-07-16T17:40:17.431841Z qemu-system-x86_64: vfio: Cannot reset device 0000:11:00.4, depends on group 37 which is not owned. So I add the [1022:1485] 11:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP device to my VM like this: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x11' slot='0x00' function='0x4'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x11' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> </hostdev> and get an error after starting the VM: internal error: Unknown PCI header type '127' for device '0000:11:00.4' Any idea, what am I missing? Edited July 16, 2021 by Maor update Quote Link to comment
Nackophilz Posted August 3, 2022 Share Posted August 3, 2022 Just to re up the topic a last time. With the last stable version of unraid, we can passtrough Audio Sound Card to our VM ! Just have to add it to exclusion via the System Devices tools (no need to took the entire IOMMU Group just the sound card) Attach it to your VM And voila ! (Work on X570 Aorus Master with Win 11 VM ) Quote Link to comment
SolarBottcher Posted September 17, 2022 Share Posted September 17, 2022 No hope with x470 gaming 5 motherboard problem exists forever no flr support with latest f63c bios and 6.10.3 unraid system The temporary solution for me is to do pcie_no_flr=1022:148a,1022:1485,1022:1486,1022:149c,1022:1487 http://patchwork.ozlabs.org/project/linux-pci/patch/[email protected]/#2758883 <--reason why im doing this and do vfio bind as shown in the pic in this case, it stop crashing when the vm starts and did actually go into the windows logon menu. However! it still crashes when im logging in. I disable the windows startup sound, and I got no problems anymore.(weird, but I guess it might be some driver issues?) I stop configuring anything by then. I have spent too much time troubleshooting, I am losing interest messing up vm stuffs. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.