Xerby

Hi, I am on my extended trial for unraid and one thing that's stopping me from buying is trying to pass through a USB Controller. So far things work well when it works, Windows 10 VM has passed through GPU successfully, when passing through just USB devices works fine but the issue is when USB devices disconnect and then have go through the web interface to re-attach the device or reboot the vm. System specs: M/B: Gigabyte Technology Co., Ltd. X570 GAMING X Version x.x (latest bios) BIOS: American Megatrends Inc. Version F11. Dated: 12/06/2019 CPU: AMD Ryzen 9 3900X 12-Core @ 3800 MHz HVM: Enabled IOMMU: Enabled Cache: 768 KiB, 6144 KiB, 65536 KiB Memory: 32 GiB DDR4 IOMMU group 0: [1022:1482] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1483] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483] 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1bb1:5016] 01:00.0 Non-Volatile memory controller: Seagate Technology PLC Device 5016 (rev 01) [1022:57ad] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream [1022:57a3] 03:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3] 03:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3] 03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3] 03:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4] 03:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4] 03:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4] 03:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [144d:a808] 04:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [10de:13bb] 05:00.0 VGA compatible controller: NVIDIA Corporation GM107GL [Quadro K620] (rev a2) [10de:0fbc] 05:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1) [10ec:8168] 06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 16) [1b21:0612] 07:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02) [1022:1485] 08:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:149c] 08:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c] 08:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:7901] 09:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) [1022:7901] 0a:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 1: [1022:1482] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 2: [1022:1482] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1483] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [10de:1187] 0b:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 760] (rev a1) [10de:0e0a] 0b:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1) IOMMU group 3: [1022:1482] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 4: [1022:1482] 00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 5: [1022:1482] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 6: [1022:1484] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 7: [1022:1482] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge IOMMU group 8: [1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 9: [1022:1484] 00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 10: [1022:1484] 00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] IOMMU group 11: [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 12: [1022:1440] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1441] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1442] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1443] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1444] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1445] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1446] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1447] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 IOMMU group 13: [1022:148a] 0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function IOMMU group 14: [1022:1485] 0d:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP IOMMU group 15: [1022:1486] 0d:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP IOMMU group 16: [1022:149c] 0d:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller IOMMU group 17: [1022:1487] 0d:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller IOMMU group 18: [1022:7901] 0e:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 19: [1022:7901] 0f:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) So IOMMU group 16: [1022:149c] 0d:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller is the back of the Mobo USB ports, looks greats its in its own group with no other devices everything I've seen/read sounds like this is the best thing i could hope for. IOMMU Group 0: [1022:149c] 08:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller This controller has the unRaid USB Issues start when i try to pass through 0d:00.3, first way i tried was append vfio-pci.ids=1022:149c but this caused unraid to not boot at all i was wondering if it was because both controllers shared the same number. i have tried the pcie_acs_override=downstream,multifunction with this as well but still unraid wouldn't boot. Next way i tried is VFIO-PCI CFG which seemed to work, it got unraid booted and i was able to select the USB Controller in Other PCI devices. Then i tried to start the VM: Jan 29 18:55:49 XiamoUnraid kernel: br0: port 2(vnet0) entered blocking state Jan 29 18:55:49 XiamoUnraid kernel: br0: port 2(vnet0) entered disabled state Jan 29 18:55:49 XiamoUnraid kernel: device vnet0 entered promiscuous mode Jan 29 18:55:49 XiamoUnraid kernel: br0: port 2(vnet0) entered blocking state Jan 29 18:55:49 XiamoUnraid kernel: br0: port 2(vnet0) entered forwarding state Jan 29 18:55:50 XiamoUnraid kernel: vfio-pci 0000:0d:00.3: enabling device (0000 -> 0002) Jan 29 18:55:51 XiamoUnraid avahi-daemon[3024]: Joining mDNS multicast group on interface vnet0.IPv6 with address fe80::fc54:ff:fef7:df28. Jan 29 18:55:51 XiamoUnraid avahi-daemon[3024]: New relevant interface vnet0.IPv6 for mDNS. Jan 29 18:55:51 XiamoUnraid avahi-daemon[3024]: Registering new address record for fe80::fc54:ff:fef7:df28 on vnet0.*. Jan 29 18:56:03 XiamoUnraid kernel: vfio-pci 0000:0d:00.3: not ready 1023ms after FLR; waiting Jan 29 18:56:05 XiamoUnraid kernel: vfio-pci 0000:0d:00.3: not ready 2047ms after FLR; waiting Jan 29 18:56:08 XiamoUnraid kernel: vfio-pci 0000:0d:00.3: not ready 4095ms after FLR; waiting Jan 29 18:56:13 XiamoUnraid kernel: vfio-pci 0000:0d:00.3: not ready 8191ms after FLR; waiting Jan 29 18:57:14 XiamoUnraid kernel: clocksource: timekeeping watchdog on CPU7: Marking clocksource 'tsc' as unstable because the skew is too large: Jan 29 18:57:14 XiamoUnraid kernel: clocksource: 'hpet' wd_now: 9f6a0fcd wd_last: 9d88d3d5 mask: ffffffff Jan 29 18:57:14 XiamoUnraid kernel: clocksource: 'tsc' cs_now: f36a63b2100 cs_last: f3596a25a74 mask: ffffffffffffffff Jan 29 18:57:14 XiamoUnraid kernel: tsc: Marking TSC unstable due to clocksource watchdog Jan 29 18:57:16 XiamoUnraid kernel: TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'. Jan 29 18:57:16 XiamoUnraid kernel: sched_clock: Marking unstable (4386715655544, -38594027)<-(4386979681370, -302640679) Once vfio-pci 0000:0d:00.3: not ready 1023ms after FLR; waiting starts unraid just starts to break quickly then i have to hard power off the PC. I have tried a PCI-E USB Card but the issue with that is the rest of my pci-e slots all are part of IOMMU Group 0 which i have tried ACS downstream with but when passing through i still get vfio-pci : not ready 1023ms after FLR; waiting and unraid hard locks. Any help with this would be great. Thanks, Xerby xiamounraid-diagnostics-20200130-1658.zip xiamounraid-syslog-20200130-2153.zip

Hi Seeringfate, I have recently been running into similar issues with USB controllers, one thing i found that has really helped me is the VFIO-PCI CFG plugin It will list all your IOMMU groups and the nice part is for USB Controllers it will list all connected USB devices and once you find a IOMMU group you can pass through you can select it through the plugin as well. As for the card you installed, What is the brand of it ? is it PCI-E device ? closest thing i see if the VIA Labs link hub