***GUIDE*** Passthrough Entire PCI USB Controller


archedraft

Recommended Posts

Hey guys,

 

I have a Gigabyte Aorus Gaming K7 (AX370) Mainboard and Unraid shows only 12 IOMMU groups:

 

PCI Devices and IOMMU Groups

IOMMU group 0:	[1022:1482] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 1:	[1022:1483] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
IOMMU group 2:	[1022:1482] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 3:	[1022:1482] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 4:	[1022:1483] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
IOMMU group 5:	[1022:1482] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 6:	[1022:1482] 00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 7:	[1022:1482] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
[1022:1484] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
[1022:148a] 08:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function
IOMMU group 8:	[1022:1482] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
[1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
[1022:1485] 09:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
[1022:1486] 09:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP
[1022:149c] 09:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
[1022:1487] 09:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller
IOMMU group 9:	[1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61)
[1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
IOMMU group 10:	[1022:1440] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0
[1022:1441] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1
[1022:1442] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2
[1022:1443] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3
[1022:1444] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4
[1022:1445] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5
[1022:1446] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6
[1022:1447] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7
IOMMU group 11:	[1022:43b9] 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X370 Series Chipset USB 3.1 xHCI Controller (rev 02)
[1022:43b5] 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X370 Series Chipset SATA Controller (rev 02)
[1022:43b0] 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X370 Series Chipset PCIe Upstream Port (rev 02)
[1022:43b4] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1022:43b4] 02:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1022:43b4] 02:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1022:43b4] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1b21:1343] 03:00.0 USB controller: ASMedia Technology Inc. ASM1143 USB 3.1 Host Controller
[8086:1539] 04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)
[1969:e0b1] 05:00.0 Ethernet controller: Qualcomm Atheros Killer E2500 Gigabit Ethernet Controller (rev 10)
IOMMU group 12:	[10de:1c03] 07:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 6GB] (rev a1)
[10de:10f1] 07:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)

My goal is to pass a USB Controller to a VM, but each of the 3 USB Controllers has other devices in its group.. In BIOS I enabled ACS (is this correct?). GPU Passes just fine.. Am I trapped with the USB Controllers? Are there any other possibilities (did I miss some BIOS option?), is my last resort some Kernel Patch?

 

Thanks for your advice!

Link to comment
  • 4 weeks later...
On 10/18/2019 at 7:01 PM, glockmane said:

Hey guys,

 

I have a Gigabyte Aorus Gaming K7 (AX370) Mainboard and Unraid shows only 12 IOMMU groups:

 

My goal is to pass a USB Controller to a VM, but each of the 3 USB Controllers has other devices in its group.. In BIOS I enabled ACS (is this correct?). GPU Passes just fine.. Am I trapped with the USB Controllers? Are there any other possibilities (did I miss some BIOS option?), is my last resort some Kernel Patch?

 

Thanks for your advice!

For my x570 Aorus Elite Wifi in BIOS I had to enable ACS Enable and Enable AER Cap to get decent IOMMU grouping. After that I STILL had to set PCIe ACS override to "both" in Unraid. I was able to pass 2 out of 3 usb controllers on my board, including the front panel gen 2 USB Type C!

 

Initially I only had one controller in it's own IOMMU group. That controller would not/could not pass, it would lock up Unraid when I started the VM. After the above changes,  same story for that controller. I switched my Unraid usb to a port on that controller. The other two controllers (now that they are isolated) pass fine!

 

I'm using /boot/config/vfio-pci.cfg to bind those PCI IDs. The USB controllers now show up under "Other PCI Devices" so I can pass them to VMs without editing any XML.

 

Edited by Skitals
  • Like 1
Link to comment
On 11/12/2019 at 4:52 PM, Skitals said:

For my x570 Aorus Elite Wifi in BIOS I had to enable ACS Enable and Enable AER Cap to get decent IOMMU grouping. After that I STILL had to set PCIe ACS override to "both" in Unraid. I was able to pass 2 out of 3 usb controllers on my board, including the front panel gen 2 USB Type C!

 

Initially I only had one controller in it's own IOMMU group. That controller would not/could not pass, it would lock up Unraid when I started the VM. After the above changes,  same story for that controller. I switched my Unraid usb to a port on that controller. The other two controllers (now that they are isolated) pass fine!

 

I'm using /boot/config/vfio-pci.cfg to bind those PCI IDs. The USB controllers now show up under "Other PCI Devices" so I can pass them to VMs without editing any XML.

 

I did the same (dunno if I enabnled AER Cap, have to look later), but I have ACS Enable and ACS Override set as you said and getting much better IOMMU Groups.. Binding like you said in your last sentence looks interesting, can you post the section from your .cfg? Thank you :)

Link to comment
On 11/13/2019 at 11:07 AM, glockmane said:

I did the same (dunno if I enabnled AER Cap, have to look later), but I have ACS Enable and ACS Override set as you said and getting much better IOMMU Groups.. Binding like you said in your last sentence looks interesting, can you post the section from your .cfg? Thank you :)

I created the file /boot/config/vfio-pci.cfg that simply contains:

 

Quote

BIND=06:00.3 06:00.1 06:00.1

Any device you list there is stubbed and available to select under "other pci devices" on the vm xml gui. My understanding is this method supercedes pci-stub and vfio-pci boot flags. Note that if you add or remove pci devices to your system the addresses can/will change. I had to update my vfio-pci.cfg and vm xmls when I pulled my second gpu.

Edited by Skitals
  • Like 1
Link to comment
  • 1 month later...

Has someone experienced problem passing a USB mouse - it works in Windows, but when I start a game and move the mouse - FPS drops from 250 to 50 back and forth. I'm passing USB Wi-Fi keyboard/mouse in 1 port, and a wired USB mouse (Lenovo Y) in another. When moving the Wi-Fi mouse - no issues, when moving wired - fps drop.

 

Tried another wired mouse (Razer) in the same port - it didn't have any issues.

 

Weird.

Link to comment
1 minute ago, ChewbaccaBG said:

The new mouse started doing the same an hour ago...

I have no idea what you mean, wired or wireless. You dont seem very keen to actually tell us what we need to be able to help you. Also you decided to post your query on an UNRELATED thread about passing the whole controller NOT about passing a USB device. Maybe have a rethink about your approach and post a new thread with a little more expansive info and you might get a solution. The community here is helpful but it isn't (yet) psychic

  • Like 1
Link to comment
Just now, ChewbaccaBG said:

If I change the cpufreq governor to "performance", the fps drop still exists, but from ~350 to ~250.

With "ondemand" - from 300 to 50.

 

I can't figure out why this occurs. Perhaps I'll try to pass one of the USB controllers directly into the VM later.


Good idea.. then at least you WOULD be the right thread and wouldnt need to post again (PS edit would have sufficed rather than another post)

Link to comment

So, this is the USB controller I'd like to pass to the VM:

3rd Gen AMD Ryzen™ Threadripper™ Desktop Processors
    - 2 x USB 3.2 Gen 2 ports (1 x Type-C™, 1 x Type A at back panel)
    - 2 x USB 3.2 Gen 1 ports at back panel

 

03:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller
../../../devices/pci0000:00/0000:00:08.1/0000:03:00.3/usb1
../../../devices/pci0000:00/0000:00:08.1/0000:03:00.3/usb2

 

IOMMU group 9:     [1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]

IOMMU group 15:    [1022:148c] 03:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller

 

Since the 1st post is 5 years old, vfio-bind doesn't exist in Unraid 6.8..

What do I need to do?

 

Here's my current config (1 GPU passthrough):

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 10</name>
  <uuid>f0f6d8a1-ff05-65df-7cd5-69d6624a3f9b</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>20971520</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>24</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='24'/>
    <vcpupin vcpu='2' cpuset='1'/>
    <vcpupin vcpu='3' cpuset='25'/>
    <vcpupin vcpu='4' cpuset='2'/>
    <vcpupin vcpu='5' cpuset='26'/>
    <vcpupin vcpu='6' cpuset='3'/>
    <vcpupin vcpu='7' cpuset='27'/>
    <vcpupin vcpu='8' cpuset='4'/>
    <vcpupin vcpu='9' cpuset='28'/>
    <vcpupin vcpu='10' cpuset='5'/>
    <vcpupin vcpu='11' cpuset='29'/>
    <vcpupin vcpu='12' cpuset='6'/>
    <vcpupin vcpu='13' cpuset='30'/>
    <vcpupin vcpu='14' cpuset='7'/>
    <vcpupin vcpu='15' cpuset='31'/>
    <vcpupin vcpu='16' cpuset='8'/>
    <vcpupin vcpu='17' cpuset='32'/>
    <vcpupin vcpu='18' cpuset='9'/>
    <vcpupin vcpu='19' cpuset='33'/>
    <vcpupin vcpu='20' cpuset='10'/>
    <vcpupin vcpu='21' cpuset='34'/>
    <vcpupin vcpu='22' cpuset='11'/>
    <vcpupin vcpu='23' cpuset='35'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-q35-4.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/f0f6d8a1-ff05-65df-7cd5-69d6624a3f9b_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='24' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='yes'/>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk2.img'/>
      <target dev='hdd' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.173.iso'/>
      <target dev='hdb' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0xb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0xc'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0xd'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0xe'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0xf'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x7'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='9' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
    <controller type='pci' index='10' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:08:d9:27'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/boot/vBIOS/ASUS-ROG-STRIX-nVidia-GTX1050Ti-GP107-KVM.rom'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

Edited by ChewbaccaBG
Link to comment
12 minutes ago, ChewbaccaBG said:

So, this is the USB controller I'd like to pass to the VM:

3rd Gen AMD Ryzen™ Threadripper™ Desktop Processors
    - 2 x USB 3.2 Gen 2 ports (1 x Type-C™, 1 x Type A at back panel)
    - 2 x USB 3.2 Gen 1 ports at back panel

 

03:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller
../../../devices/pci0000:00/0000:00:08.1/0000:03:00.3/usb1
../../../devices/pci0000:00/0000:00:08.1/0000:03:00.3/usb2

 

IOMMU group 9:     [1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]

IOMMU group 15:    [1022:148c] 03:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller

 

Since the 1st post is 5 years old, vfio-bind doesn't exist in Unraid 6.8..

What do I need to do?

 

Please do make efforts to read the thread not just give up after 1st post is 5 years old :( 

 

You have (sorry) posted only an incomplete section of your groupings (I suspect you have more than 2 controllers!). The principles are laid out here clearly in thread. Make sure you have separate groupings, make sure your unRAID USB is NOT on a controller you decide to pass. Then stub ALL the items in that group (Sorry if this isn't a spoonfeed answer but I'm not running unRAID now, so not sure of latest method but it IS in the thread) and then reboot server and in the VM template you will have those stubbed items appear as passable PCI devices. Pass all the bits and then you should be golden. 

Link to comment
1 minute ago, ChewbaccaBG said:

Thank you so much @methanoid you helped fuck up all my VMs.

Now I guess I'd have to reinstall unraid completely just to get libvirt working agian.

 

THANK YOU

 

Please dont blame me when you failed to ever provide the requested IOMMU groupings. I even PM'd you to explain how you failed to help yourself. Nothing I have suggested would have led to your VMs being deleted. 

 

I note you editted your brief previous message to add some detail but still you never provide the requested info. You wont help yourself but you want to blame me. Good show. Good luck getting anyone else to help with that attitude.  Community here is good but being rude and deliberately difficulty is not the way to get help.

Link to comment
1 minute ago, ChewbaccaBG said:

Well, right now I ain't got any VMs at all so.. you've heard of any such problem ?

You could try posting your IOMMU groupings (not your VM XML yet) and your unRAID startup options (visible at the Flash sub screen). Maybe someone else will pipe up to help. I am done wasting my time trying to help and getting pissy posts. 

Link to comment
Just now, methanoid said:

You could try posting your IOMMU groupings (not your VM XML yet) and your unRAID startup options (visible at the Flash sub screen). Maybe someone else will pipe up to help. I am done wasting my time trying to help and getting pissy posts. 

You aren't even running unraid, as you said. Why were you trying to "help" anyway?

Link to comment

So, here is it again, full information as was requested above.

 

All USB controller(s) are integrated on the mobo:
ASUSTeK COMPUTER INC. PRIME TRX40-PRO, Version Rev 1.xx, BIOS Version 0702 (latest)

 

Current syslinux config (8086:10c9 is a GPU):
kernel /bzimage
append amd_iommu=on iommu=pt pci-stub.ids=8086:10c9 video=efifb:off isolcpus=0-11,24-35 vfio_iommu_type1.allow_unsafe_interrupts=1 rd.driver.pre=vfio-pci modprobe.blacklist=radeon,amdgpu,fglrx,nouveau,nvidiafb,nvidia,nvidia_drm,snd-pcsp initrd=/bzroot

 

Using the script from https://www.youtube.com/watch?v=UilWLtMYd-U

 

Bus 1 --> 0000:03:00.3 (IOMMU group 15)
Bus 001 Device 003: ID 062a:4c01 MosArt Semiconductor Corp. 2.4G INPUT DEVICE <- wifi mouse & keyboard (dongle)
Bus 001 Device 002: ID 1532:005c Razer USA, Ltd Razer DeathAdder Elite <- wired mouse
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 2 --> 0000:03:00.3 (IOMMU group 15)
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 3 --> 0000:22:00.3 (IOMMU group 28)
Bus 003 Device 002: ID 8564:1000 Transcend Information, Inc. JetFlash <- test USB stick
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 4 --> 0000:22:00.3 (IOMMU group 28)
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 5 --> 0000:46:00.1 (IOMMU group 44)
Bus 005 Device 003: ID 0b05:1918 ASUSTek Computer, Inc. USB Audio <- unknown
Bus 005 Device 004: ID 0b05:18f3 ASUSTek Computer, Inc.
Bus 005 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 6 --> 0000:46:00.1 (IOMMU group 44)
Bus 006 Device 002: ID 0bc2:2321 Seagate RSS LLC Expansion Portable <- "unassigned" disk
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 7 --> 0000:46:00.3 (IOMMU group 44)
Bus 007 Device 002: ID 174c:2074 ASMedia Technology Inc. ASM1074 High-Speed hub
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 8 --> 0000:46:00.3 (IOMMU group 44)
Bus 008 Device 003: ID 13fe:6400 Kingston Technology Company Inc. <- unRAID usb stick
Bus 008 Device 002: ID 174c:3074 ASMedia Technology Inc. ASM1074 SuperSpeed hub
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub


IOMMU group 15
[RESET] 03:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
IOMMU group 28
[RESET] 22:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
IOMMU group 44
[RESET] 42:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
[RESET] 46:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        46:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
[RESET] 46:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]


The controller I'd like to pass to the VM is in IOMMU groups 15 & 28 (Bus 1,2,3,4).
It's the "3rd Gen AMD Ryzen™ Threadripper™ Desktop Processors", according to the MOBO manual:
    - 2 x USB 3.2 Gen 2 ports (1 x Type-C™, 1 x Type A at back panel)
    - 2 x USB 3.2 Gen 1 ports at back panel

 

Last time I attempted to pass these to the VM by using: vfio-pc.ids=1022:148c (and stub as well) the VM did not start at all, ~5 minutes later - the server went w/o network (after responding with pings of 900~1900ms for about 2 minutes) and had to be rebooted manually (no pings response at all), which then resulted in libvirt failed to start.

 

vfio-pci.ids=1022:148c

AMD Starship USB 3.0 Host Controller | USB controller (03:00.3)
AMD Starship USB 3.0 Host Controller | USB controller (22:00.3)

Both 03:00.3 and 22:00.3 passed to the vm:
Jan  6 10:22:28 beast kernel: vfio-pci 0000:03:00.3: not ready 1023ms after FLR; waiting
Jan  6 10:22:30 beast kernel: vfio-pci 0000:03:00.3: not ready 2047ms after FLR; waiting
Jan  6 10:22:33 beast kernel: vfio-pci 0000:03:00.3: not ready 4095ms after FLR; waiting
Jan  6 10:22:39 beast kernel: vfio-pci 0000:03:00.3: not ready 8191ms after FLR; waiting
Jan  6 10:22:48 beast kernel: vfio-pci 0000:03:00.3: not ready 16383ms after FLR; waiting
Jan  6 10:23:05 beast kernel: vfio-pci 0000:03:00.3: not ready 32767ms after FLR; waiting
Jan  6 10:23:41 beast kernel: clocksource: timekeeping watchdog on CPU36: Marking clocksource 'tsc' as unstable because the skew is too large:
Jan  6 10:23:41 beast kernel: clocksource:                       'hpet' wd_now: b6ece015 wd_last: b53757aa mask: ffffffff
Jan  6 10:23:41 beast kernel: clocksource:                       'tsc' cs_now: d4be71c208 cs_last: d3dc1106e4 mask: ffffffffffffffff
Jan  6 10:23:41 beast kernel: tsc: Marking TSC unstable due to clocksource watchdog
Jan  6 10:23:41 beast kernel: TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'.
Jan  6 10:23:41 beast kernel: sched_clock: Marking unstable (212891471252, -8998045)<-(213040103613, -157630481)



Only 22:00.3 passed to the vm (tsc unstable again, just didn't copy it)
Jan  6 10:28:21 beast kernel: vfio-pci 0000:22:00.3: not ready 1023ms after FLR; waiting
Jan  6 10:28:23 beast kernel: vfio-pci 0000:22:00.3: not ready 2047ms after FLR; waiting
Jan  6 10:28:26 beast kernel: vfio-pci 0000:22:00.3: not ready 4095ms after FLR; waiting
Jan  6 10:28:31 beast kernel: vfio-pci 0000:22:00.3: not ready 8191ms after FLR; waiting

 

Tried passing vfio-pc.ids=1022:1485,1022:149c
IOMMU group 44:    [1022:57a4] 42:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4
    [1022:1485] 46:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
    [1022:149c] 46:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
    [1022:149c] 46:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
    
The device 1022:1485 appears also in several other IOMMU groups by itself:
IOMMU group 14:    [1022:1485] 03:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
IOMMU group 26:    [1022:1485] 22:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
IOMMU group 52:    [1022:1485] 4b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
IOMMU group 63:    [1022:1485] 62:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP

 

The USB controller (ASMedia) was passed successfully.

 

I'll try to pass the CPU-bound controller again, this time: vfio-pc.ids=1022:1485,1022:148c - didn't work.

 

So, if someone's got an idea on how to pass 1022:148c - I'm open to any ideas.

Edited by ChewbaccaBG
Link to comment

Thank you for this tutorial.  It really helped me to figure out which IOMMU group of USB controllers to passthrough to my VM without catching my UNRAID thumbdrive in the mess.  I was able to pass my entire frontpanel through as well as both USB2 headers internally and I even wrote the XML right the first time.

Link to comment
  • 4 weeks later...
  • 3 weeks later...

@methanoid 

@saarg I'm reaching out to you since you two have been phenomenally helpful so far on this thread with your comments and would love to get your input on what I'm doing wrong with this passthrough.

I followed Spaceinvader's video to a T and read through every comment on this thread, but sadly, my Unraid won't boot unless it's booted in GUI Mode after adding the vfio code to the syslinux config.

 

My USB hub is in its own IOMMU group. Is there anything I'm missing?

 

Syslinux config:

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label Unraid OS
  menu default
  kernel /bzimage
  append vfio-pci.ids=1022:149c initrd=/bzroot
label Unraid OS GUI Mode
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui
label Unraid OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Unraid OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

 

 

All USB Controllers Busses

Bus 1 --> 0000:07:00.1 (IOMMU group 21)
Bus 001 Device 002: ID 0781:5571 SanDisk Corp. Cruzer Fit
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 2 --> 0000:07:00.1 (IOMMU group 21)
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 3 --> 0000:07:00.3 (IOMMU group 21)
Bus 003 Device 003: ID 0b05:18f3 ASUSTek Computer, Inc. AURA LED Controller
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 4 --> 0000:07:00.3 (IOMMU group 21)
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 5 --> 0000:0c:00.3 (IOMMU group 31)
Bus 005 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 6 --> 0000:0c:00.3 (IOMMU group 31)
Bus 006 Device 002: ID 05e3:0616 Genesys Logic, Inc. hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

 

IOMMU Groups -- (Group 31) Controller is in its own group

IOMMU group 0:	[1022:1482] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 1:	[1022:1483] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
IOMMU group 2:	[1022:1483] 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
IOMMU group 3:	[1022:1482] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 4:	[1022:1482] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 5:	[1022:1483] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
IOMMU group 6:	[1022:1482] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 7:	[1022:1482] 00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 8:	[1022:1482] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 9:	[1022:1484] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
IOMMU group 10:	[1022:1482] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 11:	[1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
IOMMU group 12:	[1022:1484] 00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
IOMMU group 13:	[1022:1484] 00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
IOMMU group 14:	[1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61)
[1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
IOMMU group 15:	[1022:1440] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0
[1022:1441] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1
[1022:1442] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2
[1022:1443] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3
[1022:1444] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4
[1022:1445] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5
[1022:1446] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6
[1022:1447] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7
IOMMU group 16:	[8086:f1a8] 01:00.0 Non-Volatile memory controller: Intel Corporation SSD 660P Series (rev 03)
IOMMU group 17:	[1022:57ad] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream
IOMMU group 18:	[1022:57a3] 03:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
IOMMU group 19:	[1022:57a3] 03:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
IOMMU group 20:	[1022:57a3] 03:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
IOMMU group 21:	[1022:57a4] 03:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
[1022:1485] 07:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
[1022:149c] 07:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
[1022:149c] 07:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
IOMMU group 22:	[1022:57a4] 03:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
[1022:7901] 08:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
IOMMU group 23:	[1022:57a4] 03:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
[1022:7901] 09:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
IOMMU group 24:	[1cc1:8201] 04:00.0 Non-Volatile memory controller: ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive (rev 03)
IOMMU group 25:	[10de:1c02] 05:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] (rev a1)
[10de:10f1] 05:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)
IOMMU group 26:	[10ec:8168] 06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 26)
IOMMU group 27:	[10de:1184] 0a:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 770] (rev a1)
[10de:0e0a] 0a:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)
IOMMU group 28:	[1022:148a] 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function
IOMMU group 29:	[1022:1485] 0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
IOMMU group 30:	[1022:1486] 0c:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP
IOMMU group 31:	[1022:149c] 0c:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
IOMMU group 32:	[1022:1487] 0c:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller
IOMMU group 33:	[1022:7901] 0d:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
IOMMU group 34:	[1022:7901] 0e:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)

 

Link to comment
47 minutes ago, vvolfpack said:

@methanoid 

@saarg I'm reaching out to you since you two have been phenomenally helpful so far on this thread with your comments and would love to get your input on what I'm doing wrong with this passthrough.

I followed Spaceinvader's video to a T and read through every comment on this thread, but sadly, my Unraid won't boot unless it's booted in GUI Mode after adding the vfio code to the syslinux config.

 

My USB hub is in its own IOMMU group. Is there anything I'm missing?

 

Syslinux config:


default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label Unraid OS
  menu default
  kernel /bzimage
  append vfio-pci.ids=1022:149c initrd=/bzroot
label Unraid OS GUI Mode
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui
label Unraid OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Unraid OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

 

 

All USB Controllers Busses


Bus 1 --> 0000:07:00.1 (IOMMU group 21)
Bus 001 Device 002: ID 0781:5571 SanDisk Corp. Cruzer Fit
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 2 --> 0000:07:00.1 (IOMMU group 21)
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 3 --> 0000:07:00.3 (IOMMU group 21)
Bus 003 Device 003: ID 0b05:18f3 ASUSTek Computer, Inc. AURA LED Controller
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 4 --> 0000:07:00.3 (IOMMU group 21)
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 5 --> 0000:0c:00.3 (IOMMU group 31)
Bus 005 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 6 --> 0000:0c:00.3 (IOMMU group 31)
Bus 006 Device 002: ID 05e3:0616 Genesys Logic, Inc. hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

 

IOMMU Groups -- (Group 31) Controller is in its own group


IOMMU group 0:	[1022:1482] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 1:	[1022:1483] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
IOMMU group 2:	[1022:1483] 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
IOMMU group 3:	[1022:1482] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 4:	[1022:1482] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 5:	[1022:1483] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
IOMMU group 6:	[1022:1482] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 7:	[1022:1482] 00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 8:	[1022:1482] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 9:	[1022:1484] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
IOMMU group 10:	[1022:1482] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
IOMMU group 11:	[1022:1484] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
IOMMU group 12:	[1022:1484] 00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
IOMMU group 13:	[1022:1484] 00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
IOMMU group 14:	[1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61)
[1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
IOMMU group 15:	[1022:1440] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0
[1022:1441] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1
[1022:1442] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2
[1022:1443] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3
[1022:1444] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4
[1022:1445] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5
[1022:1446] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6
[1022:1447] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7
IOMMU group 16:	[8086:f1a8] 01:00.0 Non-Volatile memory controller: Intel Corporation SSD 660P Series (rev 03)
IOMMU group 17:	[1022:57ad] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream
IOMMU group 18:	[1022:57a3] 03:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
IOMMU group 19:	[1022:57a3] 03:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
IOMMU group 20:	[1022:57a3] 03:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
IOMMU group 21:	[1022:57a4] 03:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
[1022:1485] 07:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
[1022:149c] 07:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
[1022:149c] 07:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
IOMMU group 22:	[1022:57a4] 03:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
[1022:7901] 08:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
IOMMU group 23:	[1022:57a4] 03:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
[1022:7901] 09:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
IOMMU group 24:	[1cc1:8201] 04:00.0 Non-Volatile memory controller: ADATA Technology Co., Ltd. XPG SX8200 Pro PCIe Gen3x4 M.2 2280 Solid State Drive (rev 03)
IOMMU group 25:	[10de:1c02] 05:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] (rev a1)
[10de:10f1] 05:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)
IOMMU group 26:	[10ec:8168] 06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 26)
IOMMU group 27:	[10de:1184] 0a:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 770] (rev a1)
[10de:0e0a] 0a:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)
IOMMU group 28:	[1022:148a] 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function
IOMMU group 29:	[1022:1485] 0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
IOMMU group 30:	[1022:1486] 0c:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP
IOMMU group 31:	[1022:149c] 0c:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
IOMMU group 32:	[1022:1487] 0c:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller
IOMMU group 33:	[1022:7901] 0d:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
IOMMU group 34:	[1022:7901] 0e:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)

 

The reason is that you have your unraid usb on the same usb controllers that you use vfio-pci.ids on.

Link to comment
7 hours ago, saarg said:

unraid usb on the same usb controllers that you use vfio-pci.ids on

I have 3 USB 3.0 controllers with legacy support. I also have XHCI enabled in my BIOS. My motherboard has 2x 2.0 ports and 4x 3.0 ports. Unraid is plugged into a 2.0 port

 

07:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
07:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
0c:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller

My Unraid's on Bus 1 (IOMMU Group 21 —07:00.1), whereas the Genesys USB Hub is on Bus 6 (Group 31 — 0c:00.3). 

 

How might I solve for this?

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.