PCI USB Controller Windows 10 VM Passthrough


Recommended Posts

Good day Unraid community,

 

I've successfully gotten through Part 1 and most of Part 2 of @SpaceInvaderOne's "best way to install and setup a windows 10 vm" videos. I also followed the video to pass-through my recently purchased/installed PCIe USB3 expansion card, and, as you can see below, it's available as one of the Other PCI Devices.

image.png.cfef257a70e607dca0137b2265817f1d.png

 

My issue occurs when I tick it and try to start my VM, it freezes getting stuck on the BIOS boot screen. I checked the logs before I selected this device and after and the only difference in the boot sequence is the following error:

qemu-system-x86_64: vfio_err_notifier_handler(0000:04:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest

 

I searched for the vfio_err_notifier_handler in this forum, however did not see any posts with resolutions other than the card not being able to be passed through. To be clear, this device's sole purpose will be a dedicated USB controller pass-through for my Wind10 VM so the Unraid OS stick is not plugged into it.

 

Can someone please:

1) let me know if I've done something wrong? or

2) if I've done nothing wrong in following the directions but this card just won't work, recommend a card that would work?

 

Thanks!

Link to comment
5 hours ago, bastl said:

@bsimonian Can you please post your IOMMU groupings found under tools >> system devices.

@bastl please see below; IOMMU group 12:[1912:0014] is what I added to the syslinux configuration

 

IOMMU group 0:[8086:3ec2] 00:00.0 Host bridge: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers (rev 0d)

IOMMU group 1:[8086:1901] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) (rev 0d)

[10de:13c2] 01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)

[10de:0fbb] 01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)

IOMMU group 2:[8086:3e98] 00:02.0 VGA compatible controller: Intel Corporation UHD Graphics 630 (Desktop 9 Series) (rev 02)

IOMMU group 3:[8086:a379] 00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10)

IOMMU group 4:[8086:a36d] 00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10)

[8086:a36f] 00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)

IOMMU group 5:[8086:a360] 00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10)

IOMMU group 6:[8086:2822] 00:17.0 RAID bus controller: Intel Corporation SATA Controller [RAID mode] (rev 10)

IOMMU group 7:[8086:a340] 00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0)

IOMMU group 8:[8086:a338] 00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0)

IOMMU group 9:[8086:a33f] 00:1c.7 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #8 (rev f0)

IOMMU group 10:[8086:a330] 00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0)

IOMMU group 11:[8086:a305] 00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev 10)

[8086:a348] 00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)

[8086:a323] 00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10)

[8086:a324] 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller (rev 10)

[8086:15bc] 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-V (rev 10)

IOMMU group 12:[1912:0014] 04:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03)

IOMMU group 13:[144d:a808] 05:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983

Link to comment

@bsimonian I wanted to make sure the USB controller you wanna use in the VM is in it's own separated IOMMU group. And clearly it is. It's a common mistake people make.

9 hours ago, bsimonian said:

it freezes getting stuck on the BIOS boot screen

Does it get stuck on the TianoCore screen or do you see the Uefi shell from the VM? Maybe try the following if you see the shell

 

https://wiki.unraid.net/index.php/UnRAID_OS_version_6_Upgrade_Notes#My_OVMF_VM_doesn.27t_boot_correctly

 

and post your xml from that VM inside a code block ("</>" in the toolbar above the form) for better readability

 

Edited by bastl
Link to comment
8 minutes ago, bastl said:

I wanted to make sure the USB controller you wanna use in the VM is in it's own separated IOMMU group. And clearly it is. It's a common mistake people make.

 @bastl I figured that's what you were checking; I read/watch all the tutorial videos very very carefully :)

 

4 minutes ago, bastl said:

Does it get stuck on the TianoCore screen or do you see the Uefi shell from the VM?

It gets stuck on the TianoCore screen before the spinny-whirly Windows doo-hickey starts going (yes, official term I've coined for it). Not sure if the full log file will help but I've attached just in case

 

Win10VM_log.txt

Link to comment
31 minutes ago, bastl said:

post your xml from that VM inside a code block ("</>" in the toolbar above the form) for better readability

Sorry, missed that the first time; see below:

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 10</name>
  <uuid>931922d8-9aab-38d4-6d0f-9e9136ef7ff3</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='1'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='3'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-4.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/931922d8-9aab-38d4-6d0f-9e9136ef7ff3_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='1'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Windows10.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.173-2.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:81:10:37'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </sound>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0x0841'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc52b'/>
      </source>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x048d'/>
        <product id='0x8297'/>
      </source>
      <address type='usb' bus='0' port='4'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1532'/>
        <product id='0x021e'/>
      </source>
      <address type='usb' bus='0' port='5'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

Link to comment

@bastl No, there are 3 devices (Logitech Unifying Receiver, Logitech Webcam, Razer Keyboard) that are plugged into the main motherboard USB plugs in the back and the Unraid stick plugged into one of the 4 plugs at the top of the case (Fractal Design Define R5 case).

 

The Integrated Technology Express ITE device, from what I've been able to piece together from a few forums, is an RGB controller. Regardless, I have the same issue with the same error whether I have it checked to pass-through or not.

Link to comment

Just a follow-up and close out this issue, turns out the PCIe USB card I was using is, for some reason, not compatible with Unraid and so would not work regardless of what I tried. I found THIS post by @CHBMB and, thanks to his research and testing of the various PCIe USB cards, I ended up ordering the Inateck KT5001.

 

I just installed it, went through the process to use it as a passthrough device, and the VM booted up just fine with success in read/write/delete/etc from a USB stick and another USB drive.

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.