USB devices stop working / Controller passed through

[bigwah]

I've passed through my USB controller (Sonnet Allegro Pro 4-port PCIe card) that gives me 4 separate USB controllers to pass through (found out the hard way there is a version of that card that does not have the 4 separate USB controllers on it).

 

I can see them in the correct grouping

IOMMU group 41:	[1b73:1100] 0c:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 10)
IOMMU group 42:	[1b73:1100] 0d:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 10)
IOMMU group 43:	[1b73:1100] 0e:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 10)
IOMMU group 44:	[1b73:1100] 0f:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 10)

And its been added to my flash config file

append vfio-pci.ids=1b73:1100 initrd=/bzroot,/bzroot-gui

When I go to edit the VM template, at the bottom under 'Other PCI Devices' I see

 Fresco Logic FL1100 USB 3.0 Host Controller | USB controller (0c:00.0)
 Fresco Logic FL1100 USB 3.0 Host Controller | USB controller (0d:00.0)
 Fresco Logic FL1100 USB 3.0 Host Controller | USB controller (0e:00.0)
 Fresco Logic FL1100 USB 3.0 Host Controller | USB controller (0f:00.0)

 

So everything is supposedly configured correctly. Off of each of the 4 ports I have a USB hub with 4 ports to be able to hook up kb/mouse/headset+w/e else. This works all fine and dandy for the most part.

 

But occasionally the USB devices all stop working. The entire VM is not locking up, just the USB devices all die. When this happens I don't get much information from unraid on whats happening. Sometimes I'll get 'internal error: Unknown PCI header type '127' or if I try to reboot/restart the VM I'll get ' Execution error - Requested operation is not valid: domain is already running'.

  

At which point I restarting the VM doesn't work, I need to restart the entire Unraid itself. This happens while in games or just browsing the internet sometimes, can't really force it to happen. Right now it happens about a couple times a day. Usually at the most when I need it to not happen. Pausing the VM does not work either nor does unplugging the USB devices or the hub.

 

Any help in figuring out how to stop it, or recover better from it is greatly appreciated. Thank you.

 

Here is the log from the VM itself, nothing sticks out to me outside of the Domain id=1 is tainted. I see that a lot.

-name guest=Green,debug-threads=on \
-S \
-object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-Green/master-key.aes \
-machine pc-i440fx-3.1,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off \
-cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=none \
-drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on \
-drive file=/etc/libvirt/qemu/nvram/35273121-d030-d51b-8918-afe4e16e1b2c_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 \
-m 6656 \
-realtime mlock=off \
-smp 4,sockets=1,cores=2,threads=2 \
-uuid 35273121-d030-d51b-8918-afe4e16e1b2c \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=26,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.0,addr=0x7 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
-drive file=/mnt/user/domains/Green/vdisk1.img,format=raw,if=none,id=drive-virtio-disk2,cache=writeback \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk2,id=virtio-disk2,bootindex=1,write-cache=on \
-drive file=/mnt/user/domains/Green/vdisk2.img,format=raw,if=none,id=drive-virtio-disk3,cache=writeback \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk3,id=virtio-disk3,write-cache=on \
-netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=30 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:e0:2b:f9,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=31,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x6,romfile=/mnt/user/isos/1050tibios.dump \
-device vfio-pci,host=04:00.1,id=hostdev1,bus=pci.0,addr=0x8 \
-device vfio-pci,host=0e:00.0,id=hostdev2,bus=pci.0,addr=0x9 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2020-03-25 08:08:03.416+0000: Domain id=1 is tainted: high-privileges
2020-03-25 08:08:03.416+0000: Domain id=1 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)

And finally I've attached a diagnostics as well.

black-diagnostics-20200325-1443.zip

Edited March 25, 2020 by bigwah

[MajinBlayze]

Not that i've anything to offer, but I think I'm having this issue as well.

[Andiroo2]

Trying to get this wokring for my VM as well.  It works the first time, but once I reboot the VM I lose the USB card passthrough.  Restarting Unraid fixes it, but I still can't reboot the VM without losing USB.

 

Working on it...will post if i get it working.

[Andiroo2]

I ended up scrapping this idea of passing the USB controller through.  I simply pass through my video card (AMD R570) and I pass through a Logitech USB Unifying receiver for mouse and keyboard.  When I use the VM on my monitor (instead of remote desktop), I switch my mouse and kb to the other dongle and it works like a charm.

 

It's not perfect, but I'm very happy that it works.  

[issabankz]

On 6/18/2020 at 9:00 PM, Andiroo2 said:

I ended up scrapping this idea of passing the USB controller through.  I simply pass through my video card (AMD R570) and I pass through a Logitech USB Unifying receiver for mouse and keyboard.  When I use the VM on my monitor (instead of remote desktop), I switch my mouse and kb to the other dongle and it works like a charm.

 

It's not perfect, but I'm very happy that it works.  

;( man i have the same issue, everything works and suddenly i cant boot the vm with the usb pci card