Skip to content
View in the app

A better way to browse. Learn more.
Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.
Unraid Summer Sale in live: 20% off Starter and Unleashed + buy one, get one 50% off

Supermicro A2SDi-8C-HLN4F - USB PCIe Passthrough

tatec
in VM Engine (KVM)

Featured Replies

Hello everyone,

 

maybe someone can point me in the right direction. I don't know anything else to do.

The last couple of days I was trying to passthrough a USB PCIe card for my VMs.

 

My setup:

- Unraid 6.8.3

setup8bklf.png

 

According to the A2SDi manual the board supports Intel Virtualization Technology for Directed I/O (Intel VT-X, VT-d)

 

Cards I tried so far:

- KALEA-INFORMATIQUE Controller Card PCI Express (PCI-E) to 3.1 USB (Chipset: Asmedia ASM1142)

- Inateck USB 3.0 PCIe Karte KT4006 (Chipset: FL1100)

 

The following steps were done with both cards:

After plugging in a card in the PCIe x4 slot I boot up Unraid without a problem. The USB card is detected by the system and has its own IOMMU group 16.

iommukmkvr.png

 

Following the tutorial  from @SpaceInvaderOne I double-checked if the card is in its own group and if it is able to reset. (Which both cards are)

resetayjk2.png

 

Then I set the vfio-pci.ids=1b73:1100 (for the fresco card) according to the video in the syslinux file. Later I also tried using the "VFIO-PCI Config" Plugin.

Both ways seem to work as expected and after a restart the card is bound.

 

For the last step I then select the card in one of my VMs and start the VM it. This is when things startet to act strange...

 

-------------------------------------

 

1) KALEA-INFORMATIQUE Controller Card PCI Express (PCI-E) to 3.1 USB (Chipset: Asmedia ASM1142):

I start one of my VMs and the following messages pop up in my log files: 

Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio_cap_init: 0000:02:00.0 hiding cap 0xff
Nov 11 21:19:32 Tower kernel: vfio-pci 0000:02:00.0: timed out waiting for pending transaction; performing function level reset anyway
Nov 11 21:19:33 Tower kernel: vfio-pci 0000:02:00.0: not ready 1023ms after FLR; waiting
Nov 11 21:19:35 Tower kernel: vfio-pci 0000:02:00.0: not ready 2047ms after FLR; waiting
Nov 11 21:19:37 Tower kernel: vfio-pci 0000:02:00.0: not ready 4095ms after FLR; waiting
Nov 11 21:19:41 Tower kernel: vfio-pci 0000:02:00.0: not ready 8191ms after FLR; waiting
Nov 11 21:19:50 Tower kernel: vfio-pci 0000:02:00.0: not ready 16383ms after FLR; waiting
Nov 11 21:20:07 Tower kernel: vfio-pci 0000:02:00.0: not ready 32767ms after FLR; waiting
Nov 11 21:20:43 Tower kernel: vfio-pci 0000:02:00.0: not ready 65535ms after FLR; giving up

So far my research tells me that these messages appear when the card can not be resetted (which it normally should do)

In order to try again I then have to restart my server, because of the "Unknown PCI header type '127' for device '0000:02:00.0'"

As I understand: the card isn't resetting...

 

-------------------------------------

 

2) Inateck USB 3.0 PCIe Karte KT4006 (Chipset: FL1100):

The Inateck card looked more promising, but failed anyways with a slight change in the log messages. 

Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:20 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:21 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:21 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:21 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:21 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:21 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:21 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars
Nov 12 13:57:21 Tower kernel: vfio_bar_restore: 0000:02:00.0 reset recovery - restoring bars

In the end, I also have to restart my server because the card isn't resetting itself.

So far I wasn't able to find an answer what exactly those messages try to tell me.

 

-------------------------------------

 

What I tried to solve the problem:

Set PCIe overdrive to downstream, multifunction or both

Enabled VFIO allow unsafe interrupts (and disabled it in the bios)

Looked in the bios if any setting is disabled/enabled which could cause the error.

 

Any help is really appreciated. Maybe one of you is able to help me.

If any further information helps just tell me and I will post it as soon as possible!

 

Best regards

tatec

 

  • Quote
    • Author

    Today I changed from UEFI to legacy. Still no success.

     

    I also attached the diagnostics. Maybe someone can spot anything.

     

    Best regards

    tatec

    tower-diagnostics-20201113-1327.zip

  • Quote
    • Author

    No one got any feedback? Or any input which settings etc. I can try?

  • Quote
    • Author

    Did Bios update - no changes...

     

    Anyone?

  • Quote

    • Archived

    This topic is now archived and is closed to further replies.

    Go to topic listing

    Account

    Navigation

    Search

    Search

    Configure browser push notifications
    Chrome (Android)
    1. Tap the lock icon next to the address bar.
    2. Tap Permissions → Notifications.
    3. Adjust your preference.
    Chrome (Desktop)
    1. Click the padlock icon in the address bar.
    2. Select Site settings.
    3. Find Notifications and adjust your preference.