I've been having issues trying to get a new QNAP QXG-2G4T-I225 card passed through into a pfSense/freeBSD VM.
Using the Intel I225-LM, it's a PCIEx4 card with 4 x 2.5Gbe ports, each with it's own adaptor paired with a ASMedia Technology Inc. Device 1812 PCI bridge (although there seem to be 7 bridges across 7 IOMMU groups?).
The system is using an ASUS Z390-e mobo with VT-d and IO-SV enabled in BIOS (although I've tried the below with both IO-SV enabled and disabled).
IOMMU groups as shown (truncated list, just showing the ones associated with the card).
Any time I've tried, I get the dreaded "internal error: Unknown PCI header type '127' for device '0000:0X:00.0'".
Here's what I've tried so far:
Binding the individual Ethernet Controllers to VFIO through the System Devices page.
When doing this, the ethernet controllers show up with "!!! Unknown header type 7f" when hovering over the device on the list, even after boot.
NOTE: When not binding the controllers to VFIO, they present as:
[8086:15f2] 09:00.0 Ethernet controller: Intel Corporation Ethernet Controller I225-LM (rev 03) and work fine. Note the revision change, not sure why that is.
Composite image below shows 3 out of the 4 controllers being bound to VFIO.
Put "pci-stub.ids=8086:15f2" in the syslinux config file.
When I do this, the controllers seem to start fine as shown in the System Devices *until* I try and start a VM with them, then they show "!!! Unknown header type 7f"
I also tried "pci-stub.ids=8086:15f2,1b21:1812" (added the PCI bridge)
Put "xen-pciback.hide=(05:00.0)(06:00.0)(08:00.0)(09:00.0) in the syslinux config file.
Same issues as the pci-stubs.ids method: seems to work *until* I try to start the pfSense
Also tried adding in the PCI bridges as well and it has the same effect.
Tried using the PCIe ACS overrides to no avail, although it *does* separate the ethernet controllers and PCI bridges to separate IOMMU groups.
Tried VFIO allow safe interrupts to no avail (not that I'd want to have this on 24/7)
I haven't had any issues with the 'other devices within the same IOMMU group are in use' issues that you can have and the card works fine in Unraid itself.
I had no issues with my Dell THGMP Intel I350-T4 Quad Port card in the same setup, be it passing in the whole card or half the ports, so I don't think it's related to my mobo and VT-d support.
I'm stumped. Anyone got any ideas what's up?