NIC Passthrough

Hi I have looked at a lot of posts on here trying to find an answer to my issue but have failed, so I will try and explain my issue.

I have a HP DL380e Gen 8 server that's been running for 12 months with Unraid.

It has 4 integrated NIC's which are fine and I have them bonded and working fine.

 

I am wanting to move my pfSense box to a VM so have purchased a HP NC364T quad NIC.

 

Today I installed the card (and apart from the extra fan noise  ) installed fine.

It was detected in Unraid, I then went through the procedure of trying to get it passed through to a VM.

I tried the normal of adding vfio-pci.ids= (my NIC) and rebooted it then shows as available in the VM but I get the error 

 

internal error: qemu unexpectedly closed the monitor: 2021-06-10T20:47:13.681652Z qemu-system-x86_64: -device vfio-pci,host=0000:0f:00.0,id=hostdev0,bus=pci.3,addr=0x0: vfio 0000:0f:00.0: failed to setup container for group 33: Failed to set iommu for container: Operation not permitted

 

 

my IOMMU was

 

IOMMU group 33: [111d:8018] 0e:02.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e)

                           [8086:10bc] 0f:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06)

                           [8086:10bc] 0f:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06)

IOMMU group 34: [111d:8018] 0e:04.0 PCI bridge: Microsemi / PMC / IDT PES12N3A 12-lane 3-Port PCI Express Switch (rev 0e)

                           [8086:10bc] 10:00.0 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06)

                           [8086:10bc] 10:00.1 Ethernet controller: Intel Corporation 82571EB/82571GB Gigabit Ethernet Controller (Copper) (rev 06)

 

I then used PCIe ACS override:

 

which separated each NIC onto its own group but I still got the same error.

 

I even went down the route of switching my integrated with the 364T and got the same error on the integrated when trying to passthrough.

 

So I am at a loss of what to try next.

 

I have a Windows VM running fine but using br0

I also have a sinology VM using br0

 

so virtualisation is not an issue just passing anything through is.

 

I also had the same issue trying to pass a video card through 12 months ago an Nvidia Quadra K2000.

 

Any help appreciated