brinary01

EDIT: Literally the next thing that I saw on the forum was this: Which is exactly what I needed to read, and it worked perfectly with the XML method of pass through. I was so caught up in searching for unRaid spesific stuff that I missed the HP specific stuff.... Hopefully this will serve someone else in the same spot, and further on the HP specific stuff, there is

Hello Everyone, this is my first time posting, please forgive me if I over look something. I have been dealing with this problem for 3 days, but since I poorly documented all my steps, I zeroed everything out and started from scratch to capture the process. In this post I flip between the 4xNIC and the 2xNIC but know I have tried all the listed operations with both NICs, and individual ports as applicable. Frankly, I really only need one port for my pfSense VM, though at least 2 would be nice. That aside... System: HP DL 380e built in 4x Intel NIC and I added a 2x Intel NIC to pass-through to a pfSense VM. I keep getting this error when trying to create a VM with a passed-through NIC or anytime I try to start the VM if I create it without starting it. “internal error: qemu unexpectedly closed the monitor: 2020-10-19T15:39:23.514986Z qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0,id=hostdev0,bus=pci.3,addr=0x0: vfio 0000:02:00.0: failed to setup container for group 18: Failed to set iommu for container: Operation not permitted” From the reading I have done we have 3 ways do pass-through hardware. USB devices for example pass right through (I have not tried to pass-through anything like this as of this writing, but I am about just to see what happens.) The XML method Adding: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </source> </hostdev> This should work for devices like network cards, this is where I started, and it is where the first error was generated, it should be noted that I also that I tried this as a multi function device with the 2xNIC. i.e. <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x0' multifunction='on' /> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/> </source> </hostdev> And made sure that the VM bus, slot and function agreed as you would do passing through a GPU with Sound. Next as you might do with a GPU I tried blacklisting the NIC, before adding it, same error. Just before writing this I tried XML for port 1 of the 2xNIC, and blacklisting port 2 using FVIO-PCI Config, once I did this I got the same error, (note: I also tried every other combination of this between the 2 ports with no success). I should also add that when doing this with the 2XNIC I could always see both ports, but when passing through the 4xNIC it only showed the first Ethernet port. Here you can see my IOMMU Groups for the NICs IOMMU group 18: [8086:1521] 02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 19: [8086:1521] 02:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 20: [8086:1521] 02:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 21: [8086:1521] 02:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) IOMMU group 22: [1590:005f] 06:00.0 Memory controller: Hewlett Packard Enterprise Device 005f IOMMU group 23: [8086:10c9] 07:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) IOMMU group 24: [8086:10c9] 07:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) I did find it a bit odd, every port on each NIC had its own IOMMU group, but while I find it odd, I don't see a reason it should cause and issue, especially if I have blacklisted all ports and passed all to the same VM. I have tried the guidance in this forum post https://forums.unraid.net/topic/73060-same-pci-id-in-multiple-iommu-groups-solved/, as well as allowing unsafe interrupts in the VM Manager, but still the same error. I was thinking there could be a BIOS setting causing hardware to behave in a way that is causing the problem, maybe? I have looked around there but my lack of familiarity with HP serves makes me wary of messing around too much, and information on that is escaping me. I did think for a minute it could be IRQ related since these NICs are sharing IRQs. (side note I have not thought much about IRQs since like 1995, but I did recalled conflict issues back in the day, a quick read of a paper from 2001 reminded me why I have not thought about IRQs in 25 years, and eliminated that as a possibility, mostly). About the VM: I am creating the VM per Space Invaders instructional video on how to make a pfSense VM, but I have also tried various different Vms both linux and Windows with the same result. Any thoughts on where to go next? Thank you all for your support -Brian tower-diagnostics-20201020-0133.zip