Solved integrated 4 port NIC to pfsense

[scubieman]

I want to pass through 2 of the 4 ports for my integrated 4 port NIC. I have a dell r710 server.

Right now eth1 used to access unraid and unraid things.

I want eth 3 and eth4 for wan and lan for pfsense(vm on unraid)

I tried following spaceinvader videos but wanted to double check.

Image 2.png is normal IOMMU group settings.

Image 1.png is I set VM manger > PCIe ACs override to both.

If I add 14e4:1639 to the flash boot wont it ignore my NIC card? Append vfio-pci.ids=14e4:1639 initrd=/bzroot

 

 

 

Edited February 20, 2019 by scubieman

[scubieman]

Trying using this but no go.

 

 

[scubieman]

I added this and still no go.

 

Here is the error message 

 

[1812]

You're not providing enough information. Need to post diagnostics (tools>diagnostics) after you get the error.

[scubieman]

Here is VM logs. 

2019-02-18T14:22:44.790281Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,id=hostdev0,bus=pci.1,addr=0x0: vfio error: 0000:02:00.0: failed to setup container for group 14: failed to set iommu for container: Operation not permitted
2019-02-18 14:22:44.994+0000: shutting down, reason=failed
2019-02-19 16:57:32.284+0000: starting up libvirt version: 4.7.0, qemu version: 3.0.0, kernel: 4.18.20-unRAID, hostname: Ruby
LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name guest=Pfsense,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-7-Pfsense/master-key.aes -machine pc-q35-3.0,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/235ced1c-02dd-e89c-87fa-04d30b20b7f4_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 3072 -realtime mlock=off -smp 2,sockets=1,cores=1,threads=2 -uuid 235ced1c-02dd-e89c-87fa-04d30b20b7f4 -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=28,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x1harserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=30,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 0.0.0.0:1,websocket=5701 -k en-us -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device vfio-pci,host=02:00.0,id=hostdev0,bus=pci.1,addr=0x0 -device vfio-pci,host=02:00.1,id=hostdev1,bus=pcie.0,addr=0x6 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
2019-02-19 16:57:32.284+0000: Domain id=7 is tainted: high-privileges
2019-02-19 16:57:32.284+0000: Domain id=7 is tainted: host-cpu
2019-02-19T16:57:32.375359Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/1 (label charserial0)
2019-02-19T16:57:32.476359Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,id=hostdev0,bus=pci.1,addr=0x0: vfio error: 0000:02:00.0: failed to setup container for group 14: failed to set iommu for container: Operation not permitted
2019-02-19 16:57:32.479+0000: shutting down, reason=failed

It boots fine if i remove this from the VM XML.

 

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

ruby-diagnostics-20190219-1058.zip

 

Edited February 19, 2019 by scubieman

[1812]

found your problem in the logs, or at least, the next step in your problem:

 

Feb 18 08:09:39 Ruby kernel: vfio_iommu_type1_attach_group: No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

go to the first post in the thread below and modify your syslinux.cfg to what it says in regards to unsafe interrupts, reboot, and try again. If it fails again, post a fresh set of diagnostics immediately after the vm fails. I can tell you from personal experience that "allowing unsafe interrupts" has caused me zero issues.

 

 

[scubieman]

12 hours ago, 1812 said:

found your problem in the logs, or at least, the next step in your problem:
 

Feb 18 08:09:39 Ruby kernel: vfio_iommu_type1_attach_group: No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

 

go to the first post in the thread below and modify your syslinux.cfg to what it says in regards to unsafe interrupts, reboot, and try again. If it fails again, post a fresh set of diagnostics immediately after the vm fails. I can tell you from personal experience that "allowing unsafe interrupts" has caused me zero issues.
 
 

Awesome thank you. I will test and report back

 

Solved: I turned on "allowing unsafe interrupts" and everything is working great. Thank you!

Sent from my Pixel 2 XL using Tapatalk

Edited February 20, 2019 by scubieman