SOLVED - Gpu passthrough iommu issues tried the usual solutions..

[RIzzaz]

Hi there 

 

I'm playing with setting up a HP server, running into a number of issues... one is in passing through a nvidia GPU ..

 

i get

 

Execution error

internal error: qemu unexpectedly closed the monitor: 2017-07-26T23:25:04.723285Z qemu-system-x86_64: -device vfio-pci,host=08:00.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to set iommu for container: Operation not permitted
2017-07-26T23:25:04.723322Z qemu-system-x86_64: -device vfio-pci,host=08:00.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to setup container for group 24
2017-07-26T23:25:04.723329Z qemu-system-x86_64: -device vfio-pci,host=08:00.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to get group 24
2017-07-26T23:25:04.723345Z qemu-system-x86_64: -device vfio-pci,host=08:00.0,id=hostdev0,bus=pci.0,addr=0x5: Device initialization failed

 

i have tried enabling CIe ACS Override

 

it's in its own group 

 

IOMMU group 24
	[10de:1d01] 08:00.0 VGA compatible controller: NVIDIA Corporation GP108 (rev a1)
	[10de:0fb8] 08:00.1 Audio device: NVIDIA Corporation Device 0fb8 (rev a1)

 

I've added the line vfio_iommu_type1.allow_unsafe_interrupts=1 

 

default /syslinux/menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append pcie_acs_override=downstream vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot
label unRAID OS GUI Mode
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot,/bzroot-gui
label unRAID OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot unraidsafemode
label unRAID OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

 

I am passing through the audio as well as the GPU

 

I have tried swapping to a different PCIe riser...

 

 

out of ideas!

 

 

bikecafe-diagnostics-20170726-2222.zip

Edited August 2, 2017 by RIzzaz

[RIzzaz]

Anyone got any ideas ?

 

Thanks

[1812]

first confirm in the bios that video is set for to onboard graphics primary, the gpu secondary.

 

then you need to stub your nvida gpu. and reboot.

 

then try to launch the vm.

 

if that doens't work, then you need to try  rolling back your bios to an older one. I don't see the rmrr issue discussed below in your logs, but I've always had issues with newer bios.

 

 

Also consider downgrading unRaid to 6.2.4. 6.3.x did not play nice with my servers and I went back. The 6.4 release clients operate well on my test server (for the most part) but it's a release client and not considered stable.

 

[RIzzaz]

 

Thanks 

 

I went with the release candidate version of unraid and

 

The solution seemed to be to use the backup bios which is from 2010 this worked,

 

no need to stub the graphics driver 

 

many thanks, all seems good now!