Sorry if this has been covered I have searched and am still unsure as to whether this can work on the Gen 8 

 

It has a 1256l v2 CPU and I would like to passthrough an nvidia 120 GPU to a windows 10 or 7 VM on Unraid but get this error 

Execution error

internal error: qemu unexpectedly closed the monitor: 2018-08-26T19:44:31.015252Z qemu-system-x86_64: -device vfio-pci,host=07:00.0,id=hostdev0,x-vga=on,bus=pci.0,addr=0x5: vfio error: 0000:07:00.0: failed to setup container for group 11: failed to set iommu for container: Operation not permitted

 

 

PCI Devices and IOMMU Groups

Warning: Your system has booted with the PCIe ACS Override setting enabled. The below list doesn't not reflect the way IOMMU would naturally group devices. To see natural IOMMU groups for your hardware, go to the VM Settings page and set the PCIe ACS Override setting to No.

IOMMU group 0:	[8086:0158] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/Ivy Bridge DRAM Controller (rev 09)
IOMMU group 1:	[8086:0151] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
IOMMU group 2:	[8086:015d] 00:06.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)
IOMMU group 3:	[8086:1c2d] 00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
IOMMU group 4:	[8086:1c10] 00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
IOMMU group 5:	[8086:1c18] 00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5)
IOMMU group 6:	[8086:1c1c] 00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5)
IOMMU group 7:	[8086:1c1e] 00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5)
IOMMU group 8:	[8086:1c26] 00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
IOMMU group 9:	[8086:244e] 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5)
IOMMU group 10:	[8086:1c54] 00:1f.0 ISA bridge: Intel Corporation C204 Chipset Family LPC Controller (rev 05)
	[8086:1c04] 00:1f.2 RAID bus controller: Intel Corporation 6 Series/C200 Series Desktop SATA RAID Controller (rev 05)
IOMMU group 11:	[10de:0a65] 07:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
	[10de:0be3] 07:00.1 Audio device: NVIDIA Corporation High Definition Audio Controller (rev a1)
IOMMU group 12:	[1590:005f] 02:00.0 Memory controller: Hewlett Packard Enterprise Device 005f
IOMMU group 13:	[14e4:165f] 03:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe
	[14e4:165f] 03:00.1 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe
IOMMU group 14:	[1912:0014] 04:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03)
IOMMU group 15:	[103c:3306] 01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 05)
	[103c:3307] 01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 05)
	[103c:3300] 01:00.4 USB controller: Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller (rev 02)

 

CPU Thread Pairings

Pair 1:	cpu 0 / cpu 4
Pair 2:	cpu 1 / cpu 5
Pair 3:	cpu 2 / cpu 6
Pair 4:	cpu 3 / cpu 7

 

USB Devices

Bus 001 Device 001:	ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002:	ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 003:	ID 0781:5571 SanDisk Corp. Cruzer Fit
Bus 002 Device 001:	ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002:	ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 003:	ID 0424:2660 Standard Microsystems Corp. Hub
Bus 003 Device 001:	ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001:	ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001:	ID 1d6b:0003 Linux Foundation 3.0 root hub

 

SCSI Devices

[0:0:0:0]	disk SanDisk Cruzer Fit 1.27 /dev/sda 16.0GB
[1:0:0:0]	disk ATA WDC WD30EFRX-68E 0A82 /dev/sdb 3.00TB
[2:0:0:0]	disk ATA WDC WD30EFRX-68E 0A82 /dev/sdc 3.00TB
[3:0:0:0]	disk ATA WDC WD10EARS-00Y 0A80 /dev/sdd 1.00TB
[4:0:0:0]	disk ATA WDC WD40EZRZ-00W 0A80 /dev/sde 4.00TB
[5:0:0:0]	disk ATA FCCT256M4SSD1 070H /dev/sdf 256GB

 

I see a possible fix that involves an edit to the syslinux config file but mine did no match the example so I was unsure if I could add the edit to allow unsafe interrupts

 

Could some one please advise me if unsafe interrupts is the way to go and what to add to the syslinux config file 

 

heres how mine looks 

 

 default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot
label unRAID OS GUI Mode
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot,/bzroot-gui
label unRAID OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot unraidsafemode
label unRAID OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

 

Any help gratefully received 

 

Thanks 

not sure how you could have searched and missed the big old thread about hp/proliants and unraid.... look at the link in my signature.

On 7/23/2017 at 4:19 PM, Squid said:

Are you also passing through the audio for the gpu?

 

Yes also passing through the audio

 

Where's the signature? if I click on you'r name I get your profile, 

 

I have seen various threads and was still unsure how to proceed because the syslinux file I have did not match the one people were editing and I was wondering if i just go ahead and add the text "vfio_iommu_type1.allow_unsafe_interrupts=1" after append like below?

 

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot
label unRAID OS GUI Mode
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot,/bzroot-gui
label unRAID OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot unraidsafemode
label unRAID OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

 

 

Edited August 27, 20187 yr by DGB

7 minutes ago, DGB said:

Where's the signature? if I click on you'r name I get your profile, 

 

Signatures are displayed below posts but they are not enabled by default. You have to go into your own profile settings and enable them.

Thanks John I have now found the links thanks

 

Unfortunately after following the suggestions and several restarts I have been unable to passthrough the GFX card  and now unraid will not load past the unraid boot options page loading /bzimage flashes up then restarts the 5 sec count down and continues this loop

 

It will boot if I have a keyboard pluged in and select "unraid OS GUI mode"

 

Is there any way to change it back?

 

I have checked and the syslinux file I was editing look as it was before I changed anything

10 minutes ago, DGB said:

Thanks John I have now found the links thanks

 

Unfortunately after following the suggestions and several restarts I have been unable to passthrough the GFX card  and now unraid will not load past the unraid boot options page loading /bzimage flashes up then restarts the 5 sec count down and continues this loop

 

It will boot if I have a keyboard pluged in and select "unraid OS GUI mode"

 

Is there any way to change it back?

 

I have checked and the syslinux file I was editing look as it was before I changed anything

 

 

your syslinux.cfg is incorrect if it won't boot right. are you editing it with a proper editor or just word or something?

 

either way, download a fresh zip of unraid, extract/decompress it, replace the bad syslinux file on your usb with the new one. should boot normally then.

 

the thread in my signature details working solutions for your server, if you follow it correctly. Assuming you get your syslinux.cfg set correctly, and passthough is still not working, then you need to follow the steps outlined including checking IOMMU groupings and checking for the RMRR issue in your system logs.

4 minutes ago, 1812 said:

 

 

your syslinux.cfg is incorrect if it won't boot right. are you editing it with a proper editor or just word or something?

 

either way, download a fresh zip of unraid, extract/decompress it, replace the bad syslinux file on your usb with the new one. should boot normally then.

 

the thread in my signature details working solutions for your server, if you follow it correctly. Assuming you get your syslinux.cfg set correctly, and passthough is still not working, then you need to follow the steps outlined including checking IOMMU groupings and checking for the RMRR issue in your system logs.

 

 

I was editing the syslinux config file from inside unraid, Main/Flash then editing the text is this not the best way? should I be removing the usb with unraid on it and editing it on another pc?

 

I followed the steps but the gfx card ends up in a IOMMU group with something else 

 

I will have to try sort the new problem of unraid not booting first then get back to looking at the RMRR issue you mention

 

I have updated the BIOS but this did not help the passthrough problem 

 

This link from the thread seams to be dead  https://docs.hpcloud.com/hos-4.x/helion/networking/enabling_pcipt_on_gen9.html

 

Thank you for your help I will try again another day but fear this may be beyond me 

 

 

1 hour ago, DGB said:

I will try again another day but fear this may be beyond me 

 

All of this was once beyond me. But if I can figure it out, anyone can.

 

Editing inside the web GUI is best practice, so something is probably just a little off. Replacing the syslinux filemwith a new one should resolve boot issue.

 

 

in the short term, also lookup acs override using multifunction to see if that splits up the card into its own iommu group. There is a good video on YouTube by SpaceInvaderone dealing with iommu groups and unRaid.