GPU pass through trouble - IOMMU groups - tried ACS option


bhinkle50

Recommended Posts

Here's the error I'm getting-

internal error: process exited while connecting to monitor: 2015-09-16T21:51:20.538866Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.

2015-09-16T21:51:20.538881Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 1

2015-09-16T21:51:20.538887Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed

2015-09-16T21:51:20.538893Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

 

Here are my System Devices:

 

00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v3 Processor DRAM Controller (rev 06)

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)

00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)

00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05)

00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05)

00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5)

00:1c.2 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #3 (rev d5)

00:1c.3 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 (rev d5)

00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05)

00:1f.0 ISA bridge: Intel Corporation C222 Series Chipset Family Server Essential SKU LPC Controller (rev 05)

00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05)

00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05)

00:1f.6 Signal processing controller: Intel Corporation 8 Series Chipset Family Thermal Management Controller (rev 05)

01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Bonaire XTX [Radeon R7 260X]

01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device aac0

02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05)

03:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 03)

04:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30)

05:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

 

Here's my IOMMU groups:

 

/sys/kernel/iommu_groups/0/devices/0000:00:00.0

/sys/kernel/iommu_groups/1/devices/0000:00:01.0

/sys/kernel/iommu_groups/1/devices/0000:00:01.1

/sys/kernel/iommu_groups/1/devices/0000:01:00.0

/sys/kernel/iommu_groups/1/devices/0000:01:00.1

/sys/kernel/iommu_groups/1/devices/0000:02:00.0

/sys/kernel/iommu_groups/2/devices/0000:00:14.0

/sys/kernel/iommu_groups/3/devices/0000:00:1a.0

/sys/kernel/iommu_groups/4/devices/0000:00:1c.0

/sys/kernel/iommu_groups/5/devices/0000:00:1c.2

/sys/kernel/iommu_groups/6/devices/0000:00:1c.3

/sys/kernel/iommu_groups/7/devices/0000:00:1d.0

/sys/kernel/iommu_groups/8/devices/0000:00:1f.0

/sys/kernel/iommu_groups/8/devices/0000:00:1f.2

/sys/kernel/iommu_groups/8/devices/0000:00:1f.3

/sys/kernel/iommu_groups/8/devices/0000:00:1f.6

/sys/kernel/iommu_groups/9/devices/0000:03:00.0

/sys/kernel/iommu_groups/9/devices/0000:04:00.0

/sys/kernel/iommu_groups/10/devices/0000:05:00.0

/sys/kernel/iommu_groups/11/devices/0000:06:00.0

 

I have been doing alot of reading on the PCIe ACS override and I thought I would be able to start the VM (windows 10 that I configured using VNC), but I keep getting this error no matter if the option is YES or NO.

 

Also, I have rebooted and powered down at least 10 times, and the message at the top keeps appearing.

 

Hydra_VM_settings.gif.f69abae3641f085cd3739812c1755122.gif

Link to comment

I'd guess for whatever reason the override option isn't being set correctly, hence why you continously see the reboot message. You definitely show devices in the same group, and until that is sorted this won't work unless you pass everything in the group (which is not feasible).

You can manually add the PCI acs downstream to your syslinux.cfg file, then reboot, and all should be well.

If you can't find instructions to do that, I'll post them.

Link to comment

I agree that there is something up with my syslinux.cfg file. I did an upgrade to 6 vs a fresh install and I'm not sure which syslinux.cfg I should be editing.

 

I have 1 in the root flash folder that looks like this :

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest

 

Another one is in the syslinux folder and it looks to have the option in it:

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append pcie_acs_override=downstream initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest

 

Should I add the acs override statement to the first one as well?

 

 

Link to comment

I had same kind of problem.

Try with this bolded line:

 

default /syslinux/menu.c32

menu title Lime Technology

prompt 0

timeout 50

label unRAID OS

  menu default

  kernel /bzimage

  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot

label unRAID OS Safe Mode (no plugins)

  kernel /bzimage

  append initrd=/bzroot unraidsafemode

label Memtest86+

  kernel /memtest

Link to comment

I had same kind of problem.

Try with this bolded line:

 

default /syslinux/menu.c32

menu title Lime Technology

prompt 0

timeout 50

label unRAID OS

  menu default

  kernel /bzimage

  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot

label unRAID OS Safe Mode (no plugins)

  kernel /bzimage

  append initrd=/bzroot unraidsafemode

label Memtest86+

  kernel /memtest

 

You shouldn't need this, but if you do, make sure to also add the acs downstream also.

In earlier betas we needed the intel_iommu=on, however this is done for us already (you can verify that VT-d is on in Unraid System info).

The other part vfio_iommu_type1.allow_unsafe_interrupts=1 shouldn't be needed as I believe your processor supports proper interrupts (I didn't look, mine does and I am less "server" hardware than yourself)

So for testing, it would look like this (I don't believe the order matters):

label unRAID OS
  menu default
  kernel /bzimage
  append intel_iommu=on pcie_acs_override=downstream vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot

 

I think the bigger issue is that you upgraded and something is a muck..

Did you follow the upgrade guide in the wiki?

It's pretty specific to keep very little in order to avoid conflicts from V5 to V6.

Link to comment

Just want to tell that if I add line pcie_acs_override=downstream to my syslinux.cfg I can't add audio from my HD5450 card to OpenELEC virtual machine. I get kernel panic 7 seconds in the loading. Remove acs_override and everything works.

 

Edit:

Just noticed that if I use Settings->VM Manager -> PCIe ACS Override Yes and reboot there is no kernel panic. One difference was that it added it before vfio_iommu_type1.allow_unsafe_interrupts=1. Maybe the order matters after all?

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.