January 25, 201610 yr Hi, I have just purchased UnRAID as I found the prospect of running Windows as a Virtual Machine but being able to give it it's own dedicated GPU very appealing. So I can still use my old system as a gaming system and dedicated NAS. I read and checked that my BIOS has the features I read were essential for this, specifically VT-D. All of these options relating to virtualisation are enabled in my motherboard. When I try to start my VM, I get the following Execution Error: internal error: early end of file from monitor: possible problem: 2016-01-25T21:44:05.924205Z qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to set iommu for container: Operation not permitted 2016-01-25T21:44:05.924228Z qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 12 2016-01-25T21:44:05.924235Z qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 12 2016-01-25T21:44:05.924245Z qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed 2016-01-25T21:44:05.924253Z qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized This makes very little sense to me and I was hoping someone could assist what I may have misconfigured to cause this? Motherboard: Asus P6T CPU: Intel Core i7 975 (Extreme Edition) Many thanks in advance,if there is any more information I need to provide, please let me know.
January 25, 201610 yr This has to be in the manual/wiki somewhere, but couldn't find it, so let me break it down for ya! In order to pass a device to a VM you MUST pass all devices (or "stub" ie don't use them) to that VM. Your error message is expected, in group 12 you must have more devices than what you are attempting to pass to this VM. This can be seen from the WebUI under Tools - System Devices in the IOMMU Group box. Please copy/paste that to here and we can see what is going on. You may have to enable the PCIe ACS Override to Yes in Settings - VM Manager, however let's see what devices you have 1st and if you have any other options that would work well. The ACS override option is not recommended IF you have an alternate way of getting the device by itself (move it to a different slot, disable the other device in its group, etc...). However if not, you should be fine with it, but you should probably read about what it is doing to understand why it is not on by default. A full Diagnostics output would be the most help for information, Tools - Diagnostics, Download and attach the zip it creates on your flash drive here.
January 25, 201610 yr Author Thank you for the prompt response Right so from the IOMMU groups category in System Devices I get the following: /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/2/devices/0000:00:03.0 /sys/kernel/iommu_groups/3/devices/0000:00:07.0 /sys/kernel/iommu_groups/4/devices/0000:00:14.0 /sys/kernel/iommu_groups/4/devices/0000:00:14.1 /sys/kernel/iommu_groups/4/devices/0000:00:14.2 /sys/kernel/iommu_groups/4/devices/0000:00:14.3 /sys/kernel/iommu_groups/5/devices/0000:00:1a.0 /sys/kernel/iommu_groups/5/devices/0000:00:1a.1 /sys/kernel/iommu_groups/5/devices/0000:00:1a.2 /sys/kernel/iommu_groups/5/devices/0000:00:1a.7 /sys/kernel/iommu_groups/6/devices/0000:00:1b.0 /sys/kernel/iommu_groups/7/devices/0000:00:1c.0 /sys/kernel/iommu_groups/7/devices/0000:00:1c.1 /sys/kernel/iommu_groups/7/devices/0000:00:1c.2 /sys/kernel/iommu_groups/7/devices/0000:00:1c.3 /sys/kernel/iommu_groups/7/devices/0000:00:1c.4 /sys/kernel/iommu_groups/7/devices/0000:04:00.0 /sys/kernel/iommu_groups/7/devices/0000:04:00.1 /sys/kernel/iommu_groups/7/devices/0000:05:00.0 /sys/kernel/iommu_groups/7/devices/0000:06:00.0 /sys/kernel/iommu_groups/7/devices/0000:07:00.0 /sys/kernel/iommu_groups/8/devices/0000:00:1d.0 /sys/kernel/iommu_groups/8/devices/0000:00:1d.1 /sys/kernel/iommu_groups/8/devices/0000:00:1d.2 /sys/kernel/iommu_groups/8/devices/0000:00:1d.7 /sys/kernel/iommu_groups/9/devices/0000:00:1e.0 /sys/kernel/iommu_groups/10/devices/0000:00:1f.0 /sys/kernel/iommu_groups/10/devices/0000:00:1f.2 /sys/kernel/iommu_groups/10/devices/0000:00:1f.3 /sys/kernel/iommu_groups/11/devices/0000:02:00.0 /sys/kernel/iommu_groups/11/devices/0000:02:00.1 /sys/kernel/iommu_groups/12/devices/0000:03:00.0 /sys/kernel/iommu_groups/12/devices/0000:03:00.1 /sys/kernel/iommu_groups/13/devices/0000:ff:00.0 /sys/kernel/iommu_groups/13/devices/0000:ff:00.1 /sys/kernel/iommu_groups/14/devices/0000:ff:02.0 /sys/kernel/iommu_groups/14/devices/0000:ff:02.1 /sys/kernel/iommu_groups/15/devices/0000:ff:03.0 /sys/kernel/iommu_groups/15/devices/0000:ff:03.1 /sys/kernel/iommu_groups/15/devices/0000:ff:03.4 /sys/kernel/iommu_groups/16/devices/0000:ff:04.0 /sys/kernel/iommu_groups/16/devices/0000:ff:04.1 /sys/kernel/iommu_groups/16/devices/0000:ff:04.2 /sys/kernel/iommu_groups/16/devices/0000:ff:04.3 /sys/kernel/iommu_groups/17/devices/0000:ff:05.0 /sys/kernel/iommu_groups/17/devices/0000:ff:05.1 /sys/kernel/iommu_groups/17/devices/0000:ff:05.2 /sys/kernel/iommu_groups/17/devices/0000:ff:05.3 /sys/kernel/iommu_groups/18/devices/0000:ff:06.0 /sys/kernel/iommu_groups/18/devices/0000:ff:06.1 /sys/kernel/iommu_groups/18/devices/0000:ff:06.2 /sys/kernel/iommu_groups/18/devices/0000:ff:06.3 If it helps, this is what is showing under PCI Devices also: 00:00.0 Host bridge: Intel Corporation 5520/5500/X58 I/O Hub to ESI Port (rev 13) 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13) 00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13) 00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13) 00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers (rev 13) 00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13) 00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13) 00:14.3 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Throttle Registers (rev 13) 00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5 00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6 00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller 00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1 00:1c.1 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 2 00:1c.2 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 3 00:1c.3 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 4 00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5 00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) 00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller 00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller 00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller 02:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 610] (rev a1) 02:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1) 03:00.0 VGA compatible controller: NVIDIA Corporation GK106 [GeForce GTX 660] (rev a1) 03:00.1 Audio device: NVIDIA Corporation GK106 HDMI Audio Controller (rev a1) 04:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03) 04:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 03) 05:00.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6315 Series Firewire Controller 06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 02) 07:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03) ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers (rev 05) ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder (rev 05) ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05) ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0 (rev 05) ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller (rev 05) ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder (rev 05) ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers (rev 05) ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers (rev 05) ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers (rev 05) ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers (rev 05) ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers (rev 05) ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers (rev 05) ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers (rev 05) ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers (rev 05) ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers (rev 05) ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers (rev 05) ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers (rev 05) ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers (rev 05) ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers (rev 05) I will try and get the full diagnostics output posted shortly. Thanks again!
January 25, 201610 yr Author This has to be in the manual/wiki somewhere, but couldn't find it, so let me break it down for ya! In order to pass a device to a VM you MUST pass all devices (or "stub" ie don't use them) to that VM. Your error message is expected, in group 12 you must have more devices than what you are attempting to pass to this VM. This can be seen from the WebUI under Tools - System Devices in the IOMMU Group box. Please copy/paste that to here and we can see what is going on. You may have to enable the PCIe ACS Override to Yes in Settings - VM Manager, however let's see what devices you have 1st and if you have any other options that would work well. The ACS override option is not recommended IF you have an alternate way of getting the device by itself (move it to a different slot, disable the other device in its group, etc...). However if not, you should be fine with it, but you should probably read about what it is doing to understand why it is not on by default. A full Diagnostics output would be the most help for information, Tools - Diagnostics, Download and attach the zip it creates on your flash drive here. As Requested, please find attached the Diagnostics Output generated as described above. whitecorsair-diagnostics-20160125-2348.zip
January 26, 201610 yr Well I take back what I assumed it was (which is NOT your issue) and now we have more information to go on. This seems to have been popping up on the forum lately for a couple of users. Your syslog has the following that is likely causing the issue Jan 25 21:40:52 WhiteCorsair kernel: Your BIOS is broken; DMA routed to ISOCH DMAR unit but no TLB space. Jan 25 21:40:52 WhiteCorsair kernel: Your BIOS is broken; RMRR ends before it starts! Are there any BIOS updates for your board? Information from another post below that you can try, by adding iommu=pt to your syslinux file. Your logs tell me all I need to know: Jan 14 19:57:05 Server kernel: ------------[ cut here ]------------ Jan 14 19:57:05 Server kernel: WARNING: CPU: 0 PID: 1 at drivers/iommu/intel-iommu.c:4744 init_dmars+0x258/0x454() Jan 14 19:57:05 Server kernel: Your BIOS is broken; DMA routed to ISOCH DMAR unit but no TLB space. Jan 14 19:57:05 Server kernel: BIOS vendor: American Megatrends Inc.; Ver: 1502 ; Product Version: System Version Jan 14 19:57:05 Server kernel: Modules linked in: Jan 14 19:57:05 Server kernel: CPU: 0 PID: 1 Comm: swapper/0 Tainted: G I 4.1.13-unRAID #1 Jan 14 19:57:05 Server kernel: Hardware name: System manufacturer System Product Name/Rampage III Extreme, BIOS 1502 10/03/2011 Jan 14 19:57:05 Server kernel: 0000000000000009 ffff880331137d38 ffffffff815f12b0 0000000000001d6e Jan 14 19:57:05 Server kernel: ffff880331137d88 ffff880331137d78 ffffffff8104775b ffff880330dd0000 Jan 14 19:57:05 Server kernel: ffffffff818d6767 ffff880330f54000 0000000000000000 ffff880330f6a780 Jan 14 19:57:05 Server kernel: Call Trace: Jan 14 19:57:05 Server kernel: [<ffffffff815f12b0>] dump_stack+0x4c/0x6e Jan 14 19:57:05 Server kernel: [<ffffffff8104775b>] warn_slowpath_common+0x97/0xb1 Jan 14 19:57:05 Server kernel: [<ffffffff818d6767>] ? init_dmars+0x258/0x454 Jan 14 19:57:05 Server kernel: [<ffffffff810477b6>] warn_slowpath_fmt+0x41/0x43 Jan 14 19:57:05 Server kernel: [<ffffffff818d6767>] init_dmars+0x258/0x454 Jan 14 19:57:05 Server kernel: [<ffffffff818d6bf8>] intel_iommu_init+0x295/0x585 Jan 14 19:57:05 Server kernel: [<ffffffff818ac2a8>] pci_iommu_init+0x13/0x3e Jan 14 19:57:05 Server kernel: [<ffffffff818ac295>] ? memblock_find_dma_reserve+0x146/0x146 Jan 14 19:57:05 Server kernel: [<ffffffff81002145>] do_one_initcall+0xe9/0x178 Jan 14 19:57:05 Server kernel: [<ffffffff818a405d>] kernel_init_freeable+0x17c/0x204 Jan 14 19:57:05 Server kernel: [<ffffffff818a3807>] ? initcall_blacklist+0xa5/0xa5 Jan 14 19:57:05 Server kernel: [<ffffffff815e4585>] ? rest_init+0x89/0x89 Jan 14 19:57:05 Server kernel: [<ffffffff815e458e>] kernel_init+0x9/0xd5 Jan 14 19:57:05 Server kernel: [<ffffffff815f6d92>] ret_from_fork+0x42/0x70 Jan 14 19:57:05 Server kernel: [<ffffffff815e4585>] ? rest_init+0x89/0x89 Jan 14 19:57:05 Server kernel: ---[ end trace 577486b93363598b ]--- Jan 14 19:57:05 Server kernel: IOMMU: hardware identity mapping for device 0000:00:1b.0 Jan 14 19:57:05 Server kernel: IOMMU: Setting RMRR: Jan 14 19:57:05 Server kernel: IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7dc000 - 0xbf7dbfff] Jan 14 19:57:05 Server kernel: ------------[ cut here ]------------ Jan 14 19:57:05 Server kernel: WARNING: CPU: 0 PID: 1 at drivers/iommu/intel-iommu.c:2380 iommu_prepare_identity_map+0xd2/0x1a0() Jan 14 19:57:05 Server kernel: Your BIOS is broken; RMRR ends before it starts! Jan 14 19:57:05 Server kernel: BIOS vendor: American Megatrends Inc.; Ver: 1502 ; Product Version: System Version Jan 14 19:57:05 Server kernel: Modules linked in: Jan 14 19:57:05 Server kernel: CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W I 4.1.13-unRAID #1 Jan 14 19:57:05 Server kernel: Hardware name: System manufacturer System Product Name/Rampage III Extreme, BIOS 1502 10/03/2011 Jan 14 19:57:05 Server kernel: 0000000000000009 ffff880331137cd8 ffffffff815f12b0 0000000000001ef0 Jan 14 19:57:05 Server kernel: ffff880331137d28 ffff880331137d18 ffffffff8104775b 00000000bf7dbfff Jan 14 19:57:05 Server kernel: ffffffff815f23a6 ffff880330f6a900 00000000bf7dbfff ffff880330f66098 Jan 14 19:57:05 Server kernel: Call Trace: Jan 14 19:57:05 Server kernel: [<ffffffff815f12b0>] dump_stack+0x4c/0x6e Jan 14 19:57:05 Server kernel: [<ffffffff8104775b>] warn_slowpath_common+0x97/0xb1 Jan 14 19:57:05 Server kernel: [<ffffffff815f23a6>] ? iommu_prepare_identity_map+0xd2/0x1a0 Jan 14 19:57:05 Server kernel: [<ffffffff810477b6>] warn_slowpath_fmt+0x41/0x43 Jan 14 19:57:05 Server kernel: [<ffffffff815f23a6>] iommu_prepare_identity_map+0xd2/0x1a0 Jan 14 19:57:05 Server kernel: [<ffffffff818d67fd>] init_dmars+0x2ee/0x454 Jan 14 19:57:05 Server kernel: [<ffffffff818d6bf8>] intel_iommu_init+0x295/0x585 Jan 14 19:57:05 Server kernel: [<ffffffff818ac2a8>] pci_iommu_init+0x13/0x3e Jan 14 19:57:05 Server kernel: [<ffffffff818ac295>] ? memblock_find_dma_reserve+0x146/0x146 Jan 14 19:57:05 Server kernel: [<ffffffff81002145>] do_one_initcall+0xe9/0x178 Jan 14 19:57:05 Server kernel: [<ffffffff818a405d>] kernel_init_freeable+0x17c/0x204 Jan 14 19:57:05 Server kernel: [<ffffffff818a3807>] ? initcall_blacklist+0xa5/0xa5 Jan 14 19:57:05 Server kernel: [<ffffffff815e4585>] ? rest_init+0x89/0x89 Jan 14 19:57:05 Server kernel: [<ffffffff815e458e>] kernel_init+0x9/0xd5 Jan 14 19:57:05 Server kernel: [<ffffffff815f6d92>] ret_from_fork+0x42/0x70 Jan 14 19:57:05 Server kernel: [<ffffffff815e4585>] ? rest_init+0x89/0x89 Jan 14 19:57:05 Server kernel: ---[ end trace 577486b93363598c ]--- Jan 14 19:57:05 Server kernel: IOMMU: mapping reserved region failed Jan 14 19:57:05 Server kernel: IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7dc000 - 0xbf7dbfff] this continues for a while... I would suggest procuring a higher-quality motherboard, but here's one other thing you can try to overcome this, but I make no guarantees for system stability given these errors messages on your motherboard (and this also may result in some degraded performance as compared to without this): Goto Main and click Flash to get to the Flash Device Settings Page. Locate the Syslinux Configuration Section Find and edit this section (add what I have bolded below): label unRAID OS menu default kernel /bzimage append iommu=pt pcie_acs_override=downstream initrd=/bzroot unraidsafemode Apply that change and reboot your system. Give it a whirl. You could also try looking to see if your motherboard has a BIOS update available. The full thread is here http://lime-technology.com/forum/index.php?topic=45530.msg435107#msg435107
January 26, 201610 yr Author Well I take back what I assumed it was (which is NOT your issue) and now we have more information to go on. This seems to have been popping up on the forum lately for a couple of users. Your syslog has the following that is likely causing the issue Jan 25 21:40:52 WhiteCorsair kernel: Your BIOS is broken; DMA routed to ISOCH DMAR unit but no TLB space. Jan 25 21:40:52 WhiteCorsair kernel: Your BIOS is broken; RMRR ends before it starts! Are there any BIOS updates for your board? Information from another post below that you can try, by adding iommu=pt to your syslinux file. Your logs tell me all I need to know: Jan 14 19:57:05 Server kernel: ------------[ cut here ]------------ Jan 14 19:57:05 Server kernel: WARNING: CPU: 0 PID: 1 at drivers/iommu/intel-iommu.c:4744 init_dmars+0x258/0x454() Jan 14 19:57:05 Server kernel: Your BIOS is broken; DMA routed to ISOCH DMAR unit but no TLB space. Jan 14 19:57:05 Server kernel: BIOS vendor: American Megatrends Inc.; Ver: 1502 ; Product Version: System Version Jan 14 19:57:05 Server kernel: Modules linked in: Jan 14 19:57:05 Server kernel: CPU: 0 PID: 1 Comm: swapper/0 Tainted: G I 4.1.13-unRAID #1 Jan 14 19:57:05 Server kernel: Hardware name: System manufacturer System Product Name/Rampage III Extreme, BIOS 1502 10/03/2011 Jan 14 19:57:05 Server kernel: 0000000000000009 ffff880331137d38 ffffffff815f12b0 0000000000001d6e Jan 14 19:57:05 Server kernel: ffff880331137d88 ffff880331137d78 ffffffff8104775b ffff880330dd0000 Jan 14 19:57:05 Server kernel: ffffffff818d6767 ffff880330f54000 0000000000000000 ffff880330f6a780 Jan 14 19:57:05 Server kernel: Call Trace: Jan 14 19:57:05 Server kernel: [<ffffffff815f12b0>] dump_stack+0x4c/0x6e Jan 14 19:57:05 Server kernel: [<ffffffff8104775b>] warn_slowpath_common+0x97/0xb1 Jan 14 19:57:05 Server kernel: [<ffffffff818d6767>] ? init_dmars+0x258/0x454 Jan 14 19:57:05 Server kernel: [<ffffffff810477b6>] warn_slowpath_fmt+0x41/0x43 Jan 14 19:57:05 Server kernel: [<ffffffff818d6767>] init_dmars+0x258/0x454 Jan 14 19:57:05 Server kernel: [<ffffffff818d6bf8>] intel_iommu_init+0x295/0x585 Jan 14 19:57:05 Server kernel: [<ffffffff818ac2a8>] pci_iommu_init+0x13/0x3e Jan 14 19:57:05 Server kernel: [<ffffffff818ac295>] ? memblock_find_dma_reserve+0x146/0x146 Jan 14 19:57:05 Server kernel: [<ffffffff81002145>] do_one_initcall+0xe9/0x178 Jan 14 19:57:05 Server kernel: [<ffffffff818a405d>] kernel_init_freeable+0x17c/0x204 Jan 14 19:57:05 Server kernel: [<ffffffff818a3807>] ? initcall_blacklist+0xa5/0xa5 Jan 14 19:57:05 Server kernel: [<ffffffff815e4585>] ? rest_init+0x89/0x89 Jan 14 19:57:05 Server kernel: [<ffffffff815e458e>] kernel_init+0x9/0xd5 Jan 14 19:57:05 Server kernel: [<ffffffff815f6d92>] ret_from_fork+0x42/0x70 Jan 14 19:57:05 Server kernel: [<ffffffff815e4585>] ? rest_init+0x89/0x89 Jan 14 19:57:05 Server kernel: ---[ end trace 577486b93363598b ]--- Jan 14 19:57:05 Server kernel: IOMMU: hardware identity mapping for device 0000:00:1b.0 Jan 14 19:57:05 Server kernel: IOMMU: Setting RMRR: Jan 14 19:57:05 Server kernel: IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7dc000 - 0xbf7dbfff] Jan 14 19:57:05 Server kernel: ------------[ cut here ]------------ Jan 14 19:57:05 Server kernel: WARNING: CPU: 0 PID: 1 at drivers/iommu/intel-iommu.c:2380 iommu_prepare_identity_map+0xd2/0x1a0() Jan 14 19:57:05 Server kernel: Your BIOS is broken; RMRR ends before it starts! Jan 14 19:57:05 Server kernel: BIOS vendor: American Megatrends Inc.; Ver: 1502 ; Product Version: System Version Jan 14 19:57:05 Server kernel: Modules linked in: Jan 14 19:57:05 Server kernel: CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W I 4.1.13-unRAID #1 Jan 14 19:57:05 Server kernel: Hardware name: System manufacturer System Product Name/Rampage III Extreme, BIOS 1502 10/03/2011 Jan 14 19:57:05 Server kernel: 0000000000000009 ffff880331137cd8 ffffffff815f12b0 0000000000001ef0 Jan 14 19:57:05 Server kernel: ffff880331137d28 ffff880331137d18 ffffffff8104775b 00000000bf7dbfff Jan 14 19:57:05 Server kernel: ffffffff815f23a6 ffff880330f6a900 00000000bf7dbfff ffff880330f66098 Jan 14 19:57:05 Server kernel: Call Trace: Jan 14 19:57:05 Server kernel: [<ffffffff815f12b0>] dump_stack+0x4c/0x6e Jan 14 19:57:05 Server kernel: [<ffffffff8104775b>] warn_slowpath_common+0x97/0xb1 Jan 14 19:57:05 Server kernel: [<ffffffff815f23a6>] ? iommu_prepare_identity_map+0xd2/0x1a0 Jan 14 19:57:05 Server kernel: [<ffffffff810477b6>] warn_slowpath_fmt+0x41/0x43 Jan 14 19:57:05 Server kernel: [<ffffffff815f23a6>] iommu_prepare_identity_map+0xd2/0x1a0 Jan 14 19:57:05 Server kernel: [<ffffffff818d67fd>] init_dmars+0x2ee/0x454 Jan 14 19:57:05 Server kernel: [<ffffffff818d6bf8>] intel_iommu_init+0x295/0x585 Jan 14 19:57:05 Server kernel: [<ffffffff818ac2a8>] pci_iommu_init+0x13/0x3e Jan 14 19:57:05 Server kernel: [<ffffffff818ac295>] ? memblock_find_dma_reserve+0x146/0x146 Jan 14 19:57:05 Server kernel: [<ffffffff81002145>] do_one_initcall+0xe9/0x178 Jan 14 19:57:05 Server kernel: [<ffffffff818a405d>] kernel_init_freeable+0x17c/0x204 Jan 14 19:57:05 Server kernel: [<ffffffff818a3807>] ? initcall_blacklist+0xa5/0xa5 Jan 14 19:57:05 Server kernel: [<ffffffff815e4585>] ? rest_init+0x89/0x89 Jan 14 19:57:05 Server kernel: [<ffffffff815e458e>] kernel_init+0x9/0xd5 Jan 14 19:57:05 Server kernel: [<ffffffff815f6d92>] ret_from_fork+0x42/0x70 Jan 14 19:57:05 Server kernel: [<ffffffff815e4585>] ? rest_init+0x89/0x89 Jan 14 19:57:05 Server kernel: ---[ end trace 577486b93363598c ]--- Jan 14 19:57:05 Server kernel: IOMMU: mapping reserved region failed Jan 14 19:57:05 Server kernel: IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7dc000 - 0xbf7dbfff] this continues for a while... I would suggest procuring a higher-quality motherboard, but here's one other thing you can try to overcome this, but I make no guarantees for system stability given these errors messages on your motherboard (and this also may result in some degraded performance as compared to without this): Goto Main and click Flash to get to the Flash Device Settings Page. Locate the Syslinux Configuration Section Find and edit this section (add what I have bolded below): label unRAID OS menu default kernel /bzimage append iommu=pt pcie_acs_override=downstream initrd=/bzroot unraidsafemode Apply that change and reboot your system. Give it a whirl. You could also try looking to see if your motherboard has a BIOS update available. The full thread is here http://lime-technology.com/forum/index.php?topic=45530.msg435107#msg435107 That doesn't sound good at all. Sadly this is an older board based off of the X58 chipset, no new BIOS updates for it since 09/11/2010
January 26, 201610 yr Author Have you tried passing through the audio device of the gpu also? Hi, yes I have now reconfigured the VM to include the Graphics Card's HDMI audio as well, sadly I am still getting the error
January 26, 201610 yr Author I have tried adding the label unRAID OS menu default kernel /bzimage append iommu=pt pcie_acs_override=downstream initrd=/bzroot unraidsafemode as detailed previously. Sadly I am still getting thesame error when trying to start the VM
January 26, 201610 yr I have tried adding the label unRAID OS menu default kernel /bzimage append iommu=pt pcie_acs_override=downstream initrd=/bzroot unraidsafemode as detailed previously. Sadly I am still getting thesame error when trying to start the VM Not that it is harming you, but if you ever get this to work, you don't need this part pcie_acs_override=downstream It looks as if the MB does not provide addressing for the device, in one way or another, leading to it not being able to be used. Unfortunately I don't have much more info that is of help. If you want to ask others not specifically in this forum, I'd post your question to the vfio-users mailing list and see if they have a suggestion. Google it, this is where most of the changes/testing to KVM are happening. Alex Williamson would likely have the best insight as to the issue, however no guarantee it can be resolved easily.
January 27, 201610 yr Author I seem to have got it to work! I do not understand exactly how it works or what exactly I have done, but I found the solution from another thread https://lime-technology.com/forum/index.php?topic=36110.0 SOLVED: For whatever reason, my syslinux.cfg requires the following: default /syslinux/menu.c32 menu title Lime Technology prompt 0 timeout 50 label unRAID OS kernel /bzimage append initrd=/bzroot label KVM/unRAID OS menu default kernel /bzimage append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot label unRAID OS Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot unraidsafemode label Memtest86+ kernel /memtest label Xen/unRAID OS kernel /syslinux/mboot.c32 append /xen --- /bzimage --- /bzroot label Xen/unRAID OS Safe Mode (no plugins) kernel /syslinux/mboot.c32 append /xen --- /bzimage --- /bzroot unraidsafemode --------------------- Can some decypher the error message below that I see when trying to start the VM: message: Error while starting domain: internal error: early end of file from monitor: possible problem: 2014-11-05T19:50:04.256525Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to set iommu for container: Operation not permitted 2014-11-05T19:50:04.256571Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to setup container for group 25 2014-11-05T19:50:04.256585Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 25 2014-11-05T19:50:04.256605Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed. 2014-11-05T19:50:04.256624Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized My HD6450 that I am trying to passthrough: 83:00.0 VGA compatible controller [0300]: AMD/ATI [Advanced Micro Devices, Inc.] Caicos [Radeon HD 6450] [1002:6779] Subsystem: Device [1b0a:909d] Kernel driver in use: vfio-pci 83:00.1 Audio device [0403]: AMD/ATI [Advanced Micro Devices, Inc.] Caicos HDMI Audio [Radeon HD 6400 Series] [1002:aa98] Subsystem: Device [1b0a:aa98] Kernel driver in use: vfio-pci Here is my XML: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>HTPCFAMRM</name> <uuid>c43b7542-b40b-495f-90ab-aaa4eec68e8a</uuid> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>4</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='3'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-2.1'>hvm</type> <boot dev='cdrom'/> <bootmenu enable='no'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='2' cores='2' threads='1'/> </cpu> <clock offset='localtime'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none' io='native'/> <source file='/mnt/cache/VMs/HTPCFAMRM.qcow2'/> <target dev='vdc' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/Images/xbmcbuntu-13.0~gotham_amd64.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='1' target='0' unit='0'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x2'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <controller type='ide' index='0'/> <interface type='bridge'> <mac address='52:94:00:d0:c0:dc'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> </interface> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x20a0'/> <product id='0x0001'/> </source> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </memballoon> </devices> <seclabel type='none' model='none'/> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=83:00.1,bus=pcie.0'/> </qemu:commandline> </domain> Specifically: By using the following in my syslinux.cfg intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 I can now get the VM tyo launch using my Nvidia GTX 660 with the HDMI sound from that graphics card. I have installed Windows & been able to launch a game !
Archived
This topic is now archived and is closed to further replies.