charleslabri Posted February 4, 2016 Share Posted February 4, 2016 I am testing out a dev rig with four K6000 video cards. I can create two VMs, and assign cards. Things work out well enough (getting the code 43 error on the cards, but that might be NVidia). When I go to create the 3rd or 4th VM, I get this error: VM creation error internal error: early end of file from monitor: possible problem: 2016-02-04T19:46:11.087546Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on: vfio: error opening /dev/vfio/42: Operation not permitted 2016-02-04T19:46:11.087587Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on: vfio: failed to get group 42 2016-02-04T19:46:11.087600Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on: Device initialization failed 2016-02-04T19:46:11.087611Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on: Device 'vfio-pci' could not be initialized I've got the 4 Quadro cards and 1 onboard Matrox video card. Seems that the 42-43 vfio devices are unable to be assigned. Any ideas/success stories would help. Otherwise, I might just go get knee deep in VFIO instead of using unRaid. Quote Link to comment
jonp Posted February 5, 2016 Share Posted February 5, 2016 I am testing out a dev rig with four K6000 video cards. I can create two VMs, and assign cards. Things work out well enough (getting the code 43 error on the cards, but that might be NVidia). When I go to create the 3rd or 4th VM, I get this error: VM creation error internal error: early end of file from monitor: possible problem: 2016-02-04T19:46:11.087546Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on: vfio: error opening /dev/vfio/42: Operation not permitted 2016-02-04T19:46:11.087587Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on: vfio: failed to get group 42 2016-02-04T19:46:11.087600Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on: Device initialization failed 2016-02-04T19:46:11.087611Z qemu-system-x86_64: -device vfio-pci,host=83:00.0,bus=root.1,addr=00.0,multifunction=on: Device 'vfio-pci' could not be initialized I've got the 4 Quadro cards and 1 onboard Matrox video card. Seems that the 42-43 vfio devices are unable to be assigned. Any ideas/success stories would help. Otherwise, I might just go get knee deep in VFIO instead of using unRaid. Please upload system diagnostics from the Tools > Diagnostics page in the webgui. Also goto Tools > System Devices and please copy and paste the IOMMU groups here.. Quote Link to comment
bungee91 Posted February 5, 2016 Share Posted February 5, 2016 @jonp This issue https://lime-technology.com/forum/index.php?topic=44010.msg420830#msg420830 has since been fixed, correct? OP, I assume you're running 6.1.7? Quote Link to comment
charleslabri Posted February 5, 2016 Author Share Posted February 5, 2016 @bungee91 Ah, thanks for that. Yeah, I my other two cards are up at 41-42 IOMMU groupings. Ill test this hotfix. Quote Link to comment
charleslabri Posted February 6, 2016 Author Share Posted February 6, 2016 After applying the hotfix (which just seems to add more room in the "cgroup_device_acl" directive) I was able to assign the devices to the VMs without erroring out. A problem still persists in that on the two devices { 83:00.0 VGA compatible controller: NVIDIA Corporation GK110GL [Quadro K6000] (rev a1) 84:00.0 VGA compatible controller: NVIDIA Corporation GK110GL [Quadro K6000] (rev a1) } They do not output any display when those VMs boot. I have two other VMs working great with passthrough on these two cards { 02:00.0 VGA compatible controller: NVIDIA Corporation GK110GL [Quadro K6000] (rev a1) 03:00.0 VGA compatible controller: NVIDIA Corporation GK110GL [Quadro K6000] (rev a1) } I don't have the ACS patch enabled as I am running a Xeon system. The patch doesn't change this behavior when I enable it. I have attached the diags as requested. A second post will have my PCI List (too many characters per post) Thanks so much! od-devtouch-diagnostics-20160205-1626.zip Quote Link to comment
charleslabri Posted February 6, 2016 Author Share Posted February 6, 2016 Here is my device list: PCI Devices 00:00.0 Host bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 DMI2 (rev 04) 00:01.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 1a (rev 04) 00:02.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 2a (rev 04) 00:03.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 3a (rev 04) 00:04.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 0 (rev 04) 00:04.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 1 (rev 04) 00:04.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 2 (rev 04) 00:04.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 3 (rev 04) 00:04.4 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 4 (rev 04) 00:04.5 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 5 (rev 04) 00:04.6 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 6 (rev 04) 00:04.7 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 7 (rev 04) 00:05.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 VTd/Memory Map/Misc (rev 04) 00:05.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 IIO RAS (rev 04) 00:05.4 PIC: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 IOAPIC (rev 04) 00:11.0 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Virtual Root Port (rev 06) 00:16.0 Communication controller: Intel Corporation C600/X79 series chipset MEI Controller #1 (rev 05) 00:16.1 Communication controller: Intel Corporation C600/X79 series chipset MEI Controller #2 (rev 05) 00:1a.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2 (rev 06) 00:1b.0 Audio device: Intel Corporation C600/X79 series chipset High Definition Audio Controller (rev 06) 00:1c.0 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 1 (rev b6) 00:1c.7 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 8 (rev b6) 00:1d.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #1 (rev 06) 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a6) 00:1f.0 ISA bridge: Intel Corporation C600/X79 series chipset LPC Controller (rev 06) 00:1f.2 RAID bus controller: Intel Corporation C600/X79 series chipset SATA RAID Controller (rev 06) 00:1f.3 SMBus: Intel Corporation C600/X79 series chipset SMBus Host Controller (rev 06) 00:1f.6 Signal processing controller: Intel Corporation C600/X79 series chipset Thermal Management Controller (rev 06) 01:00.0 SATA controller: Kingston Technologies Device 0010 (rev 10) 02:00.0 VGA compatible controller: NVIDIA Corporation GK110GL [Quadro K6000] (rev a1) 02:00.1 Audio device: NVIDIA Corporation GK110 HDMI Audio (rev a1) 03:00.0 VGA compatible controller: NVIDIA Corporation GK110GL [Quadro K6000] (rev a1) 03:00.1 Audio device: NVIDIA Corporation GK110 HDMI Audio (rev a1) 04:00.0 Serial Attached SCSI controller: Intel Corporation C602 chipset 4-Port SATA Storage Control Unit (rev 06) 05:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 05:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 07:00.0 PCI bridge: Renesas Technology Corp. SH7757 PCIe Switch [PS] 08:00.0 PCI bridge: Renesas Technology Corp. SH7757 PCIe Switch [PS] 08:01.0 PCI bridge: Renesas Technology Corp. SH7757 PCIe Switch [PS] 09:00.0 PCI bridge: Renesas Technology Corp. SH7757 PCIe-PCI Bridge [PPB] 0a:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. G200eR2 7f:08.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link 0 (rev 04) 7f:09.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link 1 (rev 04) 7f:0a.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 0 (rev 04) 7f:0a.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 1 (rev 04) 7f:0a.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 2 (rev 04) 7f:0a.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 3 (rev 04) 7f:0b.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 UBOX Registers (rev 04) 7f:0b.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 UBOX Registers (rev 04) 7f:0c.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) 7f:0c.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) 7f:0c.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) 7f:0d.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) 7f:0d.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) 7f:0d.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) 7f:0e.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Home Agent 0 (rev 04) 7f:0e.1 Performance counters: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Home Agent 0 (rev 04) 7f:0f.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Target Address/Thermal Registers (rev 04) 7f:0f.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 RAS Registers (rev 04) 7f:0f.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers (rev 04) 7f:0f.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers (rev 04) 7f:0f.4 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers (rev 04) 7f:0f.5 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers (rev 04) 7f:10.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 0 (rev 04) 7f:10.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 1 (rev 04) 7f:10.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 0 (rev 04) 7f:10.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 1 (rev 04) 7f:10.4 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 2 (rev 04) 7f:10.5 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 3 (rev 04) 7f:10.6 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 2 (rev 04) 7f:10.7 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 3 (rev 04) 7f:13.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 R2PCIe (rev 04) 7f:13.1 Performance counters: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 R2PCIe (rev 04) 7f:13.4 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Ring Registers (rev 04) 7f:13.5 Performance counters: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Ring Performance Ring Monitoring (rev 04) 7f:16.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 System Address Decoder (rev 04) 7f:16.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Broadcast Registers (rev 04) 7f:16.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Broadcast Registers (rev 04) 80:00.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port in DMI2 Mode (rev 04) 80:01.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 1a (rev 04) 80:02.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 2a (rev 04) 80:03.0 PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 3a (rev 04) 80:04.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 0 (rev 04) 80:04.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 1 (rev 04) 80:04.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 2 (rev 04) 80:04.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 3 (rev 04) 80:04.4 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 4 (rev 04) 80:04.5 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 5 (rev 04) 80:04.6 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 6 (rev 04) 80:04.7 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Crystal Beach DMA Channel 7 (rev 04) 80:05.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 VTd/Memory Map/Misc (rev 04) 80:05.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 IIO RAS (rev 04) 80:05.4 PIC: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 IOAPIC (rev 04) 83:00.0 VGA compatible controller: NVIDIA Corporation GK110GL [Quadro K6000] (rev a1) 83:00.1 Audio device: NVIDIA Corporation GK110 HDMI Audio (rev a1) 84:00.0 VGA compatible controller: NVIDIA Corporation GK110GL [Quadro K6000] (rev a1) 84:00.1 Audio device: NVIDIA Corporation GK110 HDMI Audio (rev a1) ff:08.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link 0 (rev 04) ff:09.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Link 1 (rev 04) ff:0a.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 0 (rev 04) ff:0a.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 1 (rev 04) ff:0a.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 2 (rev 04) ff:0a.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Power Control Unit 3 (rev 04) ff:0b.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 UBOX Registers (rev 04) ff:0b.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 UBOX Registers (rev 04) ff:0c.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) ff:0c.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) ff:0c.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) ff:0d.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) ff:0d.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) ff:0d.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Unicast Registers (rev 04) ff:0e.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Home Agent 0 (rev 04) ff:0e.1 Performance counters: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Home Agent 0 (rev 04) ff:0f.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Target Address/Thermal Registers (rev 04) ff:0f.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 RAS Registers (rev 04) ff:0f.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers (rev 04) ff:0f.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers (rev 04) ff:0f.4 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers (rev 04) ff:0f.5 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder Registers (rev 04) ff:10.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 0 (rev 04) ff:10.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 1 (rev 04) ff:10.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 0 (rev 04) ff:10.3 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 1 (rev 04) ff:10.4 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 2 (rev 04) ff:10.5 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 Thermal Control 3 (rev 04) ff:10.6 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 2 (rev 04) ff:10.7 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR Registers 3 (rev 04) ff:13.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 R2PCIe (rev 04) ff:13.1 Performance counters: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 R2PCIe (rev 04) ff:13.4 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Ring Registers (rev 04) ff:13.5 Performance counters: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 QPI Ring Performance Ring Monitoring (rev 04) ff:16.0 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 System Address Decoder (rev 04) ff:16.1 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Broadcast Registers (rev 04) ff:16.2 System peripheral: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 Broadcast Registers (rev 04) Quote Link to comment
charleslabri Posted February 6, 2016 Author Share Posted February 6, 2016 Here is my IOMMU group list: /sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/2/devices/0000:00:02.0 /sys/kernel/iommu_groups/3/devices/0000:00:03.0 /sys/kernel/iommu_groups/4/devices/0000:00:04.0 /sys/kernel/iommu_groups/4/devices/0000:00:04.1 /sys/kernel/iommu_groups/4/devices/0000:00:04.2 /sys/kernel/iommu_groups/4/devices/0000:00:04.3 /sys/kernel/iommu_groups/4/devices/0000:00:04.4 /sys/kernel/iommu_groups/4/devices/0000:00:04.5 /sys/kernel/iommu_groups/4/devices/0000:00:04.6 /sys/kernel/iommu_groups/4/devices/0000:00:04.7 /sys/kernel/iommu_groups/5/devices/0000:00:05.0 /sys/kernel/iommu_groups/5/devices/0000:00:05.2 /sys/kernel/iommu_groups/5/devices/0000:00:05.4 /sys/kernel/iommu_groups/6/devices/0000:00:11.0 /sys/kernel/iommu_groups/7/devices/0000:00:16.0 /sys/kernel/iommu_groups/7/devices/0000:00:16.1 /sys/kernel/iommu_groups/8/devices/0000:00:1a.0 /sys/kernel/iommu_groups/9/devices/0000:00:1b.0 /sys/kernel/iommu_groups/10/devices/0000:00:1c.0 /sys/kernel/iommu_groups/11/devices/0000:00:1c.7 /sys/kernel/iommu_groups/12/devices/0000:00:1d.0 /sys/kernel/iommu_groups/13/devices/0000:00:1e.0 /sys/kernel/iommu_groups/14/devices/0000:00:1f.0 /sys/kernel/iommu_groups/14/devices/0000:00:1f.2 /sys/kernel/iommu_groups/14/devices/0000:00:1f.3 /sys/kernel/iommu_groups/14/devices/0000:00:1f.6 /sys/kernel/iommu_groups/15/devices/0000:01:00.0 /sys/kernel/iommu_groups/16/devices/0000:02:00.0 /sys/kernel/iommu_groups/16/devices/0000:02:00.1 /sys/kernel/iommu_groups/17/devices/0000:03:00.0 /sys/kernel/iommu_groups/17/devices/0000:03:00.1 /sys/kernel/iommu_groups/18/devices/0000:04:00.0 /sys/kernel/iommu_groups/19/devices/0000:05:00.0 /sys/kernel/iommu_groups/20/devices/0000:05:00.1 /sys/kernel/iommu_groups/21/devices/0000:07:00.0 /sys/kernel/iommu_groups/22/devices/0000:08:00.0 /sys/kernel/iommu_groups/22/devices/0000:09:00.0 /sys/kernel/iommu_groups/22/devices/0000:0a:00.0 /sys/kernel/iommu_groups/23/devices/0000:08:01.0 /sys/kernel/iommu_groups/24/devices/0000:7f:08.0 /sys/kernel/iommu_groups/25/devices/0000:7f:09.0 /sys/kernel/iommu_groups/26/devices/0000:7f:0a.0 /sys/kernel/iommu_groups/26/devices/0000:7f:0a.1 /sys/kernel/iommu_groups/26/devices/0000:7f:0a.2 /sys/kernel/iommu_groups/26/devices/0000:7f:0a.3 /sys/kernel/iommu_groups/27/devices/0000:7f:0b.0 /sys/kernel/iommu_groups/27/devices/0000:7f:0b.3 /sys/kernel/iommu_groups/28/devices/0000:7f:0c.0 /sys/kernel/iommu_groups/28/devices/0000:7f:0c.1 /sys/kernel/iommu_groups/28/devices/0000:7f:0c.2 /sys/kernel/iommu_groups/29/devices/0000:7f:0d.0 /sys/kernel/iommu_groups/29/devices/0000:7f:0d.1 /sys/kernel/iommu_groups/29/devices/0000:7f:0d.2 /sys/kernel/iommu_groups/30/devices/0000:7f:0e.0 /sys/kernel/iommu_groups/30/devices/0000:7f:0e.1 /sys/kernel/iommu_groups/31/devices/0000:7f:0f.0 /sys/kernel/iommu_groups/31/devices/0000:7f:0f.1 /sys/kernel/iommu_groups/31/devices/0000:7f:0f.2 /sys/kernel/iommu_groups/31/devices/0000:7f:0f.3 /sys/kernel/iommu_groups/31/devices/0000:7f:0f.4 /sys/kernel/iommu_groups/31/devices/0000:7f:0f.5 /sys/kernel/iommu_groups/32/devices/0000:7f:10.0 /sys/kernel/iommu_groups/32/devices/0000:7f:10.1 /sys/kernel/iommu_groups/32/devices/0000:7f:10.2 /sys/kernel/iommu_groups/32/devices/0000:7f:10.3 /sys/kernel/iommu_groups/32/devices/0000:7f:10.4 /sys/kernel/iommu_groups/32/devices/0000:7f:10.5 /sys/kernel/iommu_groups/32/devices/0000:7f:10.6 /sys/kernel/iommu_groups/32/devices/0000:7f:10.7 /sys/kernel/iommu_groups/33/devices/0000:7f:13.0 /sys/kernel/iommu_groups/33/devices/0000:7f:13.1 /sys/kernel/iommu_groups/33/devices/0000:7f:13.4 /sys/kernel/iommu_groups/33/devices/0000:7f:13.5 /sys/kernel/iommu_groups/34/devices/0000:7f:16.0 /sys/kernel/iommu_groups/34/devices/0000:7f:16.1 /sys/kernel/iommu_groups/34/devices/0000:7f:16.2 /sys/kernel/iommu_groups/35/devices/0000:80:00.0 /sys/kernel/iommu_groups/36/devices/0000:80:01.0 /sys/kernel/iommu_groups/37/devices/0000:80:02.0 /sys/kernel/iommu_groups/38/devices/0000:80:03.0 /sys/kernel/iommu_groups/39/devices/0000:80:04.0 /sys/kernel/iommu_groups/39/devices/0000:80:04.1 /sys/kernel/iommu_groups/39/devices/0000:80:04.2 /sys/kernel/iommu_groups/39/devices/0000:80:04.3 /sys/kernel/iommu_groups/39/devices/0000:80:04.4 /sys/kernel/iommu_groups/39/devices/0000:80:04.5 /sys/kernel/iommu_groups/39/devices/0000:80:04.6 /sys/kernel/iommu_groups/39/devices/0000:80:04.7 /sys/kernel/iommu_groups/40/devices/0000:80:05.0 /sys/kernel/iommu_groups/40/devices/0000:80:05.2 /sys/kernel/iommu_groups/40/devices/0000:80:05.4 /sys/kernel/iommu_groups/41/devices/0000:83:00.0 /sys/kernel/iommu_groups/41/devices/0000:83:00.1 /sys/kernel/iommu_groups/42/devices/0000:84:00.0 /sys/kernel/iommu_groups/42/devices/0000:84:00.1 /sys/kernel/iommu_groups/43/devices/0000:ff:08.0 /sys/kernel/iommu_groups/44/devices/0000:ff:09.0 /sys/kernel/iommu_groups/45/devices/0000:ff:0a.0 /sys/kernel/iommu_groups/45/devices/0000:ff:0a.1 /sys/kernel/iommu_groups/45/devices/0000:ff:0a.2 /sys/kernel/iommu_groups/45/devices/0000:ff:0a.3 /sys/kernel/iommu_groups/46/devices/0000:ff:0b.0 /sys/kernel/iommu_groups/46/devices/0000:ff:0b.3 /sys/kernel/iommu_groups/47/devices/0000:ff:0c.0 /sys/kernel/iommu_groups/47/devices/0000:ff:0c.1 /sys/kernel/iommu_groups/47/devices/0000:ff:0c.2 /sys/kernel/iommu_groups/48/devices/0000:ff:0d.0 /sys/kernel/iommu_groups/48/devices/0000:ff:0d.1 /sys/kernel/iommu_groups/48/devices/0000:ff:0d.2 /sys/kernel/iommu_groups/49/devices/0000:ff:0e.0 /sys/kernel/iommu_groups/49/devices/0000:ff:0e.1 /sys/kernel/iommu_groups/50/devices/0000:ff:0f.0 /sys/kernel/iommu_groups/50/devices/0000:ff:0f.1 /sys/kernel/iommu_groups/50/devices/0000:ff:0f.2 /sys/kernel/iommu_groups/50/devices/0000:ff:0f.3 /sys/kernel/iommu_groups/50/devices/0000:ff:0f.4 /sys/kernel/iommu_groups/50/devices/0000:ff:0f.5 /sys/kernel/iommu_groups/51/devices/0000:ff:10.0 /sys/kernel/iommu_groups/51/devices/0000:ff:10.1 /sys/kernel/iommu_groups/51/devices/0000:ff:10.2 /sys/kernel/iommu_groups/51/devices/0000:ff:10.3 /sys/kernel/iommu_groups/51/devices/0000:ff:10.4 /sys/kernel/iommu_groups/51/devices/0000:ff:10.5 /sys/kernel/iommu_groups/51/devices/0000:ff:10.6 /sys/kernel/iommu_groups/51/devices/0000:ff:10.7 /sys/kernel/iommu_groups/52/devices/0000:ff:13.0 /sys/kernel/iommu_groups/52/devices/0000:ff:13.1 /sys/kernel/iommu_groups/52/devices/0000:ff:13.4 /sys/kernel/iommu_groups/52/devices/0000:ff:13.5 /sys/kernel/iommu_groups/53/devices/0000:ff:16.0 /sys/kernel/iommu_groups/53/devices/0000:ff:16.1 /sys/kernel/iommu_groups/53/devices/0000:ff:16.2 Quote Link to comment
charleslabri Posted February 9, 2016 Author Share Posted February 9, 2016 nobody? I followed the directions in here: http://lime-technology.com/wiki/index.php/UnRAID_6/VM_Management#Assigning_Graphics_Devices_to_Virtual_Machines_.28GPU_Pass_Through.29 x) tried OVMF machine x) BIOS update x) tried Q35 x) custom ROM for video card None of these provided the solution. The two PCIx slots that aren't working for passthrough are set in affinity to CPU2. I pulled the two working passthrough cards that were in PCIx slots associated with CPU1, and it did not change the behavior of the CPU2 PCIx passthrough slots- still just a black screen. BTW this is a Supermicro X9DRG-QF motherboard. Quote Link to comment
jonp Posted February 9, 2016 Share Posted February 9, 2016 Try these: http://lime-technology.com/wiki/index.php/UnRAID_6/VM_Management#Help.21_Failed_to_set_iommu_for_container:_Operation_not_permitted When you get to modifying the syslinux, add iommu=pt as well so it looks like this: label unRAID OS menu default kernel /bzimage append iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot Quote Link to comment
charleslabri Posted February 12, 2016 Author Share Posted February 12, 2016 That seems to do it. Thanks! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.