1812 Posted February 9, 2018 Author Share Posted February 9, 2018 On 8/12/2017 at 11:32 AM, 1812 said: append pcie_acs_override=id:XXXX:XXXX,YYYY:YYYY initrd=/bzroot 14 hours ago, burningstarIV said: Picked up a dl360e gen8 for some pfsense passthrough, but having some issues with its iommu groups and not working. Have tried the stub and vfio-pci.ids, with vfio_iommu_type1.allow_unsafe_interrupts=1 - neither has worked. I think its a 2014 bios and didn't see a newer one that I could get in the 2017 updates .... passing through a hp NC364T intel 4 port. Do you have any experience with it, or would a dell branded Broadcom work any differently ? Was trying on my 360 g6 but was running into rmrr issues even with a pre 2012 bios .. gave up and got this gen 8, which is great, but would be better if the network card worked ... IOMMU group 25 [111d:8018] 09:02.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch (rev 0e) [8086:10bc] 0a:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 0a:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) IOMMU group 26 [111d:8018] 09:04.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch (rev 0e) [8086:10bc] 0b:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 0b:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) The bridge is causing the problem on passthrough. Try this to see if you can pry it away from the ethernet ports: append pcie_acs_override=id:8086:10bc vfio_iommu_type1.allow_unsafe_interrupts=1 vfio-pci.ids=,8086:10bc initrd=/bzroot or append pcie_acs_override=id:111d:8018,8086:10bc vfio_iommu_type1.allow_unsafe_interrupts=1 vfio-pci.ids=111d:8018,8086:10bc initrd=/bzroot if that doesn't work and the bridge is still with the 2 ports in the groupings, then you can try manually adding the bridge for the group. The xml would look like this (per your posted grouping for 26 ): <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x04' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x1'/> </source> </hostdev> I removed the vm address lines as they will be populated on vm start. I'm not sure if that will work, but it could. On my machines, I use a PCIe card out of a semi-irrational belief that it's better to keep the firewall traffic slightly more isolated off my server that way, than using the internal For reference, I use append pcie_acs_override=downstream vfio_iommu_type1.allow_unsafe_interrupts=1 vfio-pci.ids=1b73:1100,8086:10e8 initrd=/bzroot and my card splits to: IOMMU group 14: [111d:8018] 02:02.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch (rev 0e) IOMMU group 15: [111d:8018] 02:04.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch (rev 0e) IOMMU group 16: [8086:10e8] 03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) IOMMU group 17: [8086:10e8] 03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) IOMMU group 18: [8086:10e8] 04:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) IOMMU group 19: [8086:10e8] 04:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) Quote Link to comment
burningstarIV Posted February 10, 2018 Share Posted February 10, 2018 added that all in but got a new and improved error - looking up on search just seeing acs enabling - let me know what you think default /syslinux/menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append pcie_acs_override=id:111d:8018,8086:10bc vfio_iommu_type1.allow_unsafe_interrupts=1 vfio-pci.ids=111d:8018,8086:10bc initrd=/bzroot label unRAID OS GUI Mode kernel /bzimage append initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest IOMMU group 25: [111d:8018] 09:02.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch (rev 0e) IOMMU group 26: [111d:8018] 09:04.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch (rev 0e) IOMMU group 27: [8086:10bc] 0a:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 0a:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) IOMMU group 28: [8086:10bc] 0b:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 0b:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x02' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0a' slot='0x00' function='0x1'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x04' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x1'/> </source> </hostdev> Quote Link to comment
D874as Posted February 10, 2018 Share Posted February 10, 2018 Hello, i am also trying to passthrough a HD 6450 from my Microserver gen8 running the latest stable unRaid version. Having the same problem. Here is my config: default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append pcie_acs_override=id:8086:0151,1002:6779,1002:aa98 vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot label unRAID OS GUI Mode kernel /bzimage append initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest IOMMU group 0: [8086:0158] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/Ivy Bridge DRAM Controller (rev 09) IOMMU group 1: [8086:0151] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) IOMMU group 2: [8086:015d] 00:06.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) IOMMU group 3: [8086:1c2d] 00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05) IOMMU group 4: [8086:1c10] 00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5) IOMMU group 5: [8086:1c18] 00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5) IOMMU group 6: [8086:1c1c] 00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5) IOMMU group 7: [8086:1c1e] 00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5) IOMMU group 8: [8086:1c26] 00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05) IOMMU group 9: [8086:244e] 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5) IOMMU group 10: [8086:1c54] 00:1f.0 ISA bridge: Intel Corporation C204 Chipset Family LPC Controller (rev 05) [8086:1c02] 00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller (rev 05) IOMMU group 11: [1002:6779] 07:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM] [1002:aa98] 07:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM] IOMMU group 12: [14e4:165f] 03:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f] 03:00.1 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe IOMMU group 13: [1912:0014] 04:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03) IOMMU group 14: [103c:3306] 01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 05) [102b:0533] 01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200EH [103c:3307] 01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 05) [103c:3300] 01:00.4 USB controller: Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller (rev 02) Here are the logs: Feb 10 10:53:14 Server kernel: vfio-pci 0000:07:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Feb 10 10:53:14 Server kernel: br0: port 2(vnet0) entered blocking state Feb 10 10:53:14 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 10:53:14 Server kernel: device vnet0 entered promiscuous mode Feb 10 10:53:14 Server kernel: br0: port 2(vnet0) entered blocking state Feb 10 10:53:14 Server kernel: br0: port 2(vnet0) entered forwarding state Feb 10 10:53:15 Server kernel: pci 0000:07:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor. Feb 10 10:53:15 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 10:53:15 Server kernel: device vnet0 left promiscuous mode Feb 10 10:53:15 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 10:53:15 Server kernel: vfio-pci 0000:07:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Quote Link to comment
1812 Posted February 10, 2018 Author Share Posted February 10, 2018 2 hours ago, D874as said: Hello, i am also trying to passthrough a HD 6450 from my Microserver gen8 running the latest stable unRaid version. Having the same problem. Here is my config: default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append pcie_acs_override=id:8086:0151,1002:6779,1002:aa98 vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot label unRAID OS GUI Mode kernel /bzimage append initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest IOMMU group 0: [8086:0158] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/Ivy Bridge DRAM Controller (rev 09) IOMMU group 1: [8086:0151] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) IOMMU group 2: [8086:015d] 00:06.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) IOMMU group 3: [8086:1c2d] 00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05) IOMMU group 4: [8086:1c10] 00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5) IOMMU group 5: [8086:1c18] 00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5) IOMMU group 6: [8086:1c1c] 00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5) IOMMU group 7: [8086:1c1e] 00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5) IOMMU group 8: [8086:1c26] 00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05) IOMMU group 9: [8086:244e] 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5) IOMMU group 10: [8086:1c54] 00:1f.0 ISA bridge: Intel Corporation C204 Chipset Family LPC Controller (rev 05) [8086:1c02] 00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller (rev 05) IOMMU group 11: [1002:6779] 07:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM] [1002:aa98] 07:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM] IOMMU group 12: [14e4:165f] 03:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f] 03:00.1 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe IOMMU group 13: [1912:0014] 04:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03) IOMMU group 14: [103c:3306] 01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 05) [102b:0533] 01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200EH [103c:3307] 01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 05) [103c:3300] 01:00.4 USB controller: Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller (rev 02) Here are the logs: Feb 10 10:53:14 Server kernel: vfio-pci 0000:07:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Feb 10 10:53:14 Server kernel: br0: port 2(vnet0) entered blocking state Feb 10 10:53:14 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 10:53:14 Server kernel: device vnet0 entered promiscuous mode Feb 10 10:53:14 Server kernel: br0: port 2(vnet0) entered blocking state Feb 10 10:53:14 Server kernel: br0: port 2(vnet0) entered forwarding state Feb 10 10:53:15 Server kernel: pci 0000:07:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor. Feb 10 10:53:15 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 10:53:15 Server kernel: device vnet0 left promiscuous mode Feb 10 10:53:15 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 10:53:15 Server kernel: vfio-pci 0000:07:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none If you read the section above regarding RMRR problems, you'll see it relates to your audio portion of the gpu. Different problem than previous posters. Try removing 8086:0151 from the override line and see if that allows the video and audio to be split into different IOUMMU groups, and if so, don't pass through the audio to the vm. Quote Link to comment
1812 Posted February 10, 2018 Author Share Posted February 10, 2018 16 hours ago, burningstarIV said: added that all in but got a new and improved error - looking up on search just seeing acs enabling - let me know what you think default /syslinux/menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append pcie_acs_override=id:111d:8018,8086:10bc vfio_iommu_type1.allow_unsafe_interrupts=1 vfio-pci.ids=111d:8018,8086:10bc initrd=/bzroot label unRAID OS GUI Mode kernel /bzimage append initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest IOMMU group 25: [111d:8018] 09:02.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch (rev 0e) IOMMU group 26: [111d:8018] 09:04.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch (rev 0e) IOMMU group 27: [8086:10bc] 0a:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 0a:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) IOMMU group 28: [8086:10bc] 0b:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) [8086:10bc] 0b:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) (rev 06) <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x02' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0a' slot='0x00' function='0x1'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x04' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> </source> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x1'/> </source> </hostdev> WELL, at least we got them divided into groups now. This is a new error one for me. Try unstubbing the bridge id and reboot. If same error occurs, try unstubbing the card id and reboot (confirming iommu groupings along the way each time.) If that doesn't work, then it may be time for research. Quote Link to comment
D874as Posted February 10, 2018 Share Posted February 10, 2018 (edited) After editing config file to: default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append pcie_acs_override=id:1002:6779,1002:aa98 vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot label unRAID OS GUI Mode kernel /bzimage append initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest I got following output: IOMMU group 0: [8086:0158] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/Ivy Bridge DRAM Controller (rev 09) IOMMU group 1: [8086:0151] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) [1002:6779] 07:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM] [1002:aa98] 07:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM] IOMMU group 2: [8086:015d] 00:06.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) IOMMU group 3: [8086:1c2d] 00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05) IOMMU group 4: [8086:1c10] 00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5) IOMMU group 5: [8086:1c18] 00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5) IOMMU group 6: [8086:1c1c] 00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5) IOMMU group 7: [8086:1c1e] 00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5) IOMMU group 8: [8086:1c26] 00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05) IOMMU group 9: [8086:244e] 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5) IOMMU group 10: [8086:1c54] 00:1f.0 ISA bridge: Intel Corporation C204 Chipset Family LPC Controller (rev 05) [8086:1c02] 00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller (rev 05) IOMMU group 11: [14e4:165f] 03:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f] 03:00.1 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe IOMMU group 12: [1912:0014] 04:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03) IOMMU group 13: [103c:3306] 01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 05) [102b:0533] 01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200EH [103c:3307] 01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 05) [103c:3300] 01:00.4 USB controller: Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller (rev 02) trying to create a vm with only the video passthrough didn't help. I Still get the same log: Feb 10 13:28:14 Server kernel: vfio-pci 0000:07:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered blocking state Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 13:28:14 Server kernel: device vnet0 entered promiscuous mode Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered blocking state Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered forwarding state Feb 10 13:28:14 Server kernel: pci 0000:07:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor. Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 13:28:14 Server kernel: device vnet0 left promiscuous mode Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 13:28:14 Server kernel: vfio-pci 0000:07:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Feb 10 13:29:13 Server kernel: usb 3-1: USB disconnect, device number 2 Feb 10 13:29:13 Server acpid: input device has been disconnected, fd 4 Edited February 10, 2018 by D874as Quote Link to comment
1812 Posted February 10, 2018 Author Share Posted February 10, 2018 20 minutes ago, D874as said: After editing config file to: default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label unRAID OS menu default kernel /bzimage append pcie_acs_override=id:1002:6779,1002:aa98 vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot label unRAID OS GUI Mode kernel /bzimage append initrd=/bzroot,/bzroot-gui label unRAID OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label unRAID OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest I got following output: IOMMU group 0: [8086:0158] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/Ivy Bridge DRAM Controller (rev 09) IOMMU group 1: [8086:0151] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) [1002:6779] 07:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM] [1002:aa98] 07:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM] IOMMU group 2: [8086:015d] 00:06.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) IOMMU group 3: [8086:1c2d] 00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05) IOMMU group 4: [8086:1c10] 00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5) IOMMU group 5: [8086:1c18] 00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5) IOMMU group 6: [8086:1c1c] 00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5) IOMMU group 7: [8086:1c1e] 00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5) IOMMU group 8: [8086:1c26] 00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05) IOMMU group 9: [8086:244e] 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5) IOMMU group 10: [8086:1c54] 00:1f.0 ISA bridge: Intel Corporation C204 Chipset Family LPC Controller (rev 05) [8086:1c02] 00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port Desktop SATA AHCI Controller (rev 05) IOMMU group 11: [14e4:165f] 03:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f] 03:00.1 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe IOMMU group 12: [1912:0014] 04:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03) IOMMU group 13: [103c:3306] 01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 05) [102b:0533] 01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200EH [103c:3307] 01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 05) [103c:3300] 01:00.4 USB controller: Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller (rev 02) trying to create a vm with only the video passthrough didn't help. I Still get the same log: Feb 10 13:28:14 Server kernel: vfio-pci 0000:07:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered blocking state Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 13:28:14 Server kernel: device vnet0 entered promiscuous mode Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered blocking state Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered forwarding state Feb 10 13:28:14 Server kernel: pci 0000:07:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor. Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 13:28:14 Server kernel: device vnet0 left promiscuous mode Feb 10 13:28:14 Server kernel: br0: port 2(vnet0) entered disabled state Feb 10 13:28:14 Server kernel: vfio-pci 0000:07:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none Feb 10 13:29:13 Server kernel: usb 3-1: USB disconnect, device number 2 Feb 10 13:29:13 Server acpid: input device has been disconnected, fd 4 It put the bridge back in with the GPU...... fun. is there a different slot you can try? my guess is no since it's a micro server.... to be clear, this is an HP issue, not unRaid issue. HP has been a pain about actually fixing this. And their "fix" is a workaround that is not a simple as "click this to make it work." Quote Link to comment
D874as Posted February 10, 2018 Share Posted February 10, 2018 3 minutes ago, 1812 said: It put the bridge back in with the GPU...... fun. is there a different slot you can try? my guess is no since it's a micro server.... to be clear, this is an HP issue, not unRaid issue. HP has been a pain about actually fixing this. And their "fix" is a workaround that is not a simple as "click this to make it work." yes I am using a Microserver .. how much of a workaround is that? I am still new to this topic, also doing some reading on IOMMU etc. Quote Link to comment
burningstarIV Posted February 11, 2018 Share Posted February 11, 2018 From reading back and forth on all this stuff it seems the micro servers have a bunch of problems. On my 580 I had to get an older 2012 firmware to get around the rmrr. Try to look into your bios and see if running the latest version. From there you may be able to change the network over to non shared memory to get past this. Not sure what the success rate is but its a problem with how they use the intel cpus Quote Link to comment
burningstarIV Posted February 11, 2018 Share Posted February 11, 2018 Taking off the bridge goes back to original error - looks like ill get a Broadcom one and see if theres a better result Quote Link to comment
1812 Posted February 11, 2018 Author Share Posted February 11, 2018 15 hours ago, D874as said: yes I am using a Microserver .. how much of a workaround is that? I am still new to this topic, also doing some reading on IOMMU etc. This is their fix for the gen 8/9 servers: https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566 Not sure how applicable it is for micro servers. Here is also a redhat white paper on it : https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf Additionally, here is a proxmox post (in German) about passthrough problems on a gen 8 microserver. https://forum.proxmox.com/threads/hp-proliant-microserver-gen8-raidcontroller-hp-p410-passthrough-probleme.30547/ They also describe changing the boot to include "iommu=on" or "intel_iommu=on" but I have not tried those. (there are also other options intended for AMD processors which include "iommu=pt" and "iommu=1" . I also remember reading somewhere that in some newer bios for newer servers that HP had an option (don't remember what it was called) that enables the exclusion of cards for passthrough (but this was in passing about 6 months ago) Quote Link to comment
D874as Posted February 11, 2018 Share Posted February 11, 2018 44 minutes ago, 1812 said: This is their fix for the gen 8/9 servers: https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566 Not sure how applicable it is for micro servers. Here is also a redhat white paper on it : https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf Additionally, here is a proxmox post (in German) about passthrough problems on a gen 8 microserver. https://forum.proxmox.com/threads/hp-proliant-microserver-gen8-raidcontroller-hp-p410-passthrough-probleme.30547/ They also describe changing the boot to include "iommu=on" or "intel_iommu=on" but I have not tried those. (there are also other options intended for AMD processors which include "iommu=pt" and "iommu=1" . I also remember reading somewhere that in some newer bios for newer servers that HP had an option (don't remember what it was called) that enables the exclusion of cards for passthrough (but this was in passing about 6 months ago) thanks @1812 seems like I need to compile my own kernel. Maybe it also works for a custom made unRaid kernel? First of all, I need to educate myself about kernel building Quote Link to comment
1812 Posted February 11, 2018 Author Share Posted February 11, 2018 32 minutes ago, D874as said: thanks @1812 seems like I need to compile my own kernel. Maybe it also works for a custom made unRaid kernel? First of all, I need to educate myself about kernel building Also, maybe look into "iommu_inclusive_mapping=1" I've seen a few instances online where it has been used to help with rmrr issues but really know very little about it. Quote Link to comment
peeqi Posted February 15, 2018 Share Posted February 15, 2018 Hi, i have a microserver gen 8 and i'm getting absolutely hammered with the ACPI errors. Quote Jan 13 16:00:52 UNRAID kernel: ACPI Error: SMBus/IPMI/GenericSerialBus write requires Buffer of length 66, found length 32 (20170728/exfield-427) Jan 13 16:00:52 UNRAID kernel: ACPI Error: Method parse/execution failed \_SB.PMI0._PMM, AE_AML_BUFFER_LIMIT (20170728/psparse-550) Jan 13 16:00:52 UNRAID kernel: ACPI Exception: AE_AML_BUFFER_LIMIT, Evaluating _PMM (20170728/power_meter-338) Jan 13 16:00:53 UNRAID kernel: ACPI Error: SMBus/IPMI/GenericSerialBus write requires Buffer of length 66, found length 32 (20170728/exfield-427) Jan 13 16:00:53 UNRAID kernel: ACPI Error: Method parse/execution failed \_SB.PMI0._PMM, AE_AML_BUFFER_LIMIT (20170728/psparse-550) Jan 13 16:00:53 UNRAID kernel: ACPI Exception: AE_AML_BUFFER_LIMIT, Evaluating _PMM (20170728/power_meter-338) .... Jan 13 16:01:01 UNRAID kernel: ACPI Error: SMBus/IPMI/GenericSerialBus write requires Buffer of length 66, found length 32 (20170728/exfield-427) Jan 13 16:01:01 UNRAID kernel: ACPI Error: Method parse/execution failed \_SB.PMI0._PMM, AE_AML_BUFFER_LIMIT (20170728/psparse-550) Jan 13 16:01:01 UNRAID kernel: ACPI Exception: AE_AML_BUFFER_LIMIT, Evaluating _PMM (20170728/power_meter-338) Jan 13 16:01:02 UNRAID kernel: ACPI Error: SMBus/IPMI/GenericSerialBus write requires Buffer of length 66, found length 32 (20170728/exfield-427) Jan 13 16:01:02 UNRAID kernel: ACPI Error: Method parse/execution failed \_SB.PMI0._PMM, AE_AML_BUFFER_LIMIT (20170728/psparse-550) Jan 13 16:01:02 UNRAID kernel: ACPI Exception: AE_AML_BUFFER_LIMIT, Evaluating _PMM (20170728/power_meter-338) I've tried the suggested fix of modifying the /etc/sensors3.conf file and added the following lines at the bottom of the file: Quote chip "power_meter-*" ignore power1 That however does nothing. I also tried to make a new file under /etc/sensors.d/sensor_quirk with the lines above added but that also does nothing. Does anyone know how to fix this? Quote Link to comment
perfecblue Posted February 15, 2018 Share Posted February 15, 2018 (edited) 13 hours ago, peeqi said: Hi, i have a microserver gen 8 and i'm getting absolutely hammered with the ACPI errors. I've tried the suggested fix of modifying the /etc/sensors3.conf file and added the following lines at the bottom of the file: That however does nothing. I also tried to make a new file under /etc/sensors.d/sensor_quirk with the lines above added but that also does nothing. Does anyone know how to fix this? Hi, I also have a Microserver Gen 8 and was having the same ACPI Error message spamming issue as well. I found THIS POST and went ahead and unload the "acpi_power_meter" module from the kernel and it solved the problem. You can add this line to the /boot/config/go file found inside the flash drive hosting unRAID system to have it unload the module during every boot: rmmod acpi_power_meter Hope this helps! p.s. I found another article from Novell which also sort of explains this situation. Edited February 15, 2018 by perfecblue 1 Quote Link to comment
ASmigielski Posted February 18, 2018 Share Posted February 18, 2018 On 8/13/2017 at 2:57 AM, IrishBiker said: Hi My current setup has the following: IOMMU group 1 [8086:0151] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) [10de:104a] 07:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 610] (rev a1) [10de:0e08] 07:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1) So using Step 2.a I added the following line to my syslinux.cfg file append pcie_acs_override=id:10de:104a,10de:0e08 initrd=/bzroot But when I rebooted the box its still the same, any ideas? Oh I also have Enable PCIe ACS Override switched on. add this instead append pcie_acs_override=8086:0151 it will separate PCI bridge and leave the other two in their own group Quote Link to comment
kabloomy Posted March 11, 2018 Share Posted March 11, 2018 (edited) On 12/02/2018 at 1:46 AM, D874as said: thanks @1812 seems like I need to compile my own kernel. Maybe it also works for a custom made unRaid kernel? First of all, I need to educate myself about kernel building I have a Microserver Gen8 too and want to passthrough my GPU to my VM. I ended up compiling my own kernel for UnRAID with the RMRR check patched out, as what the Proxmox guys did. Attached is kernel for "Linux 4.14.16-unRAID." Copy 'bzimage-new' into '/boot' and modify your syslinux to use the new kernel. I think this kernel will only work on the specific UnRAID version I'm running right now (v6.4.1). My config as follows (I didn't need to do PCS override or 'vfio_iommu_type1.allow_unsafe_interrupts=1' for me): label unRAID OS (PCIe passthrough - No RMRR chk) menu default kernel /bzimage-new append isolcpus=2,3,6,7 initrd=/bzroot The ACS override separated my GPU from the PCI bridge (IOMMU group 1) into its own IOMMU group 11. IOMMU group 0: [8086:0108] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 Processor Family DRAM Controller (rev 09) IOMMU group 1: [8086:0101] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09) ... [snip] ... IOMMU group 11: [10de:104a] 07:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 610] (rev a1) [10de:0e08] 07:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1) ... <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> I then installed the NVIDIA drivers in my VM... though after a restart Windows says the driver for my GPU can't be loaded because of a problem. I'll need to look into this some more, but the custom kernel seems to bypass the "Device is ineligible for IOMMU domain attach due to platform RMRR requirement". If you want to compile your own next time (a.k.a. every time UnRAID gets updated, I think), I used CHBMB's kernel compile script here and the Proxmox kernel patch here. I've also attached my modified script and patch. I compile on the UnRAID machine itself. My modified script just copies the Proxmox patch to '/usr/src/linux-*' and comments out fetching of CHBMB's DVB '.config' on line 44. Thanks CHBMB! bzimage-new kernel-compile-module.sh remove_rmrr_check.patch Update 14/03/18: After overcoming the RMRR hurdle, I found out the GPU driver could not be loaded (code 43) because of NVIDIA deciding to intentionally block the driver if it detects it's being loaded in a VM to get you to buy their workstation Quattro cards. This is widely documented and can be bypassed with: <features> ... <hyperv> ... <vendor_id state='on' value='blahblahblah'/> </hyperv> <kvm> <hidden state='on'/> </kvm> </features> However, this still didn't work; I still got the code 43 problem. After more googling and dumping my GPU BIOS and whatnot, I think it's because my GPU BIOS doesn't have EFI support. When I substituted my GPU BIOS in the VM config with a similar variant with UEFI support (I couldn't find one for my exact model), the driver loaded correctly and I got output through the HDMI! This was great until my whole host system crashed, probably due to using an incompatible GPU BIOS. <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/> </source> <rom file='/mnt/user/vm-iso/Gigabyte.GT610.1024.130107-UEFI.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> Update 16/03/18: I got my Gigabyte GT 610 Silent passthrough partially working. As suspected, the GPU BIOS did not contain UEFI support. I dumped my vBIOS (I did set the GPU to "secondary optional" in the HP RBSU) and confirmed the ROM was not corrupt (hopefully—the md5 happened to match Techpowerup's database and rom-parser didn't complain). I then used the GOPUpd tool to inject the UEFI bit of firmware into my BIOS image (the GOP was included in the tool for supported GPUs—luckily mine, a GF119, was. With the new ROM file, I specified it in my VM config, as above and the code 43 error was gone. Everything appears to work, but when the VM is rebooted, the whole host crashes. I wanted to actually try and flash my patched vBIOS to my GPU, but the EEPROM size on the GPU appears to be too small to fit the patched vBIOS. I did come across a QVM bug which has similar symptoms to mine, except mine crashes regardless of how long I have the VM running. I am running a Windows Server 2016 host. I did try enabling MSI to see if it will solve my host crashing problem to no avail... Update 25/03/18: I believe my GPU is inherently not compatible because of it lacking the feature to reset properly. This in turn causes my host to crash and reboot if a GPU is detached from the VM and re-attached again. Update 10/04/18: Late update, but I tried using SeaBIOS on Ubuntu with the GT610 but there was some unrelated bug and I got a login loop with the Linux Nvidia drivers installed. In the end, I swapped the GPU with a GT520 (not fanless unfortunately), followed all the same steps above with dumping the vBIOS, and it no longer crashes the host under UEFI Windows Server 2016. Edited April 10, 2018 by kabloomy 1 Quote Link to comment
1812 Posted March 11, 2018 Author Share Posted March 11, 2018 56 minutes ago, kabloomy said: I have a Microserver Gen8 too and want to passthrough my GPU to my VM. I ended up compiling my own kernel for UnRAID with the RMRR check patched out, as what the Proxmox guys did. Attached is kernel for "Linux 4.14.16-unRAID." Copy 'bzimage-new' into '/boot' and modify your syslinux to use the new kernel. I think this kernel will only work on the specific UnRAID version I'm running right now (v6.4.1). My config as follows: label unRAID OS (PCIe passthrough - No RMRR chk) menu default kernel /bzimage-new append isolcpus=2,3,6,7 pcie_acs_override=downstream vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot The ACS override separated my GPU from the PCI bridge (IOMMU group 1) into its own IOMMU group 11. IOMMU group 0: [8086:0108] 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 Processor Family DRAM Controller (rev 09) IOMMU group 1: [8086:0101] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09) ... [snip] ... IOMMU group 11: [10de:104a] 07:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 610] (rev a1) [10de:0e08] 07:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio Controller (rev a1) ... <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> I then installed the NVIDIA drivers in my VM... though after a restart Windows says the driver for my GPU can't be loaded because of a problem. I'll need to look into this some more, but the custom kernel seems to bypass the "Device is ineligible for IOMMU domain attach due to platform RMRR requirement". If you want to compile your own next time (a.k.a. every time UnRAID gets updated, I think), I used CHBMB's kernel compile script here and the Proxmox kernel patch here. I've also attached my modified script and patch. I compile on the UnRAID machine itself. My modified script just copies the Proxmox patch to '/usr/src/linux-*' and comments out fetching of CHBMB's DVB '.config' on line 44. Thanks CHBMB! bzimage-new kernel-compile-module.sh remove_rmrr_check.patch This is very cool! If the vm is using the GPU, then regardless of drivers loading or not, that means it's allowing passthrough that was blocked due to RMRR requirement. But it doesn't mean it's not necessarily playing a part of the driver issue I think.... Hey @limetech, any chance we can have an option (like the acs override selection) to enable the patch via web gui? Or is it going to be too much of a pita to maintain every time a new version of unraid comes out? Would also be nice to add the "allow unsafe interrupts" as well that way. Quote Link to comment
ang3l12 Posted March 12, 2018 Share Posted March 12, 2018 On 2/10/2018 at 7:24 PM, burningstarIV said: From reading back and forth on all this stuff it seems the micro servers have a bunch of problems. On my 580 I had to get an older 2012 firmware to get around the rmrr. Try to look into your bios and see if running the latest version. From there you may be able to change the network over to non shared memory to get past this. Not sure what the success rate is but its a problem with how they use the intel cpus Any chance you could link the firmware for the 580 you used? Im trying to get mine up and running and it has a newer one. Thanks! Quote Link to comment
burningstarIV Posted March 16, 2018 Share Posted March 16, 2018 hey - yeah it took me probably 10 hours of searching sites, directories and old ftps to get this little thing . You will need a hp proliant ssp installed to usb to inject it. Use the hp usb tool to install spp to a usb - and then navigate to the packages directory ( if I remember all right) and put this file in there. Boot sever from it, and use interactive. Then force select the older bios to install. After that you should be clear of RMRR issues. CP016412.scexe Quote Link to comment
ReidS Posted April 3, 2018 Share Posted April 3, 2018 Hi guys, great topic and good reading. This is all pretty new to me, but unRAID seems to run like a champ. I have several HP Proliant series servers and this is all very helpful. I'm hoping to not to deter the topic that's going on here, but I think it does follow the original topic of "to do's" with Proliant servers. I currently have a DL360 G7 with a PCI Creative SB Audigy FX PCIe 5.1. I cannot get this card to show in the VM assignment PCI device list, but that's ok. I'd really like to use this card as a "Sound Card" option, which now shows as a check-off option since installing it. That tells me that unRAID sees enough of it to acknowledge it. When I check the SC box, unRAID will not allow me to boot the VM due to a conflict. I'm currently not running any modified scripts or add-ons/modules. The system is a currently running VM's with zero issues, Win 2012 Term Server, Zentyal, & 2x Windows 10 Pro. I'd like to add sound support to the 2 Windows 10 Pro VM's, but so far no luck with anything. I could add video cards with audio for each VM, but that's not particularly practical seeing all I need is audio and not graphic accelerated workstations and I only have 2 PCI slots in total. I've searched and searched for answers, but keep coming across the answer of people need to assign the sound card and the corresponding video card to the same VM, which I understand, but it doesn't apply to my situation. I'll be happy to jump to another post or create a new one, but this seemed most relevant. I'll upload exact verbiage of errors and config data, I'm just not sure of the level of the issue here. Not sure if this is a simple adjustment that I'm overlooking or if need to write a custom script for this situation. Basically, I'm not new to VM environments or systems, but I am new to unRAID and it's ability to do so much with so little. Thank you for any help and efforts. Quote Link to comment
1812 Posted April 3, 2018 Author Share Posted April 3, 2018 2 hours ago, ReidS said: I'll be happy to jump to another post or create a new one, but this seemed most relevant. I'll upload exact verbiage of errors and config data, I'm just not sure of the level of the issue here. Not sure if this is a simple adjustment that I'm overlooking or if need to write a custom script for this situation. Basically, I'm not new to VM environments or systems, but I am new to unRAID and it's ability to do so much with so little. Thank you for any help and efforts. This is the best place to do it, so that if anyone else has the same problem on a proliant, then they can find the answer here. Let us know the error and upload your full diagnostics zip as well. (tools>diagnostics) Quote Link to comment
ReidS Posted April 6, 2018 Share Posted April 6, 2018 The attached image is what I'm seeing when I launch the Windows 10 system with the Sound Card option enabled for this VM. I tried a few different BIOS options in unRAID, but didn't see to have any further luck. Some time back I saw someone talk about making a change to the system config file and I tried adding a line (I forget what line exactly now) to unlock a feature, but this resulted in preventing the system from booting in general. I know that little info doesn't help, but I'm a one man IT team doing lots of work and I'm not always quick to make notes on negative results just in case I might have did something wrong. I like to leave my doors open to re-explore failed attempts. There's way more to be learned from failures then successes. diagnostics-20180406-1252.zip Quote Link to comment
1812 Posted April 6, 2018 Author Share Posted April 6, 2018 1 hour ago, ReidS said: The attached image is what I'm seeing when I launch the Windows 10 system with the Sound Card option enabled for this VM. I tried a few different BIOS options in unRAID, but didn't see to have any further luck. Some time back I saw someone talk about making a change to the system config file and I tried adding a line (I forget what line exactly now) to unlock a feature, but this resulted in preventing the system from booting in general. I know that little info doesn't help, but I'm a one man IT team doing lots of work and I'm not always quick to make notes on negative results just in case I might have did something wrong. I like to leave my doors open to re-explore failed attempts. There's way more to be learned from failures then successes. diagnostics-20180406-1252.zip You need to start with VM passthrough problems on the previous page dealing with unsafe interrupts. Without allowing unsafe interrupts (append vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot) , you will not get any closer to your goal of a working vm. Quote Link to comment
ReidS Posted April 16, 2018 Share Posted April 16, 2018 (edited) Sorry for my delay is response to this. I had to prioritize some other things over this project last week. So I added the line above to allow unsafe interrupts, reselected the audio config for the Win10 system and I'm getting the same exact error above still. I went into the BIOS and tried to manually reassign the audio card to another IRQ and memory address, but the system keeps moving the devices on the same interrupts along with it, like the RAID controller. I read earlier that the RAID controller doesn't play well and am I possible a victim of circumstances here? I have 6 disks in this system running off the RAID controller, but not using the hardware RAID. So using the onboard SATA won't work with only 2 ports. I also have an PCIe external SATA for an external array, but the array isn't plugged in at the moment and it's just a PNP, no BIOS card. I'm just tossing out some thoughts. I'm going to be trying to get this box working 100% by the end of week So I can start the next leg of projects related to this system. If I need to order another audio card, I'm open to suggestions. Thank you and everybody in advance for any help! Edited April 16, 2018 by ReidS Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.