NoobSpy Posted April 20, 2018 Share Posted April 20, 2018 (edited) Hi all, I have a system with two GPUs a RX550 and a GT730 I have a Linux VM with a RX550 passed through. This VM works only one time after a full unraid reboot. As soon as I shut down the VM and and try to restart it I get the: internal error: Unknown PCI header type '127' edit: Just found this in the VM log: Apr 21 03:17:51 Tower kernel: DMAR: [DMA Read] Request device [03:00.0] fault addr f4001d6000 [fault reason 06] PTE Read access is not set I figured out it has to do with the video card(s): When i tried to pass through the GT which is in slot two the VM doesnt boot and doesnt spit out the error. I physically switched the GPUs per slot and again after reboot it works as soon i shut down the VM and restart I get the PCI error. I am not knowledgeable enough to figure this out. Could someone please help? Thank you! ================================== I have included my IOMMU groups below: IOMMU group 0: [8086:6f81] ff:0b.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) [8086:6f36] ff:0b.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) [8086:6f37] ff:0b.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) [8086:6f76] ff:0b.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link Debug (rev 01) IOMMU group 1: [8086:6fe0] ff:0c.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6fe1] ff:0c.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6fe2] ff:0c.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6fe3] ff:0c.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6fe4] ff:0c.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6fe5] ff:0c.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6fe6] ff:0c.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6fe7] ff:0c.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) IOMMU group 2: [8086:6ff8] ff:0f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6ff9] ff:0f.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6ffc] ff:0f.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6ffd] ff:0f.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) [8086:6ffe] ff:0f.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent (rev 01) IOMMU group 3: [8086:6f1d] ff:10.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R2PCIe Agent (rev 01) [8086:6f34] ff:10.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R2PCIe Agent (rev 01) [8086:6f1e] ff:10.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Ubox (rev 01) [8086:6f7d] ff:10.6 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Ubox (rev 01) [8086:6f1f] ff:10.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Ubox (rev 01) IOMMU group 4: [8086:6fa0] ff:12.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 0 (rev 01) [8086:6f30] ff:12.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 0 (rev 01) IOMMU group 5: [8086:6fa8] ff:13.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target Address/Thermal/RAS (rev 01) [8086:6f71] ff:13.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target Address/Thermal/RAS (rev 01) [8086:6faa] ff:13.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address Decoder (rev 01) [8086:6fab] ff:13.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address Decoder (rev 01) [8086:6fac] ff:13.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address Decoder (rev 01) [8086:6fad] ff:13.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address Decoder (rev 01) [8086:6fae] ff:13.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Broadcast (rev 01) [8086:6faf] ff:13.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) IOMMU group 6: [8086:6fb0] ff:14.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Thermal Control (rev 01) [8086:6fb1] ff:14.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Thermal Control (rev 01) [8086:6fb2] ff:14.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Error (rev 01) [8086:6fb3] ff:14.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Error (rev 01) [8086:6fbc] ff:14.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) [8086:6fbd] ff:14.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) [8086:6fbe] ff:14.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) [8086:6fbf] ff:14.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface (rev 01) IOMMU group 7: [8086:6fb4] ff:15.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 2 Thermal Control (rev 01) [8086:6fb5] ff:15.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 3 Thermal Control (rev 01) [8086:6fb6] ff:15.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 2 Error (rev 01) [8086:6fb7] ff:15.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 3 Error (rev 01) IOMMU group 8: [8086:6f68] ff:16.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS (rev 01) [8086:6f6e] ff:16.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Broadcast (rev 01) [8086:6f6f] ff:16.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast (rev 01) IOMMU group 9: [8086:6fd0] ff:17.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Thermal Control (rev 01) [8086:6fb8] ff:17.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) [8086:6fb9] ff:17.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) [8086:6fba] ff:17.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) [8086:6fbb] ff:17.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface (rev 01) IOMMU group 10: [8086:6f98] ff:1e.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) [8086:6f99] ff:1e.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) [8086:6f9a] ff:1e.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) [8086:6fc0] ff:1e.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) [8086:6f9c] ff:1e.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) IOMMU group 11: [8086:6f88] ff:1f.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) [8086:6f8a] ff:1f.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit (rev 01) IOMMU group 12: [8086:6f00] 00:00.0 Host bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DMI2 (rev 01) IOMMU group 13: [8086:6f02] 00:01.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 1 (rev 01) IOMMU group 14: [8086:6f03] 00:01.1 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 1 (rev 01) IOMMU group 15: [8086:6f04] 00:02.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 2 (rev 01) IOMMU group 16: [8086:6f08] 00:03.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 3 (rev 01) IOMMU group 17: [8086:6f28] 00:05.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Map/VTd_Misc/System Management (rev 01) [8086:6f29] 00:05.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D IIO Hot Plug (rev 01) [8086:6f2a] 00:05.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D IIO RAS/Control Status/Global Errors (rev 01) [8086:6f2c] 00:05.4 PIC: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D I/O APIC (rev 01) IOMMU group 18: [8086:8d7c] 00:11.0 Unassigned class [ff00]: Intel Corporation C610/X99 series chipset SPSR (rev 05) [8086:8d62] 00:11.4 SATA controller: Intel Corporation C610/X99 series chipset sSATA Controller [AHCI mode] (rev 05) IOMMU group 19: [8086:8d31] 00:14.0 USB controller: Intel Corporation C610/X99 series chipset USB xHCI Host Controller (rev 05) IOMMU group 20: [8086:8d3a] 00:16.0 Communication controller: Intel Corporation C610/X99 series chipset MEI Controller #1 (rev 05) IOMMU group 21: [8086:15a1] 00:19.0 Ethernet controller: Intel Corporation Ethernet Connection (2) I218-V (rev 05) IOMMU group 22: [8086:8d2d] 00:1a.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #2 (rev 05) IOMMU group 23: [8086:8d20] 00:1b.0 Audio device: Intel Corporation C610/X99 series chipset HD Audio Controller (rev 05) IOMMU group 24: [8086:8d10] 00:1c.0 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #1 (rev d5) IOMMU group 25: [8086:8d14] 00:1c.2 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #3 (rev d5) IOMMU group 26: [8086:8d16] 00:1c.3 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #4 (rev d5) IOMMU group 27: [8086:8d18] 00:1c.4 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #5 (rev d5) IOMMU group 28: [8086:8d26] 00:1d.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #1 (rev 05) IOMMU group 29: [8086:8d47] 00:1f.0 ISA bridge: Intel Corporation C610/X99 series chipset LPC Controller (rev 05) [8086:8d02] 00:1f.2 SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05) [8086:8d22] 00:1f.3 SMBus: Intel Corporation C610/X99 series chipset SMBus Controller (rev 05) IOMMU group 30: [1002:699f] 03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Lexa PRO [Radeon RX 550] (rev ff) [1002:aae0] 03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device aae0 (rev ff) IOMMU group 31: [10de:1287] 04:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 730] (rev a1) [10de:0e0f] 04:00.1 Audio device: NVIDIA Corporation GK208 HDMI/DP Audio Controller (rev a1) IOMMU group 32: [8086:1539] 06:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 33: [8086:08b3] 07:00.0 Network controller: Intel Corporation Wireless 3160 (rev 83) IOMMU group 34: [1b21:1242] 08:00.0 USB controller: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller Edited April 21, 2018 by NoobSpy Quote Link to comment
Chrysen Posted April 21, 2018 Share Posted April 21, 2018 (edited) Same here with Rx 480 have try arch linux. With windows 10 all fine with gpu pass... First start works after vm reboot get the same error. unraid 6.5 installed Edited April 21, 2018 by Chrysen Quote Link to comment
NoobSpy Posted May 1, 2018 Author Share Posted May 1, 2018 running on 6.5.1: Can someone please help me and point me in the direction I need to look at? I cant figure out what is causing this behavior after a VM shutdown. Why does the VM only work once after a sever reboot: May 1 12:39:44 Tower kernel: DMAR: [DMA Read] Request device [03:00.0] fault addr f400153000 [fault reason 06] PTE Read access is not setMay 1 12:39:44 Tower kernel: DMAR: [DMA Read] Request device [03:00.0] fault addr f400154000 [fault reason 06] PTE Read access is not setMay 1 12:39:44 Tower kernel: DMAR: [DMA Read] Request device [03:00.0] fault addr f400154000 [fault reason 06] PTE Read access is not setMay 1 12:39:44 Tower kernel: DMAR: [DMA Read] Request device [03:00.0] fault addr f400154000 [fault reason 06] PTE Read access is not setMay 1 12:39:44 Tower kernel: DMAR: DRHD: handling fault status reg 300May 1 12:39:44 Tower kernel: usb 3-3: reset full-speed USB device number 2 using xhci_hcdMay 1 12:39:45 Tower kernel: usb 3-3: reset full-speed USB device number 2 using xhci_hcdMay 1 12:39:45 Tower kernel: usb 3-3: reset full-speed USB device number 2 using xhci_hcdMay 1 12:39:45 Tower kernel: usb 3-3: reset full-speed USB device number 2 using xhci_hcdMay 1 12:39:45 Tower kernel: usb 3-11: reset full-speed USB device number 6 using xhci_hcdMay 1 12:39:46 Tower kernel: usb 3-11: reset full-speed USB device number 6 using xhci_hcdMay 1 12:39:46 Tower kernel: usb 3-11: reset full-speed USB device number 6 using xhci_hcdMay 1 12:39:46 Tower kernel: usb 3-11: reset full-speed USB device number 6 using xhci_hcdMay 1 12:39:48 Tower dnsmasq-dhcp[22930]: DHCPDISCOVER(virbr0) 192.168.122.15 52:54:00:7a:46:4c May 1 12:39:48 Tower dnsmasq-dhcp[22930]: DHCPOFFER(virbr0) 192.168.122.15 52:54:00:7a:46:4c May 1 12:39:48 Tower dnsmasq-dhcp[22930]: DHCPREQUEST(virbr0) 192.168.122.15 52:54:00:7a:46:4c May 1 12:39:48 Tower dnsmasq-dhcp[22930]: DHCPACK(virbr0) 192.168.122.15 52:54:00:7a:46:4c SomfyMay 1 12:40:12 Tower avahi-daemon[10408]: Interface vnet0.IPv6 no longer relevant for mDNS.May 1 12:40:12 Tower avahi-daemon[10408]: Leaving mDNS multicast group on interface vnet0.IPv6 with address fe80::fc54:ff:fe7a:464c.May 1 12:40:12 Tower kernel: virbr0: port 2(vnet0) entered disabled stateMay 1 12:40:12 Tower kernel: device vnet0 left promiscuous modeMay 1 12:40:12 Tower kernel: virbr0: port 2(vnet0) entered disabled stateMay 1 12:40:12 Tower avahi-daemon[10408]: Withdrawing address record for fe80::fc54:ff:fe7a:464c on vnet0.May 1 12:40:12 Tower kernel: logitech-djreceiver 0003:046D:C52B.0007: hiddev96,hidraw0: USB HID v1.11 Device [Logitech USB Receiver] on usb-0000:00:14.0-3/input2May 1 12:40:12 Tower kernel: input: Logitech K830 as /devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3:1.2/0003:046D:C52B.0007/0003:046D:404C.0008/input/input4May 1 12:40:12 Tower kernel: logitech-hidpp-device 0003:046D:404C.0008: input,hidraw1: USB HID v1.11 Keyboard [Logitech K830] on usb-0000:00:14.0-3:1May 1 12:40:12 Tower kernel: pcieport 0000:00:02.0: AER: Uncorrected (Fatal) error received: id=0010May 1 12:40:12 Tower kernel: pcieport 0000:00:02.0: PCIe Bus Error: severity=Uncorrected (Fatal), type=Transaction Layer, id=0010(Receiver ID)May 1 12:40:12 Tower kernel: pcieport 0000:00:02.0: device [8086:6f04] error status/mask=00000020/00000000May 1 12:40:12 Tower kernel: pcieport 0000:00:02.0: [ 5] Surprise Down Error (First)May 1 12:40:12 Tower kernel: pcieport 0000:00:02.0: broadcast error_detected messageMay 1 12:40:14 Tower kernel: pcieport 0000:00:02.0: Root Port link has been resetMay 1 12:40:14 Tower kernel: pcieport 0000:00:02.0: broadcast mmio_enabled messageMay 1 12:40:14 Tower kernel: pcieport 0000:00:02.0: broadcast resume messageMay 1 12:40:14 Tower kernel: pcieport 0000:00:02.0: AER: Device recovery successfulMay 1 12:40:14 Tower kernel: vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=noneMay 1 12:40:14 Tower kernel: vfio-pci 0000:03:00.0: Refused to change power state, currently in D3May 1 12:40:14 Tower kernel: vfio-pci 0000:03:00.1: Refused to change power state, currently in D3 Quote Link to comment
arg Posted September 22, 2019 Share Posted September 22, 2019 i have the same problem with a 1050 ti how to fix it Quote Link to comment
yellowcooln Posted January 5, 2020 Share Posted January 5, 2020 On 9/22/2019 at 2:24 AM, arg said: i have the same problem with a 1050 ti how to fix it I have the exact same issue with my 1050 ti. Would really like it to work without having to reset the whole server. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.