ASR Posted October 29, 2019 Share Posted October 29, 2019 (edited) Hi, to start I'm french, so sorry for my rusty english ! I did an unraid server for my work (environnemental restore) with : Asus X570 WS PRO ACE (for ECC and 3 PCIE 16X...) 2600 AMD (waiting 3900X stock) 32 GB ECC Ram Crucial 3 X AMD RX550 for VM 3 HDD RED 4To (1 parity / 2 DATA) 1 SSD 500 Go Cache 1 SSD NVME 500 to Passthrought specific VM My plan is to use this server with - 1 VM for a specific worker (W10 pro, 8 threads, 8G ram, 1 RX550 passtrought, 1 NVME idem...) ==> this is working without issue now in Q35-4.1 only, HyperV yes, OVMF, nvme is passthrough... I have error 127 but it's not a problem ; - NAS UNRAID to store some sharing data with my team and I'll do partition save/data save of their computer in autonomatic mode later ==> is working easily ; - At least 1 VM (actually passthrought but later in headless) on linux to working in SQL data base (fish data bases...) => and this is my issue, I can use Debian or another in VNC mode, but I can't passthrough another card. Unraid is starting on the first slot, I can't chose in the bios :(. The bottom card is use for the first Win10 VM, but It's impossible to use the middle card ! If I start this VM, it'll go in suspend mode (yellow icon) and I got this log on the VM : 2019-10-29T07:42:44.673194Z qemu-system-x86_64: vfio_err_notifier_handler(0000:0b:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest 2019-10-29T07:42:45.366243Z qemu-system-x86_64: vfio_err_notifier_handler(0000:0b:00.0) Unrecoverable error detected. Please collect any data possible and then kill the guest This log in general : Oct 29 08:42:44 Tower kernel: pcieport 0000:00:03.2: AER: Uncorrected (Non-Fatal) error received: 0000:00:00.0 Oct 29 08:42:44 Tower kernel: pcieport 0000:00:03.2: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Receiver ID) Oct 29 08:42:44 Tower kernel: pcieport 0000:00:03.2: AER: device [1022:1453] error status/mask=00200000/04400000 Oct 29 08:42:44 Tower kernel: pcieport 0000:00:03.2: AER: [21] ACSViol (First) Oct 29 08:42:44 Tower kernel: pcieport 0000:00:03.2: AER: Device recovery failed Oct 29 08:42:44 Tower kernel: AMD-Vi: Completion-Wait loop timed out Oct 29 08:42:44 Tower kernel: AMD-Vi: Completion-Wait loop timed out Oct 29 08:42:45 Tower kernel: AMD-Vi: Completion-Wait loop timed out Oct 29 08:42:45 Tower kernel: iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=0b:00.0 address=0x83b073e80] Oct 29 08:42:45 Tower kernel: pcieport 0000:00:03.2: AER: Uncorrected (Non-Fatal) error received: 0000:00:00.0 Oct 29 08:42:45 Tower kernel: pcieport 0000:00:03.2: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Receiver ID) Oct 29 08:42:45 Tower kernel: pcieport 0000:00:03.2: AER: device [1022:1453] error status/mask=00200000/04400000 Oct 29 08:42:45 Tower kernel: pcieport 0000:00:03.2: AER: [21] ACSViol (First) Oct 29 08:42:45 Tower kernel: pcieport 0000:00:03.2: AER: Device recovery failed My system information : Model: N/A M/B: ASUSTeK COMPUTER INC. Pro WS X570-ACE Version Rev X.0x - s/n: 190552238900195 LAST BIOS BIOS: American Megatrends Inc. Version 1001. Dated: 09/09/2019 CPU: AMD Ryzen 5 2600 Six-Core @ 3400 MHz HVM: Enabled IOMMU: Enabled Cache: 576 KiB, 3072 KiB, 16384 KiB Memory: 32 GiB DDR4 Multi-bit ECC (max. installable capacity 128 GiB) Network: eth0: 1000 Mbps, full duplex, mtu 1500 Kernel: Linux 5.3.7-Unraid x86_64 OpenSSL: 1.1.1d I have VFIO allow unsafe interrupts : YES, no need PCIe ACS override because my IOMMU are this and it's ok for me : PCI Devices and IOMMU Groups IOMMU group 0: [1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 1: [1022:1453] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 2: [1022:1453] 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 3: [1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 4: [1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 5: [1022:1453] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 6: [1022:1453] 00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 7: [1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 8: [1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 9: [1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 10: [1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 11: [1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 12: [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 13: [1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 IOMMU group 14: [8086:f1a8] 01:00.0 Non-Volatile memory controller: Intel Corporation SSD 660P Series (rev 03) IOMMU group 15: [1022:57ad] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57ad IOMMU group 16: [1022:57a3] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a3 IOMMU group 17: [1022:57a3] 03:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a3 IOMMU group 18: [1022:57a3] 03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a3 IOMMU group 19: [1022:57a4] 03:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 [1022:1485] 07:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:149c] 07:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c] 07:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller IOMMU group 20: [1022:57a4] 03:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 [1022:7901] 08:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 21: [1022:57a4] 03:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 57a4 [1022:7901] 09:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 22: [1002:67ff] 04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (rev ff) [1002:aae0] 04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] IOMMU group 23: [8086:1539] 05:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 24: [10ec:816e] 06:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. Device 816e (rev 1a) [10ec:8168] 06:00.1 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 1a) [10ec:816a] 06:00.2 Serial controller: Realtek Semiconductor Co., Ltd. Device 816a (rev 1a) [10ec:816c] 06:00.7 IPMI Interface: Realtek Semiconductor Co., Ltd. Device 816c (rev 1a) IOMMU group 25: [1002:67ff] 0a:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (rev ff) [1002:aae0] 0a:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] IOMMU group 26: [1002:67ff] 0b:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (rev ff) [1002:aae0] 0b:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] (rev ff) IOMMU group 27: [1022:145a] 0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function IOMMU group 28: [1022:1456] 0c:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor IOMMU group 29: [1022:145f] 0c:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Zeppelin USB 3.0 Host controller IOMMU group 30: [1022:1455] 0d:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function IOMMU group 31: [1022:7901] 0d:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 32: [1022:1457] 0d:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller Thank you if you can help me ! really. 10 days remaning before I had to buy a licence. Have a nice day ! Edited October 29, 2019 by ASR Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.