Thanks for the bios dump - but the same monitor which shows normally boot screen/ unraid login and stuff in TXT mode - turns OFF soon as i start the Win10 VM with linked DUMP file.
I assume my dump is right as i have 10x Zotac 1070 cards , so should be done right.
Cards have their own group even without Enable PCIe ACS Override:set to NO
IOMMU group 0: [1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge
IOMMU group 1: [1022:1453] 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
IOMMU group 2: [1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge
IOMMU group 3: [1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge
IOMMU group 4: [1022:1453] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
IOMMU group 5: [1022:1453] 00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge
IOMMU group 6: [1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge
IOMMU group 7: [1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge
IOMMU group 8: [1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
IOMMU group 9: [1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge
IOMMU group 10: [1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
IOMMU group 11: [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59)
[1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
IOMMU group 12: [1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0
[1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1
[1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2
[1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3
[1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4
[1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5
[1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric Device 18h Function 6
[1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7
IOMMU group 13: [1022:43b9] 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43b9 (rev 02)
[1022:43b5] 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] Device 43b5 (rev 02)
[1022:43b0] 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b0 (rev 02)
[1022:43b4] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1022:43b4] 02:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1022:43b4] 02:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1022:43b4] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1022:43b4] 02:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1022:43b4] 02:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
[1b21:1343] 06:00.0 USB controller: ASMedia Technology Inc. Device 1343
[8086:1539] 07:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)
IOMMU group 14: [10de:1b81] 09:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1)
[10de:10f0] 09:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1)
IOMMU group 15: [10de:1b81] 0a:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1)
[10de:10f0] 0a:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1)
IOMMU group 16: [1022:145a] 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 145a
IOMMU group 17: [1022:1456] 0b:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor
IOMMU group 18: [1022:145c] 0b:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller
IOMMU group 19: [1022:1455] 0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device 1455
IOMMU group 20: [1022:7901] 0c:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51)
IOMMU group 21: [1022:1457] 0c:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller
This setup with 2 cards (2card1070a.xml)
This is what im getting in the VM LOG:
2018-01-01T23:41:56.045938Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124c830, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.045958Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124c838, 0x0,1) failed: Device or resource busy
2018-01-01T23:41:56.045978Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124c839, 0x0,1) failed: Device or resource busy
2018-01-01T23:41:56.045997Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124c83a, 0x0,1) failed: Device or resource busy
2018-01-01T23:41:56.046017Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124c83b, 0xff,1) failed: Device or resource busy
2018-01-01T23:41:56.046037Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7c4, 0xff000000,4) failed: Device or resource busy
2018-01-01T23:41:56.046048Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7c8, 0xff000000,4) failed: Device or resource busy
2018-01-01T23:41:56.046062Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7cc, 0xff000000,4) failed: Device or resource busy
2018-01-01T23:41:56.046073Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7d0, 0xff000000,4) failed: Device or resource busy
2018-01-01T23:41:56.046093Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7d0, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046108Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7d8, 0xff2f2f2fff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046128Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7e0, 0xff9f9f9fffafafaf,8) failed: Device or resource busy
2018-01-01T23:41:56.046143Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7e8, 0xff000000ff0f0f0f,8) failed: Device or resource busy
2018-01-01T23:41:56.046163Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7f0, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046178Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d7f8, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046198Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d800, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046213Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d808, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046233Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d810, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046248Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d818, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046268Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d820, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046283Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d828, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046303Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d830, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046323Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d838, 0x0,1) failed: Device or resource busy
2018-01-01T23:41:56.046344Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d839, 0x0,1) failed: Device or resource busy
2018-01-01T23:41:56.046364Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d83a, 0x0,1) failed: Device or resource busy
2018-01-01T23:41:56.046383Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124d83b, 0xff,1) failed: Device or resource busy
2018-01-01T23:41:56.046404Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7c4, 0xff000000,4) failed: Device or resource busy
2018-01-01T23:41:56.046414Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7c8, 0xff000000,4) failed: Device or resource busy
2018-01-01T23:41:56.046429Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7cc, 0xff000000,4) failed: Device or resource busy
2018-01-01T23:41:56.046439Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7d0, 0xff000000,4) failed: Device or resource busy
2018-01-01T23:41:56.046459Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7d0, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046474Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7d8, 0xffdfdfdfff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046494Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7e0, 0xffffffffffffffff,8) failed: Device or resource busy
2018-01-01T23:41:56.046509Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7e8, 0xff000000ff9f9f9f,8) failed: Device or resource busy
2018-01-01T23:41:56.046529Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7f0, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046544Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e7f8, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046595Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e800, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046612Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e808, 0xff000000ff000000,8) failed: Device or resource busy
2018-01-01T23:41:56.046633Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x124e810, 0xff000000ff000000,8) failed: Device or resource busy
2card1070a.xml
Zotac1070mini.rom
Zotac1070mini-mod.dump