AinzOolGown Posted November 16, 2019 Share Posted November 16, 2019 (edited) Hi there ! I'm new to unRAID but i managed to build a near perfect system, very pleasant software with powerfull options ^^ But, there is a but, i'm only missing a VM to replace my old tower by the new unRAIDed one, so close ! My problem is with grouping, i think, and especially passthrough of my graphic card to-the-said VM. I have read multiple post on the subject, tried to enable ACS (was the ACS option on the Motherboard BIOS, right ?) but that does'nt work :/. There also is this ROM extract thing mentionned by SpaceInvader One, but i have to boot the VM at least once to do that (and i cannot). I'll post the most important information bellow but if you need some other, don't hesitate Config details: - Corsair Obsidian 1000D - AsRock X399 Taichi - AMD Threadripper 1920 - NVidia (eVGA) RTX 2080 Black Edition Gaming 8Gb (on the thirds pcie port, she's the only graphic card in the tower) - G.Skill 4x16Gb TridentZ RGB - unRAID Version: 6.8.0-rc5 The error message i get is : internal error: qemu unexpectedly closed the monitor: 2019-11-16T12:13:37.352937Z qemu-system-x86_64: -device vfio-pci,host=0000:08:00.0,id=hostdev0,bus=pci.0,addr=0x5: vfio 0000:08:00.0: group 14 is not viable Please ensure all devices within the iommu_group are bound to their vfio bus driver. Below is my VM config : <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10</name> <uuid>03fb8fb3-3ef3-90ce-3860-b74c04d11f3a</uuid> <description>VM-Win10Pro [Gaming]</description> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>16777216</memory> <currentMemory unit='KiB'>16777216</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='12'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='14'/> <vcpupin vcpu='4' cpuset='4'/> <vcpupin vcpu='5' cpuset='16'/> <vcpupin vcpu='6' cpuset='6'/> <vcpupin vcpu='7' cpuset='18'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-4.0'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/03fb8fb3-3ef3-90ce-3860-b74c04d11f3a_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='8' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source dev='/dev/disk/by-id/nvme-Samsung_SSD_970_EVO_1TB_S467NX0KA03529E'/> <target dev='hdc' bus='scsi'/> <boot order='1'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/MS Windows10x64 - 1903.iso'/> <target dev='hda' bus='ide'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='nec-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <controller type='scsi' index='0' model='virtio-scsi'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='pci' index='0' model='pci-root'/> <interface type='bridge'> <mac address='52:54:00:bb:d3:ca'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1b1c'/> <product id='0x1d00'/> </source> <address type='usb' bus='0' port='2'/> </hostdev> <memballoon model='none'/> </devices> </domain> Capture : https://imgur.com/a/uboYoem And here is my IOMMU Grouping ( RTX Group is the 14) : PCI Devices and IOMMU Groups IOMMU group 0: [1022:1452] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 1: [1022:1453] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 2: [1022:1452] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 3: [1022:1452] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 4: [1022:1453] 00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 5: [1022:1452] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 6: [1022:1452] 00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 7: [1022:1454] 00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 8: [1022:1452] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 9: [1022:1454] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 10: [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 59) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 11: [1022:1460] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1461] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1462] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1463] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1464] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1465] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1466] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1467] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 IOMMU group 12: [1022:1460] 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1461] 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1462] 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1463] 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1464] 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1465] 00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1466] 00:19.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1467] 00:19.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 IOMMU group 13: [1022:43ba] 01:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset USB 3.1 xHCI Controller (rev 02) [1022:43b6] 01:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset SATA Controller (rev 02) [1022:43b1] 01:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] X399 Series Chipset PCIe Bridge (rev 02) [1022:43b4] 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [1022:43b4] 02:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02) [8086:1539] 04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) [8086:24fb] 05:00.0 Network controller: Intel Corporation Dual Band Wireless-AC 3168NGW [Stone Peak] (rev 10) [8086:1539] 06:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03) IOMMU group 14: [10de:1e82] 08:00.0 VGA compatible controller: NVIDIA Corporation TU104 [GeForce RTX 2080] (rev a1) [10de:10f8] 08:00.1 Audio device: NVIDIA Corporation Device 10f8 (rev a1) [10de:1ad8] 08:00.2 USB controller: NVIDIA Corporation Device 1ad8 (rev a1) [10de:1ad9] 08:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1ad9 (rev a1) IOMMU group 15: [1022:145a] 09:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function IOMMU group 16: [1022:1456] 09:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor IOMMU group 17: [1022:145c] 09:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller IOMMU group 18: [1022:1455] 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function IOMMU group 19: [1022:7901] 0a:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) IOMMU group 20: [1022:1457] 0a:00.3 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller IOMMU group 21: [1022:1452] 40:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 22: [1022:1453] 40:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge IOMMU group 23: [1022:1452] 40:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 24: [1022:1452] 40:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 25: [1022:1452] 40:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 26: [1022:1452] 40:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 27: [1022:1454] 40:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 28: [1022:1452] 40:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge IOMMU group 29: [1022:1454] 40:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B IOMMU group 30: [144d:a808] 41:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 IOMMU group 31: [1022:145a] 42:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function IOMMU group 32: [1022:1456] 42:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor IOMMU group 33: [1022:145c] 42:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller IOMMU group 34: [1022:1455] 43:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function IOMMU group 35: [1022:7901] 43:00.2 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) CPU Thread Pairings Pair 1: cpu 0 / cpu 12 Pair 2: cpu 1 / cpu 13 Pair 3: cpu 2 / cpu 14 Pair 4: cpu 3 / cpu 15 Pair 5: cpu 4 / cpu 16 Pair 6: cpu 5 / cpu 17 Pair 7: cpu 6 / cpu 18 Pair 8: cpu 7 / cpu 19 Pair 9: cpu 8 / cpu 20 Pair 10: cpu 9 / cpu 21 Pair 11: cpu 10 / cpu 22 Pair 12: cpu 11 / cpu 23 USB Devices Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002: ID 8087:0aa7 Intel Corp. Bus 001 Device 003: ID 1b1c:1d00 Corsair 1000D Bus 001 Device 004: ID 05e3:0610 Genesys Logic, Inc. 4-port hub Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 005 Device 002: ID 413c:2003 Dell Computer Corp. Keyboard Bus 005 Device 003: ID 046d:c52b Logitech, Inc. Unifying Receiver Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 006 Device 002: ID 090c:1000 Silicon Motion, Inc. - Taiwan (formerly Feiya Technology Corp.) Flash Drive Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub SCSI Devices [0:0:0:0] disk Samsung Flash Drive 1100 /dev/sda 32.0GB [2:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdb 500GB [3:0:0:0] disk ATA Samsung SSD 860 3B6Q /dev/sdc 500GB [5:0:0:0] disk ATA WDC WD100EMAZ-00 0A83 /dev/sdd 10.0TB [6:0:0:0] disk ATA WDC WD100EMAZ-00 0A83 /dev/sde 10.0TB [7:0:0:0] disk ATA WDC WD100EZAZ-11 0A83 /dev/sdf 10.0TB [8:0:0:0] disk ATA WDC WD100EZAZ-11 0A83 /dev/sdg 10.0TB [N:0:4:1] disk Samsung SSD 970 EVO 1TB__1 /dev/nvme0n1 1.00TB Thank you ! Edited November 17, 2019 by AinzOolGown GC details Quote Link to comment
bastl Posted November 16, 2019 Share Posted November 16, 2019 @AinzOolGown Try it with the following XML based on the one you posted. <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10</name> <uuid>03fb8fb3-3ef3-90ce-3860-b74c04d11f3a</uuid> <description>VM-Win10Pro [Gaming]</description> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>16777216</memory> <currentMemory unit='KiB'>16777216</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='12'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='14'/> <vcpupin vcpu='4' cpuset='4'/> <vcpupin vcpu='5' cpuset='16'/> <vcpupin vcpu='6' cpuset='6'/> <vcpupin vcpu='7' cpuset='18'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-4.0'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/03fb8fb3-3ef3-90ce-3860-b74c04d11f3a_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='8' threads='1'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source dev='/dev/disk/by-id/nvme-Samsung_SSD_970_EVO_1TB_S467NX0KA03529E'/> <target dev='hdc' bus='scsi'/> <boot order='1'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/MS Windows10x64 - 1903.iso'/> <target dev='hda' bus='ide'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='nec-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <controller type='scsi' index='0' model='virtio-scsi'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='pci' index='0' model='pci-root'/> <interface type='bridge'> <mac address='52:54:00:bb:d3:ca'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x3'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1b1c'/> <product id='0x1d00'/> </source> <address type='usb' bus='0' port='2'/> </hostdev> <memballoon model='none'/> </devices> </domain> I only added the 2 extra devices from group 14 2 hours ago, AinzOolGown said: [10de:1ad8] 08:00.2 USB controller: NVIDIA Corporation Device 1ad8 (rev a1) [10de:1ad9] 08:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1ad9 (rev a1) <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x08' slot='0x00' function='0x3'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> If this doesn't work check if you have a second GPU laying around, put it in the first slot for Unraid to pick up at boot. Small hint, don't use core0 and its HT for any VM. Unraid will always use the first core itself for stuff in the backround. Quote Link to comment
AinzOolGown Posted November 16, 2019 Author Share Posted November 16, 2019 (edited) Hi bastl and thank you very much for your reply Done, now the VM is labelled as "started" but there is not reaction of my screen pluged on the NVidia. Have disabled the Core 0/HT 12 at the same time, thanks for your tips There is definitely some progress here ! ^^ I'm searching if i have a second GPU (do i have to let it in the tower ? (i have built a watercooling system, and this second GPU where not planned in ^^). Edited November 16, 2019 by AinzOolGown Quote Link to comment
bastl Posted November 16, 2019 Share Posted November 16, 2019 @AinzOolGown What is the VM log showing when starting up the VM? How is the CPU utilisation in the Unraid Dashboard? Quote Link to comment
AinzOolGown Posted November 16, 2019 Author Share Posted November 16, 2019 Log show this : -overcommit mem-lock=off \ -smp 8,sockets=1,cores=8,threads=1 \ -uuid 03fb8fb3-3ef3-90ce-3860-b74c04d11f3a \ -display none \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=35,server,nowait \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=localtime \ -no-hpet \ -no-shutdown \ -boot strict=on \ -device nec-usb-xhci,p2=15,p3=15,id=usb,bus=pci.0,addr=0x7 \ -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x4 \ -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \ -drive file=/dev/disk/by-id/nvme-Samsung_SSD_970_EVO_1TB_S467NX0KA03529E,format=raw,if=none,id=drive-scsi0-0-0-2,cache=writeback \ -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,device_id=drive-scsi0-0-0-2,drive=drive-scsi0-0-0-2,id=scsi0-0-0-2,bootindex=1,write-cache=on \ -drive 'file=/mnt/user/isos/MS Windows10x64 - 1903.iso,format=raw,if=none,id=drive-ide0-0-0,readonly=on' \ -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 \ -drive file=/mnt/user/isos/virtio-win-0.1.160-1.iso,format=raw,if=none,id=drive-ide0-0-1,readonly=on \ -device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 \ -netdev tap,fd=37,id=hostnet0,vhost=on,vhostfd=38 \ -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:bb:d3:ca,bus=pci.0,addr=0x2 \ -chardev pty,id=charserial0 \ -device isa-serial,chardev=charserial0,id=serial0 \ -chardev socket,id=charchannel0,fd=39,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \ -device usb-tablet,id=input0,bus=usb.0,port=1 \ -device vfio-pci,host=0000:08:00.0,id=hostdev0,bus=pci.0,addr=0x5 \ -device vfio-pci,host=0000:08:00.1,id=hostdev1,bus=pci.0,addr=0x6 \ -device vfio-pci,host=0000:08:00.2,id=hostdev2,bus=pci.0,addr=0x8 \ -device vfio-pci,host=0000:08:00.3,id=hostdev3,bus=pci.0,addr=0x9 \ -device usb-host,hostbus=5,hostaddr=3,id=hostdev4,bus=usb.0,port=2 \ -device usb-host,hostbus=1,hostaddr=3,id=hostdev5,bus=usb.0,port=3 \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2019-11-16 15:52:28.197+0000: Domain id=3 is tainted: high-privileges 2019-11-16 15:52:28.197+0000: Domain id=3 is tainted: host-cpu char device redirected to /dev/pts/0 (label charserial0) 2019-11-16T15:52:30.876612Z qemu-system-x86_64: -device vfio-pci,host=0000:08:00.0,id=hostdev0,bus=pci.0,addr=0x5: Failed to mmap 0000:08:00.0 BAR 3. Performance may be slow And wow, CPU is at 100% on every core/HT Quote Link to comment
bastl Posted November 16, 2019 Share Posted November 16, 2019 22 minutes ago, AinzOolGown said: Failed to mmap 0000:08:00.0 BAR 3. Performance may be slow You propably need a vbios to make it work with a single GPU only. Search for your exact model and try it with one of the available BIOS revisions https://www.techpowerup.com/vgabios/ If none of them are working, try to manually edit the BIOS file with a hex editor like SpaceInvaderOne described here Other option is to plug in a second card in the first slot and make sure Unraid picks it up for itself. Quote Link to comment
AinzOolGown Posted November 16, 2019 Author Share Posted November 16, 2019 (edited) Ok, thanks I found a Gigabyte Radeon HD6950, will try that in about 30minutes and tell the result Edited November 16, 2019 by AinzOolGown Quote Link to comment
AinzOolGown Posted November 16, 2019 Author Share Posted November 16, 2019 Not working I can't get any image on any graphic card with the 2 installed. Checked techpowerup's site, my RTX isn't in there, will investigate your BIOS editing option, thanks Quote Link to comment
bastl Posted November 16, 2019 Share Posted November 16, 2019 Also check your BIOS options how the GPUs are setup. You'll often find some settings to configure which card should be the first one to boot with. Another thing to mention, have a monitor plugged in and turned on when booting up. In some scenarios with a passthrough GPU the guest OS won't boot up if no monitor is connected. 2 hours ago, AinzOolGown said: I can't get any image on any graphic card with the 2 installed. You can't see any output when booting the server? Quote Link to comment
bastl Posted November 16, 2019 Share Posted November 16, 2019 @AinzOolGown Forgot something, which BIOS Agesa version is your board using? The first BIOS revisions almost all had some issues. On my ASRock Fatal1ty X399 Professional Gaming when I bought end of 2017 had the version 1.20 Agesa 1.0.0.2 and I think the first revisions which worked flawless for me where with Agesa version 1.0.0.4. So to speak if you're on the newer 3.30 or newer you should be fine. I'am on 3.50 and have 2 GPUs installed and both are able to passthrough. Shouldn't be a big difference in our 2 boards. Quote Link to comment
AinzOolGown Posted November 17, 2019 Author Share Posted November 17, 2019 12 hours ago, bastl said: Also check your BIOS options how the GPUs are setup. You'll often find some settings to configure which card should be the first one to boot with. I did not know, will search that option, thanks ^^ 12 hours ago, bastl said: You can't see any output when booting the server? Right, no output at all, but i have connected only 1 monitor that i'm switching between the 2 graphics card, maybe i need one per card, will try that too ^^ 12 hours ago, bastl said: @AinzOolGown Forgot something, which BIOS Agesa version is your board using? The first BIOS revisions almost all had some issues. On my ASRock Fatal1ty X399 Professional Gaming when I bought end of 2017 had the version 1.20 Agesa 1.0.0.2 and I think the first revisions which worked flawless for me where with Agesa version 1.0.0.4. So to speak if you're on the newer 3.30 or newer you should be fine. I'am on 3.50 and have 2 GPUs installed and both are able to passthrough. Shouldn't be a big difference in our 2 boards. BIOS is the latest 3.80 with AMD AGESA ThreadRipperPI-SP3r2 1.1.0.2 Quote Link to comment
bastl Posted November 17, 2019 Share Posted November 17, 2019 1 hour ago, AinzOolGown said: but i have connected only 1 monitor that i'm switching between the 2 graphics card, maybe i need one per card, will try that too My scenario is, my monitor has 2 Inputs for 2 different devices. So in my setup I've connected one VM via displayport and the other VM via HDMI to the monitor. I have to switch manually between the VMs, but on both the monitor is recognized even if the correct input isn't selected. The first card in the first slot is picked up by Unraid when I boot up the server. As soon as the autostarted VM boots up, the monitor is handed over to the VM. In my case this doesn't work without a vbios for this card or no monitor connected to that specific card. I could use a HDMI dongle for a couple bucks to emulate a monitor but I don't really need to. Quote Link to comment
AinzOolGown Posted November 17, 2019 Author Share Posted November 17, 2019 (edited) It works ! Have connected 2 monitor, one on each graphic card, the primary one was picked by unraid and my nvidia by the VM when i started it, yahooo \o/ Now i think i will try to extract the vbios of the nvidia Thanks bastl, almost finished ! Edited November 17, 2019 by AinzOolGown 1 Quote Link to comment
AinzOolGown Posted November 17, 2019 Author Share Posted November 17, 2019 BIOS extracted and all is running up ! Very big thanks to you bastl !! Was having an odd problem at first, when the VM booted, i had 2 blinking cursors on the Windows desktop (artifacts of unRAID console), i then installed NVidia's Drivers for my card and after a reboot they disappeared :) Will test all this component and see if i can game & edit smoothly ^^. Again, thank you very much ^^ Quote Link to comment
bastl Posted November 17, 2019 Share Posted November 17, 2019 @AinzOolGown There are a couple custom tweaks you can apply to increase performance on a Threadripper system. But at first make sure, everything is stable and running smothly. The 1920x is basically 2 6core CPUs so to say. As long as you don't use cores from both dies for the same VM you should be fine. First half of your cores unraid shows is the first die, second half is the second die. If you use cores from both dies in a VM they have to communicate between each other which will increase the latency. In general you won't notice much but in low latency depending applications (some benchmarks) or games you can see the difference. Lower fps, sometimes stuttering. Best advice I can give you is to isolate the second half of the cores so nothing uses it unless you configure it like for VMs. Use these cores for only 1 VM for the best possible performance. 6 cores 12 threads is plenty enough for gaming and the rest for dockers and Unraid itself, maybe some small VMs on the first die also is more than enough. Maybe have a look in the following thread about CPU pinning and further to the end, last couple pages, you will find some tweaks for threadripper to increase the performance. But first make sure it's running stable without. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.