zwarnimont Posted May 11, 2021 Share Posted May 11, 2021 (edited) [SOLVED] Passing through the gpu in slot 2 works so that's what I'm doing for now I feel like I have tried every combination of fixes under the sun to get this to work. I am trying to boot a Windows 10 VM from an NVME I pulled from my previous machine. It boots flawlessly when using VNC, but I get a black screen when passing through my 3080. After I switch back to VNC, after using the GPU, I have to choose my boot device in UEFI, which is easy on VNC and it boots up again like normal. Then when I go back to GPU again, I have to choose my boot device but without seeing the screen. However, I simply wrote down the keystrokes I have to do to boot my NVME, and it worked PERFECTLY... one time. I have also tried using VNC and my GPU, but the VM freezes at Windows login. I would be fine doing the blind boot drive selection but it hasn't worked for me since. I also have Chrome Remote Desktop set up, so if it makes it far enough I should be able to use that to update drivers. Since then, I have tried several solutions to prevent it from going to UEFI in the first place (with the <boot order='1'/> added at my NVME) and enabling multifunction on my GPU (though it worked without this before). I have also dumped my own vbios (though it worked before with vbios from techpowerup) and my NVME and GPU (both parts) are bound to VFIO at boot. I have watched all of spaceinvader's videos on the topic, so I have nowhere else to turn. This xml includes all the changes that should make it work, but it did boot for me once without multifunction, <boot order='1'/>, and with vbios from techpowerup. <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 10</name> <uuid>15963ece-b5b8-8d06-61a3-47714909116d</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>25165824</memory> <currentMemory unit='KiB'>2097152</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>12</vcpu> <cputune> <vcpupin vcpu='0' cpuset='2'/> <vcpupin vcpu='1' cpuset='10'/> <vcpupin vcpu='2' cpuset='3'/> <vcpupin vcpu='3' cpuset='11'/> <vcpupin vcpu='4' cpuset='4'/> <vcpupin vcpu='5' cpuset='12'/> <vcpupin vcpu='6' cpuset='5'/> <vcpupin vcpu='7' cpuset='13'/> <vcpupin vcpu='8' cpuset='6'/> <vcpupin vcpu='9' cpuset='14'/> <vcpupin vcpu='10' cpuset='7'/> <vcpupin vcpu='11' cpuset='15'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-5.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/15963ece-b5b8-8d06-61a3-47714909116d_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='6' threads='2'/> <cache mode='passthrough'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.190-1.iso'/> <target dev='hdb' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:83:0c:0b'/> <source bridge='br0'/> <model type='virtio-net'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='6'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/isos/vbios/thisoneevga3080ftw.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </source> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1532'/> <product id='0x0067'/> </source> <address type='usb' bus='0' port='1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x4b42'/> <product id='0x6067'/> </source> <address type='usb' bus='0' port='2'/> </hostdev> <memballoon model='none'/> </devices> </domain> IOMMU group 0: [8086:3e30] 00:00.0 Host bridge: Intel Corporation 8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S] (rev 0d) IOMMU group 1: [8086:1901] 00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 0d) IOMMU group 2: [8086:1905] 00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) (rev 0d) IOMMU group 3: [8086:a379] 00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10) IOMMU group 4: [8086:a36d] 00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10) Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002: ID 2109:2813 VIA Labs, Inc. USB2.0 Hub Bus 001 Device 003: ID 05e3:0608 Genesys Logic, Inc. Hub Bus 001 Device 004: ID 1a40:0101 Terminus Technology Inc. Hub Bus 001 Device 005: ID 048d:8297 Integrated Technology Express, Inc. ITE Device(8595) Bus 001 Device 006: ID 1a40:0101 Terminus Technology Inc. Hub Bus 001 Device 007: ID 0764:0601 Cyber Power System, Inc. PR1500LCDRT2U UPS Bus 001 Device 008: ID 05e3:0608 Genesys Logic, Inc. Hub Bus 001 Device 009: ID 0fd9:0060 Elgato Systems GmbH Stream Deck Bus 001 Device 010: ID 4b42:6067 KBDFans KBD67v2 Bus 001 Device 011: ID 8087:0aaa Intel Corp. Bus 001 Device 012: ID 13fe:4100 Kingston Technology Company Inc. Flash drive Bus 001 Device 013: ID 046d:0892 Logitech, Inc. OrbiCam Bus 001 Device 014: ID 1532:0067 Razer USA, Ltd Razer Naga Trinity Bus 001 Device 015: ID 20b1:3066 XMOS Ltd D30 Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 002 Device 002: ID 2109:0813 VIA Labs, Inc. USB3.0 Hub [8086:a36f] 00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10) IOMMU group 5: [8086:a370] 00:14.3 Network controller: Intel Corporation Wireless-AC 9560 [Jefferson Peak] (rev 10) IOMMU group 6: [8086:a360] 00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10) IOMMU group 7: [8086:a352] 00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10) [1:0:0:0] disk ATA WDC WD120EMFZ-11 0A81 /dev/sdb 12.0TB [2:0:0:0] disk ATA WDC WD120EMFZ-11 0A81 /dev/sdc 12.0TB [3:0:0:0] disk ATA WDC WD120EDAZ-11 0A81 /dev/sdd 12.0TB [4:0:0:0] disk ATA Samsung SSD 860 4B6Q /dev/sde 1.00TB IOMMU group 8: [8086:a340] 00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0) IOMMU group 9: [8086:a338] 00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0) IOMMU group 10: [8086:a33d] 00:1c.5 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #6 (rev f0) IOMMU group 11: [8086:a330] 00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0) IOMMU group 12: [8086:a305] 00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev 10) [8086:a348] 00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10) [8086:a323] 00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10) [8086:a324] 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller (rev 10) [8086:15bc] 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-V (rev 10) IOMMU group 13: [10de:2206] 01:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3080] (rev a1) IOMMU group 14: [10de:1aef] 01:00.1 Audio device: NVIDIA Corporation GA102 High Definition Audio Controller (rev a1) IOMMU group 15: [10de:2206] 02:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3080] (rev a1) IOMMU group 16: [10de:1aef] 02:00.1 Audio device: NVIDIA Corporation GA102 High Definition Audio Controller (rev a1) IOMMU group 17: [15b7:5002] 03:00.0 Non-Volatile memory controller: Sandisk Corp WD Black 2018/SN750 / PC SN720 NVMe SSD [N:0:8215:1] disk WDS200T3X0C-00SJG0__1 /dev/nvme0n1 2.00TB IOMMU group 18: [1b4b:9215] 05:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9215 PCIe 2.0 x1 4-port SATA 6 Gb/s Controller (rev 11) [5:0:0:0] disk ATA WDC WD100EDAZ-11 0A81 /dev/sdf 10.0TB [6:0:0:0] disk ATA WDC WD100EMAZ-00 0A83 /dev/sdg 10.0TB [7:0:0:0] disk ATA WDC WD80EMAZ-00W 0A83 /dev/sdh 8.00TB [8:0:0:0] disk ATA WDC WD80EMAZ-00W 0A83 /dev/sdi 8.00TB [8:1:0:0] disk ATA WDC WD80EMAZ-00W 0A83 /dev/sdj 8.00TB [8:2:0:0] disk ATA WDC WD100EMAZ-00 0A83 /dev/sdk 10.0TB IOMMU group 19: [15b7:5002] 06:00.0 Non-Volatile memory controller: Sandisk Corp WD Black 2018/SN750 / PC SN720 NVMe SSD This controller is bound to vfio, connected drives are not visible. PS: It seems like the sticking point here is my GPU, but I do have a SATA SSD that is a mirror of the NVME I'm trying to use, so if there's a setup using that that works, please let me know. (but my nvme has more space so I'd rather use that) PPS: does the new driver enabling GPU passthrough from nvidia make this easier? I tried to install it when I booted through VNC but it failed (presumably because there was no GPU) Edited May 12, 2021 by zwarnimont Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.