May 22, 20233 yr I am trying to dual boot my bare-metal win11 install as VM without luck. I've been following spaceinvaders tutorial, except binding the controller via the new checkboxes in System Devices. Whenever I try to start the vm, 95% of the time it just freezes on boot with the TianoCore splash still showing or the remaining 5%, I manage to get into windows, with the unraid host crashing shortly after (<5m). Running the VM without gpu passthrough (VNC) does work. I think I have tried every possible combination of stubbing devices (including the gpu), removing them after setup, using dumped vbios, etc. An interesting behavior is that after force killing the vm and restarting, the windows automatic repair launches just fine but choosing restart will lead to another freeze on boot. Right now, I have more devices stubbed and attached to the VM (usb controller, sata, ...) but the minimal setup with just the nvme controller and gpu exposes the same problems. Other things I have tried: - force all pcie slots to gen3 (including m.2) - disable r-bar - enable sr-iov - force enable iommu (instead of auto) - set pcie ACS override to both - update bios - another vm with the nvme in the second slot passed through (older win11 install) same behavior IOMMU groups Spoiler IOMMU group 0: [1022:14da] 00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14da IOMMU group 1: [1022:14db] 00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14db IOMMU group 2: [1022:14db] 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14db IOMMU group 3: [1022:14da] 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14da IOMMU group 4: [1022:14db] 00:02.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14db IOMMU group 5: [1022:14db] 00:02.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14db IOMMU group 6: [1022:14da] 00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14da IOMMU group 7: [1022:14da] 00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14da IOMMU group 8: [1022:14da] 00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14da IOMMU group 9: [1022:14dd] 00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14dd IOMMU group 10: [1022:14dd] 00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 14dd IOMMU group 11: [1022:790b] 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 71) [1022:790e] 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51) IOMMU group 12: [1022:14e0] 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14e0 [1022:14e1] 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14e1 [1022:14e2] 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14e2 [1022:14e3] 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14e3 [1022:14e4] 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14e4 [1022:14e5] 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14e5 [1022:14e6] 00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14e6 [1022:14e7] 00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Device 14e7 IOMMU group 13: [10de:2484] 01:00.0 VGA compatible controller: NVIDIA Corporation GA104 [GeForce RTX 3070] (rev a1) [10de:228b] 01:00.1 Audio device: NVIDIA Corporation GA104 High Definition Audio Controller (rev a1) IOMMU group 14: [15b7:5030] 02:00.0 Non-Volatile memory controller: Sandisk Corp Western Digital WD Black SN850X NVMe SSD (rev 01) This controller is bound to vfio, connected drives are not visible. IOMMU group 15: [1022:43f4] 03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43f4 (rev 01) IOMMU group 16: [1022:43f5] 04:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43f5 (rev 01) [c0a9:2263] 05:00.0 Non-Volatile memory controller: Micron/Crucial Technology P1 NVMe PCIe SSD (rev 03) [N:0:1:1] disk CT1000P1SSD8__1 /dev/nvme0n1 1.00TB IOMMU group 17: [1022:43f5] 04:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43f5 (rev 01) [1b21:1166] 06:00.0 SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02) [3:0:0:0] disk ATA ST4000VN008-2DR1 SC60 /dev/sdb 4.00TB [4:0:0:0] disk ATA ST4000VN008-2DR1 SC60 /dev/sdc 4.00TB [5:0:0:0] disk ATA WDC WD40EFAX-68J 0A82 /dev/sdd 4.00TB [6:0:0:0] disk ATA WDC WD40EFAX-68J 0A82 /dev/sde 4.00TB IOMMU group 18: [1022:43f5] 04:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43f5 (rev 01) IOMMU group 19: [1022:43f5] 04:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43f5 (rev 01) [8086:15f3] 08:00.0 Ethernet controller: Intel Corporation Ethernet Controller I225-V (rev 03) IOMMU group 20: [1022:43f5] 04:0b.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43f5 (rev 01) [14c3:0616] 09:00.0 Network controller: MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter IOMMU group 21: [1022:43f5] 04:0c.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43f5 (rev 01) [1022:43f7] 0a:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43f7 (rev 01) This controller is bound to vfio, connected USB devices are not visible. IOMMU group 22: [1022:43f5] 04:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43f5 (rev 01) [1022:43f6] 0b:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] Device 43f6 (rev 01) This controller is bound to vfio, connected drives are not visible. IOMMU group 23: [1987:5012] 0c:00.0 Non-Volatile memory controller: Phison Electronics Corporation E12 NVMe Controller (rev 01) This controller is bound to vfio, connected drives are not visible. IOMMU group 24: [1002:164e] 0d:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Raphael (rev cb) IOMMU group 25: [1002:1640] 0d:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller IOMMU group 26: [1022:1649] 0d:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] VanGogh PSP/CCP IOMMU group 27: [1022:15b6] 0d:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15b6 Bus 003 Device 001 Port 3-0 ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 004 Device 001 Port 4-0 ID 1d6b:0003 Linux Foundation 3.0 root hub IOMMU group 28: [1022:15b7] 0d:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15b7 Bus 005 Device 001 Port 5-0 ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 005 Device 002 Port 5-1 ID 0930:6544 Toshiba Corp. TransMemory-Mini / Kingston DataTraveler 2.0 Stick Bus 005 Device 003 Port 5-2 ID 17a0:0304 Samson Technologies Corp. Q2U handheld mic with XLR Bus 006 Device 001 Port 6-0 ID 1d6b:0003 Linux Foundation 3.0 root hub IOMMU group 29: [1022:15b8] 0e:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15b8 Bus 007 Device 001 Port 7-0 ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 008 Device 001 Port 8-0 ID 1d6b:0003 Linux Foundation 3.0 root hub XML: Spoiler <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm' id='1'> <name>Windows 11</name> <uuid>3317b80a-4598-a68f-52e0-c87f540a42b0</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/> </metadata> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>12</vcpu> <cputune> <vcpupin vcpu='0' cpuset='2'/> <vcpupin vcpu='1' cpuset='10'/> <vcpupin vcpu='2' cpuset='3'/> <vcpupin vcpu='3' cpuset='11'/> <vcpupin vcpu='4' cpuset='4'/> <vcpupin vcpu='5' cpuset='12'/> <vcpupin vcpu='6' cpuset='5'/> <vcpupin vcpu='7' cpuset='13'/> <vcpupin vcpu='8' cpuset='6'/> <vcpupin vcpu='9' cpuset='14'/> <vcpupin vcpu='10' cpuset='7'/> <vcpupin vcpu='11' cpuset='15'/> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-7.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader> <nvram>/etc/libvirt/qemu/nvram/3317b80a-4598-a68f-52e0-c87f540a42b0_VARS-pure-efi-tpm.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv mode='custom'> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='6' threads='2'/> <cache mode='passthrough'/> <feature policy='require' name='topoext'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <interface type='bridge'> <mac address='52:54:00:74:49:ad'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio-net'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-Windows 11/org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <alias name='input0'/> <address type='usb' bus='0' port='2'/> </input> <input type='mouse' bus='ps2'> <alias name='input1'/> </input> <input type='keyboard' bus='ps2'> <alias name='input2'/> </input> <tpm model='tpm-tis'> <backend type='emulator' version='2.0' persistent_state='yes'/> <alias name='tpm0'/> </tpm> <audio id='1' type='none'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> <rom file='/mnt/user/isos/gpubios.rom'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </source> <boot order='1'/> <alias name='hostdev2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </source> <alias name='hostdev3'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/> </source> <alias name='hostdev4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/> </source> <alias name='hostdev5'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/> </source> <boot order='2'/> <alias name='hostdev6'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x17a0'/> <product id='0x0304'/> <address bus='5' device='3'/> </source> <alias name='hostdev7'/> <address type='usb' bus='0' port='3'/> </hostdev> <hub type='usb'> <alias name='hub0'/> <address type='usb' bus='0' port='1'/> </hub> <memballoon model='none'/> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+100</label> <imagelabel>+0:+100</imagelabel> </seclabel> </domain> System specs: Spoiler MB: Asus B650-A GAMING WIFI CPU: AMD 7800X3D GPU: RTX 3070 32GB RAM 3x Nvme 1x SSD 6x HDD tower-diagnostics-20230522-2336.zip Edited May 23, 20233 yr by Alintya
May 26, 20233 yr Author I just tried to start over completely; new unraid install with 6.11.5, cleared all (unraid)disks, setup unraid with 1 array disk, 1 cache and nothing changed regarding the VM. Array autostart wouldn't work before as it always claimed my system setup had changed.
May 26, 20233 yr 47 minutes ago, Alintya said: Array autostart wouldn't work before as it always claimed my system setup had changed. A new config should fix that.
May 27, 20233 yr Author On 5/26/2023 at 11:32 AM, JorgeB said: A new config should fix that. Yea I meant to say that that has been fixed by reinstalling but the vm issues remain.
June 21, 20233 yr Author I just gave it another shot with 6.12 stable and a brand-new windows install but issues remain; the vm booted up and crashed the unraid host after ~3min Edited June 21, 20233 yr by Alintya
November 2, 20232 yr Author Still happening with updated bios, 6.12.4 and nothing configured but 1 array disk and nvme bind. Any clues on this? Very annoying as my license goes to waste Edited November 2, 20232 yr by Alintya
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.