Jump to content

NVIDIA GPU PASSTHROUGH + WINDOWS 11 WSL2 / VIRTUAL MACHINE PLATFORM AM5


Recommended Posts

Hello All,

 

I've had a successful working AM4 build using Unraid working after some struggle but now on another AM5 based system I'm struggling.

 

I can 100% get the GPU pass-through working on Windows 11 including EUFI/Secure Boot but the moment I enable "Virtual Machine Platform" which is required for WSL2 to work, I start getting the dreaded error 43. I have both an AM5 system with an NVIDIA 3060 and another AM5 system with an NVIDIA 3090 and both have an AMD 7950X in them. Both have the same issue the moment I enable the Windows feature "Virtual Machine Platform".

 

Some context:

- I start with the default Windows 11 template + TPM support (required by my work)

- Everything about the GPU works (until the Virtual Machine Platform optional feature is turned on)

- I have the vbios passed through in both cases (without which the GPU passthrough does not work)

- I have EIFI boot working with primary GPU using 

video=efifb:off

- If I use the virtual display adapter, then nested virtualization works fine (but then I don't have my GPU working)

- I am always performing a "fresh" OS install of Windows 11 so there's no "migration" stuff happening

 

Some things I have tried (ALL attempts do not work for me):

- enabling CSM / Legacy booting (without EIFU)

- Hidding the KVM from the GPU 

<kvm>
   <hidden state="on"/>
</kvm>

- Tweaking the hiding of the KVM from GPU

  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='123456789123'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
    <ioapic driver='kvm'/>

- Syslinux confige changes, like:

kernel /bzimage
append isolcpus=1,3-15,17,19-31 amd_iommu=on iommu=pt nested=1 video=efifb:off initrd=/bzroot

- VM adanced settings: "VFIO allow unsafe interrupts:" changed to "Yes"

- Feature changes, requiring/disabling "hypervisor" feature, svm, trying "host-model" which I had success with previously

  <cpu mode='host-passthrough' check='partial' migratable='off'>
    <topology sockets='1' dies='1' cores='11' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
    <feature policy='disable' name='hypervisor'/>
    <feature policy='require' name='invtsc'/>
    <feature policy='require' name='svm'/>
  </cpu>

- Put the graphics and GPU on the same slot (which I don't believe is required any more with the latest NVIDIA drivers)

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/vbios/MSI NVIDIA GeForce RTX 3060 - 94.06.25.00.87.extracted.gpuz.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>

- Making the primary display a virtual graphics and the 2nd GPU being the NVIDIA card

- bcdedit on the hypervisorschedulertype changing to Classic

bcdedit /set hypervisorschedulertype Classic

- machine using Q35-7.1 instead of i440fx-7.1

- USB controller 3.0 (qemu XHCI) instead of 2.0 EHCI (should be N/A)

- Installing entire virtio drivers/guest host tools, or only installing NetKVM (Red Hat VirtIO Ethernet Adapter) / amd64 (Red Hat VirtIO SCSI controller) drivers (should be N/A)

- Editing power plan to High performance, disabling HD sleep and USB interrupt sleeping (should be N/A)

- Disabling fastboot (should be N/A)

- Disabling hybernation (should be N/A)

powercfg -h off

- Installing first with virtual graphics adapter then swithing to GPU

- echo 0 to unbind the primary display (which is N/A because the GPU is never attached to anything on a fresh Unraid reboot anyway)

- Upgrading to the latest BIOS version

- Enabling SVM in the BIOS

 

Some other notes:

- Obviously I "bind" to VFIO during boot the IOMMU groups for NVIDIA GPU, without which GPU passthrough would not work at all

IOMMU group 13:			 	[10de:2504] 01:00.0 VGA compatible controller: NVIDIA Corporation GA106 [GeForce RTX 3060 Lite Hash Rate] (rev a1)
 	[10de:228e] 01:00.1 Audio device: NVIDIA Corporation GA106 High Definition Audio Controller (rev a1)

- Some of these attempts I know should no longer be applicable, but I have been despirately attempting to get it working so I've tried things that don't make sense "just in case"

- I reboot Unraid after changes just to be certain that one run isn't polluting another run

- As I understand it, latest NVIDIA drivers don't require KVM hidding anymore so I feel that is a dead-end persuit at this point anyway but I tried "just in case"

- When the GPU fails, I see the boot-up screen with the Windows spinner, but once the GPU initializes, the screen freezes

- If I install a remote desktop software, I can SOMETIMES attach to the desktop (othertimes I need to reboot the unraid system first as the machine state goes bad it seems), and there's a screen at default basic resolution, but not displaying through the GPU

- I I remove the GPU and reattach the virtual display, the VM works again

 

For my job, I require Docker running inside Windows which requires WSL2 / Virtual Machine Platform. I wouldn't care otherwise. At this point, I'm about to give up as I have nothing left from google searching that is worth trying.

 

 

Some information from the main display:

Gigabyte Technology Co., Ltd. B650I AORUS ULTRA , Version x.x
American Megatrends International, LLC., Version F4
BIOS dated: Wed 22 Mar 2023 12:00:00 AM PDT

AMD Ryzen 9 7950X 16-Core @ 4500 MHz

 

From system information:

M/B:Gigabyte Technology Co., Ltd. B650I AORUS ULTRA Version x.x - s/n: Default string

BIOS:American Megatrends International, LLC. Version F4. Dated: 03/22/2023

CPU:AMD Ryzen 9 7950X 16-Core @ 4500 MHz

HVM:Enabled

IOMMU:Enabled

Cache:1 MB, 16 MB, 64 MB

Memory:64 GiB DDR5 (max. installable capacity 128 GiB)

Network:bond0: fault-tolerance (active-backup), mtu 1500

Kernel:Linux 5.19.17-Unraid x86_64

OpenSSL:1.1.1s

 

Virtual machine logs when using virtual graphics adapter (full install then shutdown):

2023-04-23 15:10:45.492+0000: Starting external device: TPM Emulator
/usr/bin/swtpm socket --ctrl 'type=unixio,path=/run/libvirt/qemu/swtpm/1-Windows 11-swtpm.sock,mode=0600' --tpmstate dir=/var/lib/libvirt/swtpm/0890b0b6-381e-5830-f588-37be8f91061b/tpm2,mode=0600 --log 'file=/var/log/swtpm/libvirt/qemu/Windows 11-swtpm.log' --terminate --tpm2
2023-04-23 15:10:45.505+0000: starting up libvirt version: 8.7.0, qemu version: 7.1.0, kernel: 5.19.17-Unraid, hostname: Nano
LC_ALL=C \
PATH=/bin:/sbin:/usr/bin:/usr/sbin \
HOME='/var/lib/libvirt/qemu/domain-1-Windows 11' \
XDG_DATA_HOME='/var/lib/libvirt/qemu/domain-1-Windows 11/.local/share' \
XDG_CACHE_HOME='/var/lib/libvirt/qemu/domain-1-Windows 11/.cache' \
XDG_CONFIG_HOME='/var/lib/libvirt/qemu/domain-1-Windows 11/.config' \
/usr/local/sbin/qemu \
-name 'guest=Windows 11,debug-threads=on' \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-Windows 11/master-key.aes"}' \
-blockdev '{"driver":"file","filename":"/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \
-blockdev '{"driver":"file","filename":"/etc/libvirt/qemu/nvram/0890b0b6-381e-5830-f588-37be8f91061b_VARS-pure-efi-tpm.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \
-machine pc-i440fx-7.1,usb=off,dump-guest-core=off,mem-merge=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \
-accel kvm \
-cpu host,migratable=on,topoext=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=none,host-cache-info=on,l3-cache=off \
-m 32768 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":34359738368}' \
-overcommit mem-lock=off \
-smp 22,sockets=1,dies=1,cores=11,threads=2 \
-uuid 0890b0b6-381e-5830-f588-37be8f91061b \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=36,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.0","addr":"0x7"}' \
-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.0","addr":"0x4"}' \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 11/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \
-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x5","drive":"libvirt-3-format","id":"virtio-disk2","bootindex":1,"write-cache":"on"}' \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Win11_22H2_English_x64v1.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device '{"driver":"ide-cd","bus":"ide.0","unit":0,"drive":"libvirt-2-format","id":"ide0-0-0","bootindex":2}' \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.229-1.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
-device '{"driver":"ide-cd","bus":"ide.0","unit":1,"drive":"libvirt-1-format","id":"ide0-0-1"}' \
-netdev tap,fd=37,id=hostnet0 \
-device '{"driver":"virtio-net","netdev":"hostnet0","id":"net0","mac":"52:54:00:95:89:9a","bus":"pci.0","addr":"0x3"}' \
-chardev pty,id=charserial0 \
-device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
-chardev socket,id=charchannel0,fd=35,server=on,wait=off \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
-chardev 'socket,id=chrtpm,path=/run/libvirt/qemu/swtpm/1-Windows 11-swtpm.sock' \
-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
-device '{"driver":"tpm-tis","tpmdev":"tpm-tpm0","id":"tpm0"}' \
-device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"4"}' \
-audiodev '{"id":"audio1","driver":"none"}' \
-vnc 0.0.0.0:0,websocket=5700,audiodev=audio1 \
-k en-us \
-device '{"driver":"qxl-vga","id":"video0","max_outputs":1,"ram_size":67108864,"vram_size":67108864,"vram64_size_mb":0,"vgamem_mb":16,"bus":"pci.0","addr":"0x2"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/009","id":"hostdev0","bus":"usb.0","port":"1"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/008","id":"hostdev1","bus":"usb.0","port":"2"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/005/002","id":"hostdev2","bus":"usb.0","port":"3"}' \
-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","addr":"0x6"}' \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/0 (label charserial0)


qxl_send_events: spice-server bug: guest stopped, ignoring
2023-04-23T15:24:36.356518Z qemu-system-x86_64: terminating on signal 15 from pid 8303 (/usr/sbin/libvirtd)
2023-04-23 15:24:38.978+0000: shutting down, reason=shutdown
  

 

Rebooting after changing to GPU (followed by a shutdown):

2023-04-23 15:31:20.514+0000: Starting external device: TPM Emulator
/usr/bin/swtpm socket --ctrl 'type=unixio,path=/run/libvirt/qemu/swtpm/1-Windows 11-swtpm.sock,mode=0600' --tpmstate dir=/var/lib/libvirt/swtpm/0890b0b6-381e-5830-f588-37be8f91061b/tpm2,mode=0600 --log 'file=/var/log/swtpm/libvirt/qemu/Windows 11-swtpm.log' --terminate --tpm2
2023-04-23 15:31:20.527+0000: starting up libvirt version: 8.7.0, qemu version: 7.1.0, kernel: 5.19.17-Unraid, hostname: Nano
LC_ALL=C \
PATH=/bin:/sbin:/usr/bin:/usr/sbin \
HOME='/var/lib/libvirt/qemu/domain-1-Windows 11' \
XDG_DATA_HOME='/var/lib/libvirt/qemu/domain-1-Windows 11/.local/share' \
XDG_CACHE_HOME='/var/lib/libvirt/qemu/domain-1-Windows 11/.cache' \
XDG_CONFIG_HOME='/var/lib/libvirt/qemu/domain-1-Windows 11/.config' \
/usr/local/sbin/qemu \
-name 'guest=Windows 11,debug-threads=on' \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-Windows 11/master-key.aes"}' \
-blockdev '{"driver":"file","filename":"/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \
-blockdev '{"driver":"file","filename":"/etc/libvirt/qemu/nvram/0890b0b6-381e-5830-f588-37be8f91061b_VARS-pure-efi-tpm.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \
-machine pc-i440fx-7.1,usb=off,dump-guest-core=off,mem-merge=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \
-accel kvm \
-cpu host,migratable=on,topoext=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=none,host-cache-info=on,l3-cache=off \
-m 32768 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":34359738368}' \
-overcommit mem-lock=off \
-smp 22,sockets=1,dies=1,cores=11,threads=2 \
-uuid 0890b0b6-381e-5830-f588-37be8f91061b \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=36,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.0","addr":"0x7"}' \
-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.0","addr":"0x4"}' \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 11/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \
-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"libvirt-3-format","id":"virtio-disk2","bootindex":1,"write-cache":"on"}' \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Win11_22H2_English_x64v1.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device '{"driver":"ide-cd","bus":"ide.0","unit":0,"drive":"libvirt-2-format","id":"ide0-0-0","bootindex":2}' \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.229-1.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
-device '{"driver":"ide-cd","bus":"ide.0","unit":1,"drive":"libvirt-1-format","id":"ide0-0-1"}' \
-netdev tap,fd=37,id=hostnet0 \
-device '{"driver":"virtio-net","netdev":"hostnet0","id":"net0","mac":"52:54:00:95:89:9a","bus":"pci.0","addr":"0x2"}' \
-chardev pty,id=charserial0 \
-device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
-chardev socket,id=charchannel0,fd=35,server=on,wait=off \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
-chardev 'socket,id=chrtpm,path=/run/libvirt/qemu/swtpm/1-Windows 11-swtpm.sock' \
-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
-device '{"driver":"tpm-tis","tpmdev":"tpm-tpm0","id":"tpm0"}' \
-device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"4"}' \
-audiodev '{"id":"audio1","driver":"none"}' \
-device '{"driver":"vfio-pci","host":"0000:01:00.0","id":"hostdev0","bus":"pci.0","addr":"0x5","romfile":"/mnt/user/isos/vbios/MSI NVIDIA GeForce RTX 3060 - 94.06.25.00.87.extracted.gpuz.rom"}' \
-device '{"driver":"vfio-pci","host":"0000:01:00.1","id":"hostdev1","bus":"pci.0","addr":"0x6"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/009","id":"hostdev2","bus":"usb.0","port":"1"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/008","id":"hostdev3","bus":"usb.0","port":"2"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/005/002","id":"hostdev4","bus":"usb.0","port":"3"}' \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/0 (label charserial0)


2023-04-23T15:33:49.265336Z qemu-system-x86_64: terminating on signal 15 from pid 9430 (/usr/sbin/libvirtd)
2023-04-23 15:33:52.100+0000: shutting down, reason=shutdown

 

 

After turning on the virtual machine platform (GPU fails to load with error code 43 so I used remote desktop shutdown to close VM):

2023-04-23 16:08:27.879+0000: Starting external device: TPM Emulator
/usr/bin/swtpm socket --ctrl 'type=unixio,path=/run/libvirt/qemu/swtpm/1-Windows 11-swtpm.sock,mode=0600' --tpmstate dir=/var/lib/libvirt/swtpm/0890b0b6-381e-5830-f588-37be8f91061b/tpm2,mode=0600 --log 'file=/var/log/swtpm/libvirt/qemu/Windows 11-swtpm.log' --terminate --tpm2
2023-04-23 16:08:27.891+0000: starting up libvirt version: 8.7.0, qemu version: 7.1.0, kernel: 5.19.17-Unraid, hostname: Nano
LC_ALL=C \
PATH=/bin:/sbin:/usr/bin:/usr/sbin \
HOME='/var/lib/libvirt/qemu/domain-1-Windows 11' \
XDG_DATA_HOME='/var/lib/libvirt/qemu/domain-1-Windows 11/.local/share' \
XDG_CACHE_HOME='/var/lib/libvirt/qemu/domain-1-Windows 11/.cache' \
XDG_CONFIG_HOME='/var/lib/libvirt/qemu/domain-1-Windows 11/.config' \
/usr/local/sbin/qemu \
-name 'guest=Windows 11,debug-threads=on' \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-Windows 11/master-key.aes"}' \
-blockdev '{"driver":"file","filename":"/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \
-blockdev '{"driver":"file","filename":"/etc/libvirt/qemu/nvram/0890b0b6-381e-5830-f588-37be8f91061b_VARS-pure-efi-tpm.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \
-machine pc-i440fx-7.1,usb=off,dump-guest-core=off,mem-merge=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \
-accel kvm \
-cpu host,migratable=on,topoext=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=none,host-cache-info=on,l3-cache=off \
-m 32768 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":34359738368}' \
-overcommit mem-lock=off \
-smp 22,sockets=1,dies=1,cores=11,threads=2 \
-uuid 0890b0b6-381e-5830-f588-37be8f91061b \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=36,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.0","addr":"0x7"}' \
-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.0","addr":"0x4"}' \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 11/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \
-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x3","drive":"libvirt-3-format","id":"virtio-disk2","bootindex":1,"write-cache":"on"}' \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Win11_22H2_English_x64v1.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device '{"driver":"ide-cd","bus":"ide.0","unit":0,"drive":"libvirt-2-format","id":"ide0-0-0","bootindex":2}' \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.229-1.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
-device '{"driver":"ide-cd","bus":"ide.0","unit":1,"drive":"libvirt-1-format","id":"ide0-0-1"}' \
-netdev tap,fd=37,id=hostnet0 \
-device '{"driver":"virtio-net","netdev":"hostnet0","id":"net0","mac":"52:54:00:95:89:9a","bus":"pci.0","addr":"0x2"}' \
-chardev pty,id=charserial0 \
-device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
-chardev socket,id=charchannel0,fd=35,server=on,wait=off \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
-chardev 'socket,id=chrtpm,path=/run/libvirt/qemu/swtpm/1-Windows 11-swtpm.sock' \
-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
-device '{"driver":"tpm-tis","tpmdev":"tpm-tpm0","id":"tpm0"}' \
-device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"4"}' \
-audiodev '{"id":"audio1","driver":"none"}' \
-device '{"driver":"vfio-pci","host":"0000:01:00.0","id":"hostdev0","bus":"pci.0","addr":"0x5","romfile":"/mnt/user/isos/vbios/MSI NVIDIA GeForce RTX 3060 - 94.06.25.00.87.extracted.gpuz.rom"}' \
-device '{"driver":"vfio-pci","host":"0000:01:00.1","id":"hostdev1","bus":"pci.0","addr":"0x6"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/009","id":"hostdev2","bus":"usb.0","port":"1"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/008","id":"hostdev3","bus":"usb.0","port":"2"}' \
-device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/005/002","id":"hostdev4","bus":"usb.0","port":"3"}' \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/0 (label charserial0)


2023-04-23T16:13:15.456610Z qemu-system-x86_64: terminating on signal 15 from pid 9516 (/usr/sbin/libvirtd)
2023-04-23 16:13:18.296+0000: shutting down, reason=shutdown

 

If anyone has any ideas or suggestions I'd greatly appreciate it as I'm about ready to throw in the towel on making it work (which would be such a shame).

 

 

 

Link to comment
  • 3 months later...

Sadly no. I never was able to successfully have NVidia GPU passthrough (neither RTX 3090 / PNY Quadro) while enabling Virtual Machine Platform on AM4 nor AM5. The moment I enable VMP the GPU drivers stop working. The underlying machine works, as I can VNC to the machine but the GPU passthrough is toast. If anyone ever has success I'd love to replicate the scenario. I had AM4 working for a brief period of time but I can never replicate the success and once an update happened it was toast.

Edited by Robin R
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...