I've got a WIndows 10 VM which I'm trying to do a GPU passthrough.  I installed vnc server before switching from the virtual vnc graphics to my GPU so I could still connect after.  Upon turning the VM on, VNC connects but it is a black screen.  I plugged in a monitor to the GPU, and it also has no connection.  RDC connects fine. 

 

I am trying to set this up so devices on my network can connect to the VM and play roms on Retroarch using Sunshine/Moonlight.  Using the vnc virtual graphics card it worked flawless, clients were able to connect and play games.  The issue is that I need opengl to play N64 games, which upon lots of reading I need to do a GPU passthrough.  But whatever is causing the black screens when the VM is running with only the GPU is also causing Moonlight to be unable to connect.  I can select "Desktop" in the Moonlight options, but it also just gives a black screen.  Selecting Retroarch throws an error about rtsp timeout.

 

I have tried a downloaded vbios and removed the header myself, and I also used SpaceInvaderOne's vbios dump script to create my own.  Don't see a difference using either one of them.

 

tl;dr

GPU passthrough enabled - black screen on vnc and monitor, cannot connect using Moonlight app.  RDC still works.  Need to make it so VNC and monitor are no longer black screens, which will probably solve my moonlight issue.

 

Thank you!!

 

VM log:

Quote

2025-03-25 07:43:55.752+0000: Starting external device: virtiofsd
/usr/libexec/virtiofsd --fd=36 -o source=/mnt/user/roms,cache=always,sandbox=chroot,xattr
2025-03-25 07:43:55.775+0000: starting up libvirt version: 8.7.0, qemu version: 7.2.0, kernel: 6.1.126-Unraid, hostname: dNAS
LC_ALL=C \
PATH=/bin:/sbin:/usr/bin:/usr/sbin \
HOME='/var/lib/libvirt/qemu/domain-3-Windows 10' \
XDG_DATA_HOME='/var/lib/libvirt/qemu/domain-3-Windows 10/.local/share' \
XDG_CACHE_HOME='/var/lib/libvirt/qemu/domain-3-Windows 10/.cache' \
XDG_CONFIG_HOME='/var/lib/libvirt/qemu/domain-3-Windows 10/.config' \
/usr/local/sbin/qemu \
-name 'guest=Windows 10,debug-threads=on' \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-3-Windows 10/master-key.aes"}' \
-blockdev '{"driver":"file","filename":"/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \
-blockdev '{"driver":"file","filename":"/etc/libvirt/qemu/nvram/86248f71-8087-7613-8a86-544e6bd02621_VARS-pure-efi.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \
-machine pc-q35-7.2,usb=off,dump-guest-core=off,mem-merge=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \
-accel kvm \
-cpu host,migratable=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=none,host-cache-info=on,l3-cache=off \
-m 4096 \
-object '{"qom-type":"memory-backend-memfd","id":"pc.ram","share":true,"x-use-canonical-path-for-ramblock-id":false,"size":4294967296}' \
-overcommit mem-lock=off \
-smp 8,sockets=1,dies=1,cores=8,threads=1 \
-uuid 86248f71-8087-7613-8a86-544e6bd02621 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=35,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
-device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
-device '{"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"}' \
-device '{"driver":"pcie-root-port","port":11,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x3"}' \
-device '{"driver":"pcie-root-port","port":12,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x4"}' \
-device '{"driver":"pcie-root-port","port":13,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x1.0x5"}' \
-device '{"driver":"pcie-root-port","port":14,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x1.0x6"}' \
-device '{"driver":"ich9-usb-ehci1","id":"usb","bus":"pcie.0","addr":"0x7.0x7"}' \
-device '{"driver":"ich9-usb-uhci1","masterbus":"usb.0","firstport":0,"bus":"pcie.0","multifunction":true,"addr":"0x7"}' \
-device '{"driver":"ich9-usb-uhci2","masterbus":"usb.0","firstport":2,"bus":"pcie.0","addr":"0x7.0x1"}' \
-device '{"driver":"ich9-usb-uhci3","masterbus":"usb.0","firstport":4,"bus":"pcie.0","addr":"0x7.0x2"}' \
-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"}' \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 10/vdisk1.img","node-name":"libvirt-1-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}' \
-device '{"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk2","write-cache":"on","serial":"vdisk1"}' \
-chardev 'socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-3-Windows 10/fs0-fs.sock' \
-device '{"driver":"vhost-user-fs-pci","id":"fs0","chardev":"chr-vu-fs0","queue-size":1024,"tag":"roms","bus":"pci.1","addr":"0x0"}' \
-netdev tap,fd=36,id=hostnet0 \
-device '{"driver":"virtio-net","netdev":"hostnet0","id":"net0","mac":"52:54:00:b2:37:1f","bus":"pci.2","addr":"0x0"}' \
-chardev pty,id=charserial0 \
-device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
-chardev socket,id=charchannel0,fd=34,server=on,wait=off \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
-device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \
-audiodev '{"id":"audio1","driver":"none"}' \
-device '{"driver":"vfio-pci","host":"0000:01:00.0","id":"hostdev0","bus":"pci.5","multifunction":true,"addr":"0x0","romfile":"/mnt/user/isos/vbios/GeForce RTX 4060.rom"}' \
-device '{"driver":"vfio-pci","host":"0000:01:00.1","id":"hostdev1","bus":"pci.5","addr":"0x0.0x1"}' \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/1 (label charserial0)

 

VM xml:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='7'>
  <name>Windows 10</name>
  <uuid>847d5471-eeb6-764e-c429-aef66e7bc611</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>6291456</memory>
  <currentMemory unit='KiB'>6291456</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <source type='memfd'/>
    <access mode='shared'/>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='1'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='3'/>
    <vcpupin vcpu='4' cpuset='4'/>
    <vcpupin vcpu='5' cpuset='5'/>
    <vcpupin vcpu='6' cpuset='6'/>
    <vcpupin vcpu='7' cpuset='7'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-7.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/847d5471-eeb6-764e-c429-aef66e7bc611_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='8' threads='1'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img' index='2'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <serial>vdisk1</serial>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.262-2.iso' index='1'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='pci' index='1' model='pci-bridge'>
      <model name='pci-bridge'/>
      <target chassisNr='1'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
    <filesystem type='mount' accessmode='passthrough'>
      <driver type='virtiofs' queue='1024'/>
      <binary path='/usr/libexec/virtiofsd' xattr='on'>
        <cache mode='always'/>
        <sandbox mode='chroot'/>
      </binary>
      <source dir='/mnt/user/roms/romm/'/>
      <target dir='roms'/>
      <alias name='fs0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </filesystem>
    <interface type='bridge'>
      <mac address='52:54:00:f1:df:32'/>
      <source bridge='br0'/>
      <target dev='vnet6'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-7-Windows 10/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/mnt/user/isos/Asus.RTX4060Ti.8192.2303291.rom'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x06' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x06' function='0x1'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

 

syslinux.cfg:

Quote

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label Unraid OS
  menu default
  kernel /bzimage
  append vfio-pci.ids=10de:2882,10de:22be pcie_acs_override=multifunction video=efifb:off,vesafb:off initrd=/bzroot
label Unraid OS GUI Mode
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui
label Unraid OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Unraid OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest
 

 

 

Okay, so I ran through everything again.  I did notice that going to Main -> Flash -> SysLinux in the webGUI did not output what is saved in my syslinux.cfg, so I added the same lines there.

Rebooted and started the VM. If I connect through VNC as soon as it begins booting, I can see the desktop load, but then the VNC connection freezes.  So it seems something (most likely a driver) loading at startup is causing the remote view to mess up.  I've gone into device manager and uninstalled the nvidia drivers for the 4060 and rebooted, but still have the same issue.  I read disabling UEFI may help, so I did that and rebooted the server, still did not help..

 

 

So I thought it may just be the VM so I installed a fresh copy of Win11, set up VNC server after creating an account, shut it down and switched from virtual to the GPU.  Still get a black screen connecting through VNC, and still have no signal on the monitor.  The log does show this new error though, and I haven't found anything on google about it.  Disregard them being on a different bus, I already fixed that.

 

syslog shows this however:

Mar 28 03:03:30 dNAS kernel: vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x1e@0x258
Mar 28 03:03:30 dNAS kernel: vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x19@0x900
Mar 28 03:03:30 dNAS kernel: vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x26@0xc1c
Mar 28 03:03:30 dNAS kernel: vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x27@0xd00
Mar 28 03:03:30 dNAS kernel: vfio-pci 0000:01:00.0: vfio_ecap_init: hiding ecap 0x25@0xe00
Mar 28 03:03:30 dNAS kernel: vfio-pci 0000:01:00.1: vfio_ecap_init: hiding ecap 0x25@0x160

 

 

 

*edit* Holy cowwww.  OKAY! So I changed the machine from i440fx to q35, changed cdrom bus to SATA, removed the vbios, and passed a keyboard through.  I have no idea which option fixed it (I changed to q35 before and had no success) but I'm able to keep the virtual gpu and add my physical card as a second gpu. I'm able to connect using VNC, monitor shows output, and most of all I'm able to connect using moonlight and play N64 games through retroarch.

Edited March 28, 20251 yr by Valr