No display output with IGPU passthrough (Windows VM)

santinilor · May 27, 2022

Hi,

I start by saying that I'm a newbie and I'm using Unraid since less than a month. Until now I've successfully used it as NAS, Nextcloud server and Plex Media server and I'm loving it. However I decided to leverage the VM capabilities of the software and tried installing Windows 11.

I've gone through the whole install process using the integrated VNC Remote and everything went smoothly.

Then I tried changing the graphic card setting from "VNC" to "Intel UHD 630" in order to use the display I have connected via HDMI to my Unraid server, but each time I try to start the VM the screen goes black and nothing is shown.

I tried troubleshooting the problem looking at other topics on the forum but I haven't been able to make any progress.

Now from what I read I didn't understand if what I want to do is even possible. Is it possible to passthrough the only GPU I have (the integrated one) to the VM? If so what am I doing wrong? If not, is there any other way to connect to the vm with an external monitor?

I hope someone can help me with this, and I thank in advance everyone.

Here I add my Vm configuration.

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 11</name>
  <uuid>186a5cec-caf2-3ff5-0e0a-c88dc3840378</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/>
  </metadata>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>9</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='6'/>
    <vcpupin vcpu='2' cpuset='1'/>
    <vcpupin vcpu='3' cpuset='2'/>
    <vcpupin vcpu='4' cpuset='8'/>
    <vcpupin vcpu='5' cpuset='3'/>
    <vcpupin vcpu='6' cpuset='4'/>
    <vcpupin vcpu='7' cpuset='10'/>
    <vcpupin vcpu='8' cpuset='5'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/186a5cec-caf2-3ff5-0e0a-c88dc3840378_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='9' threads='1'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 11/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Win11_EnglishInternational_x64v1.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.217-1.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:a4:0c:e4'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='3'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1f' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc05a'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x05ac'/>
        <product id='0x0267'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

Edited May 28, 2022 by santinilor

santinilor · May 27, 2022

Little update:

I tried configuring the VM to use VNC as first graphics card and my IGPU as second graphics card. Now inside the windows setting two display are shown (see images below), but the external monitor keeps having no signal.

santinilor · May 28, 2022

Ok,

I have tried making a brand new windows 11 install and now the device manager shows the IGPU but gives the infamous error 43 (see below).

If anyone can help any assistance would be very appreciated.

Furthermore looking into the system log I noticed this error:

"May 28 20:27:19 Jarvis kernel: pcieport 0000:00:1c.4: AER: Uncorrected (Non-Fatal) error received: 0000:00:1c.4

May 28 20:27:19 Jarvis kernel: pcieport 0000:00:1c.4: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID) May 28 20:27:19 Jarvis kernel: pcieport 0000:00:1c.4: device [8086:a294] error status/mask=00100000/00010000"

Thanks in advance,

This is the new xml configuration:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='6'>
  <name>Windows 11 newtest</name>
  <uuid>00faec1f-e687-dbb1-7f86-cfd3363df5f4</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/>
  </metadata>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='6'/>
    <vcpupin vcpu='2' cpuset='1'/>
    <vcpupin vcpu='3' cpuset='2'/>
    <vcpupin vcpu='4' cpuset='8'/>
    <vcpupin vcpu='5' cpuset='3'/>
    <vcpupin vcpu='6' cpuset='4'/>
    <vcpupin vcpu='7' cpuset='10'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/00faec1f-e687-dbb1-7f86-cfd3363df5f4_VARS-pure-efi-tpm.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='4' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 11 newtest/vdisk1.img' index='3'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Win11_EnglishInternational_x64v1.iso' index='2'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.217-1.iso' index='1'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:0f:5d:db'/>
      <source bridge='br0'/>
      <target dev='vnet4'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-6-Windows 11 newtest/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='3'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
      <alias name='tpm0'/>
    </tpm>
    <graphics type='vnc' port='5900' autoport='yes' websocket='5700' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/mnt/user/isos/vbios/intel-igpu.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1f' function='0x3'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x16' function='0x0'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc05a'/>
        <address bus='1' device='3'/>
      </source>
      <alias name='hostdev3'/>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x05ac'/>
        <product id='0x0267'/>
        <address bus='1' device='4'/>
      </source>
      <alias name='hostdev4'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

Edited May 28, 2022 by santinilor

Naldinho · May 28, 2022

I'm not very knowledgeable about this but as I research my own issue I have come across Error 43 today. It wasn't relevant to my problem but if I remember correctly it has something to do either with groupings or with failing to pass the graphic cards sound -- if the card has sound which almost all do you need to pass through the sound as well even if you don't plan to use it.

On YouTube Space Invader has excellent videos -- if you go and search for Windows or passthrough one of those will result in a video about creating Windows 10 VM and and mentions what causes a Error 43. Wish I could remember specific video but I don't.

santinilor · May 28, 2022

Hi, thank you for the answer.

Is this one the video you're talking about?

https://www.youtube.com/watch?v=miYUGWq6l24&pp=ugMICgJpdBABGAE%3D

Regarding the sound card I am indeed passing it through, but the error still shows up... however I didn't really understand the IOMMU grouping well, so if you have any resource I could look into I would appreciate it.

santinilor · May 29, 2022

So I finally got the display working by changing the bios from OVMF to SaBios. This unfortunately means that I can't install Windows 11 because of the missing TPM support of the BIOS (from what I've understand).

However, despite I would prefer to use Windows 11, is not a major problem for me and hence I installed Windows 10.

Now I got Windows 10 working but the IGPU still gives me Error 43 in the device manager and the screen quality is very poor.

I tried installing the Intel drivers but that didn't improve the situation.

Tried adding this following lines to the XML file as I read in another thread but no success with that either.

<hyperv>
  <relaxed state='on'/>
  <vapic state='on'/>
  <spinlocks state='on' retries='8191'/>
  <vpindex state='on'/>
  <synic state='on'/>
  <stimer state='on'/>
  <reset state='on'/>
  <vendor_id state='on' value='1234567890ab'/>
  <frequencies state='on'/>
</hyperv>

I followed this guide of Spaceinvader which between the other things suggested to use SeaBios.

The best way to install and setup a windows 10 vm Part 2 Hardware Passthrough

Now he goes on explaining how to use a non EFI compatible GPU with OVMF but the sites where he pulls down the BIOS doesn't have one for the Intel UHD 630; I imagine this bios trick is only for external GPUS? Is there any way to make this work with the OVMF bios (and maybe solve the error 43 and the poor image quality)?

Edited May 29, 2022 by santinilor
Update

ghost82 · May 30, 2022

16 hours ago, santinilor said:

Now he goes on explaining how to use a non EFI compatible GPU with OVMF

Where? Are you sure about this?if bios is uefi as far as I know there's no way for a legacy gpu to output anything, but this isn't your case.

You igpu should work both with uefi and legacy bios.

Error 43 is a generic error meaning that the gpu cannot be initialized, this depends on several reasons.

If you get a black screen with ovmf usually it means the gpu is used by something else: make sure to isolate it, check that the igpu is not used by efifb, if it's used disable it in syslinux config (or boot unraid in legacy bios mode), setup the gpu as multifunction (so pass also the audio part), use your own bios dump.

On some builds with 'bugged' bioses this may not be possible at all.

santinilor · May 30, 2022

Hey thank you for your time.

1) SpaceInvader talks about "How to convert GPU to work with EFI for OVMF compatibility" in this video at the minute 11:06. He actually never talks about IGPUs but since the "symptoms" were the same as in my system I figured it could be related

(The best way to install and setup a windows 10 vm Part 2 Hardware Passthrough)

2) I actually found out that error 43 in my system is solved by increasing the vm memory and for this reason I've decided to purchase more ram for my system

3) Researching online I've found a bunch of parameters people suggested to add to the syslinux configuration:

kernel /bzimage
append video=efifb:off,vesafb:off modprobe.blacklist=i2c_i801,i2c_smbus,snd_hda_intel,snd_hda_codec_hdmi,i915,drm,drm_kms_helper,i2c_algo_bit,i915.alpha_support=1 initrd=/bzroot

I see that between them there is the video=efifb:off parameter too so I guess ogpu shouldn't be used by efifb (? I'm very new to all of this so my guesses are pretty much garbage..)

4) I've booted unraid in legacy but that didn't change anything

5) I already was using my bios dump but I tried running spaceinvader script again and the script gives me back this message:

Okay dumping vbios file named intel-igpu2.rom to the location /mnt/user/isos/vbios/ 
/mnt/user/isos/vbios/intel-igpu2.rom is less than 70kb
This seems small but maybe its correct. Please try it. All done !

I tried using the new dumped vbios but no luck with that either.

6) I tried setting "PCIe ACS override" to multi-function but no display output with this setting either.

Now I do think unraid is already using the IGPU because prior to the vm launch the display is showing unraid related log messages but when I start the vm It goes to "no input signal".

This is the xml file of my windows 11 vm with OVMF that I'm trying to use. While for windows 10 increasing ram did solve the error 43, for this windows 11 vm doesn't seem to work. It could be that it needs more than 5 gb but I can't allocate more memory without the vm crashing.

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 11 ovmf</name>
  <uuid>cec42a70-1808-056b-87d4-84ab2c171d86</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/>
  </metadata>
  <memory unit='KiB'>5242880</memory>
  <currentMemory unit='KiB'>5242880</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='8'/>
    <vcpupin vcpu='2' cpuset='3'/>
    <vcpupin vcpu='3' cpuset='9'/>
    <vcpupin vcpu='4' cpuset='4'/>
    <vcpupin vcpu='5' cpuset='10'/>
    <vcpupin vcpu='6' cpuset='5'/>
    <vcpupin vcpu='7' cpuset='11'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/cec42a70-1808-056b-87d4-84ab2c171d86_VARS-pure-efi-tpm.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='4' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 11 ovmf/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Win11_EnglishInternational_x64v1.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.217-1.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:e5:ff:f1'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='3'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
    </tpm>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/vbios/intel-igpu2.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1f' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc05a'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x05ac'/>
        <product id='0x0267'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

I hope all of this makes sense and I'm sorry if I've not been clear.

ghost82 · May 30, 2022

5 hours ago, santinilor said:

1) SpaceInvader talks about "How to convert GPU to work with EFI for OVMF compatibility" in this video at the minute 11:06. He actually never talks about IGPUs but since the "symptoms" were the same as in my system I figured it could be related

Thank you for explaining, now I understand: what he's doing is using a modified bios of an "old" gpu that didn't include efi; older gpus have only legacy bios but for some of them you can modify the vbios and inject also the efi portion, so that they can be used on uefi systems.

But this is not your case, the vbios of your igpu already include efi.

5 hours ago, santinilor said:

I see that between them there is the video=efifb:off parameter too so I guess ogpu shouldn't be used by efifb (? I'm very new to all of this so my guesses are pretty much garbage..)

yes, efifb:off disable the efi framebuffer. If unraid is booted in uefi mode it may attach the efi framebuffer to the primary gpu; if efifb is attached vfio could not reserve bar and you will get black screen or garbage screen, depending on the case.

5 hours ago, santinilor said:

Now I do think unraid is already using the IGPU because prior to the vm launch the display is showing unraid related log messages but when I start the vm It goes to "no input signal".

Yes, this is ok, when booting some lines will be shown on the screen, but as soon as it attaches the gpu to vfio the screen will look like if it's frozen.

This is your video portion passthrough:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/vbios/intel-igpu2.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>

As I wrote earlier you need also to bind to vfio and pass to the vm the audio portion.

I would suggest to use the more modern q35 machine type instead of i440fx, other than that I'm out of ideas.

You can try to build a new q35 vm, after that attach diagnostics file to see if we can help more.

5 hours ago, santinilor said:

Okay dumping vbios file named intel-igpu2.rom to the location /mnt/user/isos/vbios/ 
/mnt/user/isos/vbios/intel-igpu2.rom is less than 70kb
This seems small but maybe its correct. Please try it. All done !

igpu vbios should be about 65 kB, so it should be good (but if you want to be sure just check the file content with a hex editor).

Edited May 30, 2022 by ghost82

santinilor · May 30, 2022

22 minutes ago, ghost82 said:

As I wrote earlier you need also to bind to vfio and pass to the vm the audio portion.

What do you mean? How can I do that?

23 minutes ago, ghost82 said:

You can try to build a new q35 vm, after that attach diagnostics file to see if we can help more.

Ok I'm gonna give it a try right now

I'll keep you updated, and thank you for your time

ghost82 · May 30, 2022

Waiting for diagnostics, it will be a lot easier to explain.

santinilor · May 30, 2022

Ok here's the diagnostic file.

In the meanwhile I'm trying to setup a new vm with q35 and looking online for my igpu bios

jarvis-diagnostics-20220530-1229.zip

ghost82 · May 30, 2022

Try this, for vm "Windows 11 ovmf q35":

1. Replace xml with this:

<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
  virsh edit 1cf47542-bffa-f1a7-50dc-5b1d2ab4250a
or other application using the libvirt API.
-->

<domain type='kvm'>
  <name>Windows 11 ovmf q35</name>
  <uuid>1cf47542-bffa-f1a7-50dc-5b1d2ab4250a</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/>
  </metadata>
  <memory unit='KiB'>5242880</memory>
  <currentMemory unit='KiB'>5242880</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='8'/>
    <vcpupin vcpu='2' cpuset='3'/>
    <vcpupin vcpu='3' cpuset='9'/>
    <vcpupin vcpu='4' cpuset='4'/>
    <vcpupin vcpu='5' cpuset='10'/>
    <vcpupin vcpu='6' cpuset='5'/>
    <vcpupin vcpu='7' cpuset='11'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-q35-6.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/1cf47542-bffa-f1a7-50dc-5b1d2ab4250a_VARS-pure-efi-tpm.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='4' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 11 ovmf q35/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Win11_EnglishInternational_x64v1.iso'/>
      <target dev='hda' bus='sata'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.217-1.iso'/>
      <target dev='hdb' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x18'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x19'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x1a'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x1b'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x1c'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x4'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:fb:4d:6f'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='3'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
    </tpm>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/vbios/i915ovmf.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc05a'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x05ac'/>
        <product id='0x0267'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
</domain>

2. Download the attached zip, extract and save i915ovmf.rom into /mnt/user/isos/vbios/i915ovmf.rom

Note: there's no vnc, leave only the passed through igpu, attach a monitor to the igpu.

Try and pray (optional, but recommended)

i915ovmf.rom.zip

santinilor · May 30, 2022

Unfortunately it seems the prayers weren't enough..🙄😂

No display output and this error message appeared on the vm log

qemu-system-x86_64: -device vfio-pci,host=0000:00:02.0,id=hostdev0,bus=pcie.0,addr=0x2,romfile=/mnt/cache/isos/vbios/i915ovmf.rom: IGD device 0000:00:02.0 cannot support legacy mode due to existing devices at address 1f.0

Here's the updated diagnostic file

jarvis-diagnostics-20220530-1311.zip

ghost82 · May 30, 2022

mmm odd...try to create a i440fx machine type with ovmf, then use the same vbios. After that reattach diagnostics.

santinilor · May 30, 2022

So if I try to modify the "Windows 11 ovmf q35" to use i440fx the vm gives me back this error

XML error: The device at PCI address 0000:00:03.3 cannot be plugged into the PCI controller with index='0'. It requires a controller that accepts a pcie-root-port.

I tried modifying an existing vm I had which used i440fx and the entire server crashed and rebooted (I tried two times, both times a crash). Here's the xml file

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 11 ovmf</name>
  <uuid>cec42a70-1808-056b-87d4-84ab2c171d86</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/>
  </metadata>
  <memory unit='KiB'>5242880</memory>
  <currentMemory unit='KiB'>5242880</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='8'/>
    <vcpupin vcpu='2' cpuset='3'/>
    <vcpupin vcpu='3' cpuset='9'/>
    <vcpupin vcpu='4' cpuset='4'/>
    <vcpupin vcpu='5' cpuset='10'/>
    <vcpupin vcpu='6' cpuset='5'/>
    <vcpupin vcpu='7' cpuset='11'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/cec42a70-1808-056b-87d4-84ab2c171d86_VARS-pure-efi-tpm.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='4' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 11 ovmf/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Win11_EnglishInternational_x64v1.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.217-1.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:e5:ff:f1'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='3'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
    </tpm>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/vbios/i915ovmf.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1f' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc05a'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x05ac'/>
        <product id='0x0267'/>
      </source>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

I created a new vm with i440fx and the server crashed again.

I must say this crash thing is new....but I wouldn't really consider it an upgrade..🙄

Then I tried adding the motherboard sound chip and the server didn't crash but became unreachable on the network forcing me to restart it

This project is becoming an emotional rollercoast..

jarvis-diagnostics-20220530-1353.zip

ghost82 · May 30, 2022

There are no vm defined in your latest zip.

Please define vms and enable also syslog server:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/#comment-781601

so if the system crashes you will have the syslog saved in the logs folder of the flash drive

Attach diagnostics again and that syslog

santinilor · May 30, 2022

Here's the new diagnostic and the syslog

jarvis-diagnostics-20220530-1423.zip

Edited May 30, 2022 by santinilor

ghost82 · May 30, 2022

It seems it's becoming challenging and without having tried directly I'm quite shooting in the dark.

Some words about what we are doing:

your igpu could be passed through in 2 ways:

1. Universal Pass-Through (UPT)

The igpu is attached to the vm as a secondary gpu, without any video output; video output could be obtained with x-igd-opregion (experimental) setting in the vm.

In this case you need another primary gpu, like qxl video driver.

2. Legacy mode

This should be better, because you should have video output and hardware acceleration; target address of the igpu in the vm must be bus 0x00, slot 0x02, function 0x00.

What we are trying to do is gvt-d (direct access).

The i915ovmf.rom comes from:

https://github.com/patmagauran/i915ovmfPkg

Quote

This project attempts to create a UEFI driver for the intel integrated GPUS so that they can be used in VFIO Passthrough. Prior to this driver, there was no easy or reliable solution to both virtualized and direct passthrough(GVT-G/D). This driver adds an opRegion for the iGPU to utilize during the Boot Process, allowing for access to the UEFI menus and any other interfaces that are created before an operating system level driver is initialized. As a bonus, this allows for MacOS to boot in this virtual environment.

A note:

Quote

HDMI Output may not work on your system.

You have some dmar errors in the log, but accordingly to the wiki:

Quote

Additionally, IGD device are known to generate small numbers of DMAR faults
when initially assigned. It is believed that this is simply the IGD attempting
to access the reserved GTT space after reset, which it no longer has access to
when accessed from userspace. So long as the DMAR faults are small in number
and most importantly, not ongoing, these are not an indication of an error.

so, it should be good..

On that github you can download the release of the bios:

https://github.com/patmagauran/i915ovmfPkg/releases

Inside you will find 2 rom files, what I attached before is the NOT "simple" file.

In real I don't know what "simple" means but trying also the simple rom worth a try.

Unfortunately, I'm out of ideas.

You may want to have a read at gvt-g too:

https://forums.unraid.net/topic/108650-plugin-intel-gvt-g/#comment-993616

Edited May 30, 2022 by ghost82

ghost82 · May 30, 2022

2 hours ago, santinilor said:
Unfortunately it seems the prayers weren't enough..🙄😂

No display output and this error message appeared on the vm log
qemu-system-x86_64: -device vfio-pci,host=0000:00:02.0,id=hostdev0,bus=pcie.0,addr=0x2,romfile=/mnt/cache/isos/vbios/i915ovmf.rom: IGD device 0000:00:02.0 cannot support legacy mode due to existing devices at address 1f.0
Here's the updated diagnostic file

About this:

the owner of the repository stated that he also was receiving this error, but the vm boots:

Quote

Okay, So I will play around with this thing!
Meanwhile can you confirm this error is not showing in your qemu (occurs for me after copying yours qemu configs):

qemu-system-x86_64: -device vfio-pci,host=0000:00:02.0,id=hostdev0,bus=pcie.0,addr=0x2,romfile=i915ovmf.rom: IGD device 0000:00:02.0 cannot support legacy mode due to existing devices at address 1f.0

I know qemu will say this as we are on Q35, but I am still figuring out how this was working with your case!

Quote

It does show up with mine but doesn't seem to cause any issue. Based on what I am seeing, somehow my system is happily forcing it into legacy mode despite the fact it shouldn't and other's people's systems are not doing that, instead running in UPT mode. I can boot up MacOS no problem through the ROM. I will try to do some research and if I can't figure it out, I plan to reach out to the original developer of the igd-passthrough tech in QEMU.

You could also try to boot with vnc/qxl, install remote desktop inside windows, stop all, remove vnc/qxl, add igpu on target 00:02.0 (q35/i400fx + ovmf), connect to remote desktop, see what is happening and try to install intel video drivers.

santinilor · May 30, 2022

I tried to use the "simple" rom and the same crash/reboot problem occurred.

So you're telling me that another solution may be UPT but for that I need an additional GPU?

Regarding gvt-g I already stumbled upon it while researching for this topic but is it possible to have HDMI output with it?

Fortunately at least windows 10 works with the SEABios so I will be using that...

santinilor · May 30, 2022

2 minutes ago, ghost82 said:

You could also try to boot with vnc/qxl, install remote desktop inside windows, stop all, remove vnc/qxl, add igpu on target 00:02.0 (q35/i400fx + ovmf), connect to remote desktop, see what is happening and try to install intel video drivers.

Ok I'll try this

ghost82 · May 30, 2022

40 minutes ago, santinilor said:

Regarding gvt-g I already stumbled upon it while researching for this topic but is it possible to have HDMI output with it?

it seems no output, ich777 porposes a displaylink device for video output:

https://forums.unraid.net/topic/108650-plugin-intel-gvt-g/?do=findComment&comment=994459

40 minutes ago, santinilor said:

So you're telling me that another solution may be UPT but for that I need an additional GPU?

Official documentation says:

Quote

Universal Pass-Through (UPT) mode:

In this mode the IGD device is added as a *secondary* (ie. non-primary)

graphics device in combination with an emulated primary graphics device.

This mode *requires* guest driver support to remove the external

dependencies generally associated with IGD (see below). Those guest

drivers only support this mode for Broadwell and newer IGD, according to

Intel. Additionally, this mode by default, and as officially supported

by Intel, does not support direct video output. The intention is to use

this mode either to provide hardware acceleration to the emulated graphics

or to use this mode in combination with guest-based remote access software,

for example VNC (see below for optional output support). This mode

theoretically has no device specific handling dependencies on vfio-pci or

the VM firmware.

So primary card could be emulated with qxl (on bus 0x00), and with upt target address of igpu will be different than 00:02.0

then it adds:

Quote

For UPT mode, retain the default emulated graphics and simply add the vfio-pci

device making use of any other bus address other than 02.0. libvirt will

default to assigning the device a UPT compatible address while legacy mode

users will need to manually edit the XML if using a tool like virt-manager

where the VM device address is not expressly specified.

An experimental vfio-pci option also exists to enable OpRegion, and thus

external monitor support, for UPT mode. This can be enabled by adding

"x-igd-opregion=on" to the vfio-pci device options for the IGD device.

adding x-igd-opregion=on was something like this in the xml:

<qemu:commandline>
  <qemu:arg value='-set'/>
  <qemu:arg value='device.hostdev0.x-igd-opregion=on'/>
  <qemu:arg value='-set'/>
  <qemu:arg value='device.hostdev0.x-igd-gms=1'/>
</qemu:commandline>

with hostdev0 being an example of the alias of the igpu (so an alias for the passed through igpu must be defined in the xml).

But this worked on older version of qemu/libvirt.

the -set commandline has been deprecated by libvirt.

In newer unraid version which includes libvirt 8.2.0 and qemu 6.2 there should be <qemu:override>, but it seems it doesn't work properly at the moment:

https://forums.unraid.net/bug-reports/stable-releases/6100rc7-6102-odd-behaviour-of-libvirt-r1958/

---> issue n.2

Edited May 30, 2022 by ghost82

santinilor · May 30, 2022

Ok thank you. For now I'm pursuing the "setup-via-rdp way" and I'll see if I make any progress.

I must say that this last part with UTP got a bit over my head... it's far more complex than I expected it to be. I'll see what I can do.

I do really appreciate, however, your help and all the time you've put into this thread.

ghost82 · May 30, 2022

29 minutes ago, santinilor said:

I must say that this last part with UTP got a bit over my head... it's far more complex than I expected it to be.

In real there's nothing really difficult, in this mode you need:

In a q35 machine, which has additional bus numbers other than 0x00 (pcie-root-ports):

1. primary gpu (qxl+vnc) on bus 0x00, i.e.:

    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </video>

2. passthrough of the igpu with vbios, source address 00:02.0, target address with an address different than 00:02.0, and maybe in a bus different than 0x00, i.e. (target in this ex is 05:00.0):

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/vbios/i915ovmf.rom'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
      <alias name='ua-igpu'/>
    </hostdev>

with an alias added to it (ua-igpu for example)

3. Add opregion and gms with qemu arguments to enable video output, pointing to the igpu alias:

    <qemu:override>
      <qemu:device alias='ua-igpu'>
        <qemu:frontend>
          <qemu:property name='x-igd-opregion' type='bool' value='true'/>
          <qemu:property name='x-igd-gms' type='unsigned' value='1'/>
        </qemu:frontend>
      </qemu:device>
    </qemu:override>

This for libvirt 8.2.0+, for older versions it was the -set argument:

<qemu:commandline>
  <qemu:arg value='-set'/>
  <qemu:arg value='device.ua-igpu.x-igd-opregion=on'/>
  <qemu:arg value='-set'/>
  <qemu:arg value='device.ua-igpu.x-igd-gms=1'/>
</qemu:commandline>

But (3) for libvirt 8.2.0+ as said is not working correctly at the moment and (3) for libvirt <8.2.0 may require disabling device.json in qemu (quite a mess to follow depending on libvirt/qemu versions), with this additional block:

  <qemu:capabilities>
    <qemu:del capability='device.json'/>
  </qemu:capabilities>

Of course this is only theory for me..

Edited May 30, 2022 by ghost82

No display output with IGPU passthrough (Windows VM)

Recommended Posts

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation