AMD APU Ryzen 5700G iGPU Passthrough on 6.9.2

[wx-Rmt]

Therefore, assuming that the principle of amd igpu and gpu reset and repair is the same,

it is feasible to try to solve the reset method of amd igpu,

which is to add the igpu hardware to https://github.com/gnif/vendor-reset, after compiling loaded into the host system

 

I have stopped tossing unraid and related amd igpu passthrough

 

good luck everyone

 

 

[thomas]

On 9/1/2022 at 3:58 PM, Name_0901 said:

 

I was successful on PVE 7.2. The iGPU can be passthrough in both Windows 10 and Ubuntu 20.04 VM, and works well.

Hi, have you done anything special to passthrough the igpu  under ubuntu? 

For me, I can only boot it with modeset parameter otherwise the screen will turn off and in dmesg I will see amdgpu giving errors. Due to modeset there is no video acceleration, so it is not usable.

 

Thanks

[Name_0901]

16 hours ago, thomas said:

Hi, have you done anything special to passthrough the igpu  under ubuntu? 

For me, I can only boot it with modeset parameter otherwise the screen will turn off and in dmesg I will see amdgpu giving errors. Due to modeset there is no video acceleration, so it is not usable.

 

Thanks

Try reboot your PVE host before do the iGPU passthrough. This is because the AMD GPU reset bug, you can only use the iGPU in passthrough once per host startup. If you want to use it next time, or your VM rebooted, you need to restart your host machine to re-active the iGPU.

 

wx-Rmt posted a solution, you can take a look here at page 7. But since I'm a C noob, there's nothing I can do to rewrite the patch.

 

You also can googling "AMD GPU/iGPU reset bug" for more details.

Edited September 16, 2022 by Name_0901

[beetlebug515]

Or maybe someone at UnRaid can just fix this...........

[thomas]

On 9/16/2022 at 2:34 PM, Name_0901 said:

Try reboot your PVE host before do the iGPU passthrough. This is because the AMD GPU reset bug, you can only use the iGPU in passthrough once per host startup. If you want to use it next time, or your VM rebooted, you need to restart your host machine to re-active the iGPU.

 

Thanks. Unfortunately for me Ubuntu or any other linux is not working. Windows is always fine with the RadeonReset service from https://github.com/inga-lovinde/RadeonResetBugFix

For context, I'm using an Asrock deskmini X300W with proxmox.

[ven0m]

Nothing new I guess?

Read through the whole Thread and tried everything.
VBIOS extraction worked for my AsRock A520M-HVS.

No matter what I do, it always ends with 

 

qemu-system-x86_64: -device vfio-pci,host=0000:30:00.0,id=hostdev0,bus=pci.4,multifunction=on,addr=0x0,romfile=/mnt/cache/test12345/vbios_1638.rom: Failed to mmap 0000:30:00.0 BAR 0. Performance may be slow

In the VM-Log and it stays on the Unraid Login screen...

Attached my VBIOS if it may help anyone.

vbios_1638.rom

 

 

If I use VNC and HD Audio I get sound over HDMI, that works.

Edited September 26, 2022 by ven0m

[onyxdrew]

I thought I would provide a quick update.  I ditched my AMD rig; 5700G + 5500XT, and replaced it with Intel i5 12400 + 1050ti.  Plex natively supports intel apu and it worked without any drivers or effort at all, just pass device and turn it on.  It seems plenty powerful.  I also was able to pass my 1050ti into a VM with zero issues, no need to extract BIOS or anythign fancy.  I even have it working with UNRAID 6.11 and Windows 11 with TPM.  It was crazy how easy this just worked, with all the pain I had with AMD rig.  The AMD stuff was more power efficient, but honestly worth the trade off.  I know this isn't overly helpful here, but know that this AMD + APU stuff is probably just not even worth the effort.  The performance was not good, and overall nothing just works on this platform.  Happy unraider now unsubscribing.

[optiluca]

Just wanted to thank Tanne for the guide. By following it I managed to get my 5600G iGPU passed through successfully.  Additionally to all the points listed here, I had to change a few things (which took the best part of a day to figure out, so here they are for everyone else!):

• In your BIOS
  • Make sure that the iGPU is enabled as the default (my ASRock mobo had external GPU as default.  But I have no external GPU!).
  • I disabled CSM as well - no idea if it was needed or not, but didn't seem to hurt.
• I think the notation used in step 6.3 might be obsolete.  I was getting error messages about "hostdev0" not being a valid device.  I had to change that whole section to:

    <qemu:override>
      <qemu:device alias='hostdev0'>
        <qemu:frontend>
          <qemu:property name='x-vga' type='bool' value='true'/>
        </qemu:frontend>
      </qemu:device>
    </qemu:override>

• I was seeing the same warning message as @ven0m above.  Unraid was using the GPU for its terminal output, I had to modify the Syslinux configuration in Unraid and add  video=efifb:off to the kernel argument list.  After a reboot, the warning was gone (and indeed Unraid stopped writing output to my monitor during the boot sequence!)
• In general, I found myself "fighting" against whatever xml validation mechanism is in place for the VM configuration in Unraid.  Often I'd make changes, save them, but re-opening the config would reveal that my configuration had been removed.  I guess you guys already know this, but I found it quite confusing at first...

On my Mobo I had to set PCIe ACS override=both and VFIO allow unsafe interrupts=yes, as my VGA and HDMI audio controllers shared a group with a tonne of other devices.

 

I tested the system for an hour or so yesterday, and everything seemed OK, apart from some slight audio stutters every now and then.  I'll be keeping it monitored.

 

Hopefully someone will find these additions helpful.

 

Luca

Edited December 12, 2022 by optiluca

[Manthony]

On 12/12/2022 at 3:10 AM, optiluca said:

Just wanted to thank Tanne for the guide. By following it I managed to get my 5600G iGPU passed through successfully.  

Optiluca, after seeing you were successful I decided to try it myself. For flavor text, I've been snubbed this issue since 2018 with a raven ridge 2400 G, and I've long since given up on this ever actually working in a windows VM. Trying all of the steps you followed including your modifications, I'll admit that this is closer to "actually working" than it's ever been before, but still with quite a number of issues sadly.

Something about the initial iGPU handoff is going buggy for me - I can start the VM cleanly, but as soon as I remote desktop into it, unraid loses connection to it's flash drive (I lose a good chunk of the webUI functionality, but the array doesn't crash).
Within the VM, the vega graphics device is correctly showing but the display driver has a yellow alert next to it - Windows reports that the device has issues and the driver isn't properly installed (code 43). As a result of all this, I'm not able to get any HDMI out from the server itself. Really not sure what to try next, I can tell it's sort-of working because the remote desktop is full resolution and whatnot, but I have no idea how to iron out what's going on behind the scenes. I believe the driver issue is just a symptom of a bigger underlying problem.

 

One thing I'm really curious about is the <qemu:override> section, I am also seeing the XML file seemingly revert that section when I reopen it, which makes me think there is something invalid there which is causing the document to not save properly. I also got the "hostdev0 not valid" issue when I used the code snippet in Tanne's guide, but using that code the XML at least saved properly...

I'm still holding onto a flicker of hope that this can work, because there's no room in my server for a dedicated GPU and I would hate to have to give up/start over with a different processor after all this. But man, I really can't say I recommend using AMD APUs with unraid, considering all this hassle.

[ghost82]

6 hours ago, Manthony said:

I am also seeing the XML file seemingly revert that section when I reopen it

Make sure you have this at the top of your xml:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

otherwise the qemu:override section is considered not valid.

 

6 hours ago, Manthony said:

I also got the "hostdev0 not valid" issue

hostdev0 is an alias for the device; if not specified in the xml it is assigned automatically by libvirt/qemu, so yours can be different than hostdev0 (look at the libvirt/qemu logs in your diagnostics and you will find what is your alias); you can assign manually your alias with what you want in the hostdev section of your xml.

Quote

The <qemu:device> sub-element groups overrides for a device identified via the alias attribute. The alias corresponds to the <alias name=''> property of a device. It's strongly recommended to use user-specified aliases for devices with overridden properties.

 

Note also that you need at least libvirt 8.2.0 to use qemu:frontend

Edited December 15, 2022 by ghost82

[Manthony]

14 hours ago, ghost82 said:

Make sure you have this at the top of your xml:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

otherwise the qemu:override section is considered not valid.

 

Thanks for the tip, not having any issues saving the XLM document after adding that in, so at least that's working better.

As for actually getting a VM working, it's somehow going much worse today than yesterday. I did see a brief blip of HDMI out, which was a new first and probably a good sign, but every time I start the VM unraid loses the flash drive after a few moments, the VM becomes unreachable, and I need to reboot everything.

If I had to guess, either it's attempting to pass through all of the USB ports and losing the flash drive, or maybe it's just more bad igpu passthrough issues dropping the ball. Really baffling because it worked at least a couple of times yesterday, but today turning on the VM instantly breaks it, and I can't remote desktop at all. I've tried making a brand new VM following the instructions in this thread to a T, XML config is good, IOMMU groups are good, iGPU and audio are in their own groups without any USB controllers, so I don't know why the flash drive would be getting bumped off while the rest of the array stays 'on'

 

 

EDIT, took a peek at the array logs when I kick off the VM. what stands out to me is that a bunch of things are timing out, but really I don't have a clue what this is about or how to fix it. Snippet ends when bunch of USBs disconnect which leads to all of the errors associated with the flash drive being gone.

 

Edit 2: flash drive doesn't disconnect after I moved it to the front panel - not ideal for aesthetic reasons but maybe not a permanent change. I can now attempt to spin up the VM as much as I want without it impacting the array. Still consistently not taking off properly when using igpu passthrough though, logs continue to show many timeouts and remoting in is a no-go. Adding the server logs

 

 

logs.txt VMlogs.txt

Edited December 15, 2022 by Manthony
updated logs

[ghost82]

11 hours ago, Manthony said:

logs continue to show many timeouts

I'm not confident it will work, but you could try to add to your syslinux config:

iommu=soft

and/or

pci=noats

 

[Manthony]

11 hours ago, ghost82 said:

I'm not confident it will work, but you could try to add to your syslinux config:

iommu=soft

and/or

pci=noats

 

Thanks, I included both of those and it seems like pci=noats fixes the timeout issue without causing any other problems (iommu=soft didn't work on it's own and didn't seem to make a difference when set). With that out of the way, the next issue is that the kernel gets stuck at

kernel: vfio-pci 0000:09:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none

afterwards nothing else happens and the VM doesn't start.
 

spent some time on google today looking into this one, a fair number of amd VM users, apparently. I couldn't find any definitive solution though, at least not any that seemed applicable.

[ludaye]

unraid 6.92版本有问题 ，我的配置是 5700g 32g 3200频率内存条 主板微星b450m 迫击炮 ，6.92时候怎么搞都不行显卡直通后来 直接用 6.11.5 版本直接搞定了 在win10 虚拟机 显示ok  ，方法从官网下载微星主板对应的固件 然后用UBU_v1_79_17 提取vblos 固件 就是 vbios_1638.dat 这个文件 然后其他就是照常配置虚拟机就行了 然后虚拟机里面安装驱动可以正确识别 apu 显卡。我的建议是如果是新的硬件尽量用新的unraid 版本！

[Goldino]

Hi everyone, i have a 5700g, on a msi x470 gaming plus max, with the lates bios.
i used the "UBU" tool to extract the rom, renamed it with "name.rom" and follow several times the step previusly posted.

The unraid version of my server is 6.11.5, my flash setup is:

kernel /bzimage
append pcie_acs_override=downstream,multifunction initrd=/bzroot vfio_iommu_type1.allow_unsafe_interrupts=1 video=efifb:off

and i created the amdgpu.config in /boot/config/modprobe.d

i have other 2 gpu in my server, for other 2 VM (1 gpu each VM), and it works great.

when i try to run my 3rd vm, with the 5700g GPU and few cores, looks like everything is ok, (i follow all step to get this point like:
install windows with no gpu
enable rdp

disable windows update and autoinstall driver

but nothing let me use my onboard graphics (even unplug the HDMI before start and plug after the VM is "ready" works).

The only problem i found is this:

i cannot use the code:

  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-vga=on'/>
  </qemu:commandline>

because i get this error:

so, and i use

<qemu:override>
  <qemu:device alias='hostdev0'>
    <qemu:frontend>
      <qemu:property name='x-vga' type='bool' value='true'/>
    </qemu:frontend>
  </qemu:device>
</qemu:override>

and the machine is running, the gpu is in the device list, has no error, latest driver installed but i wont show any video.

 

i almost forgot to say: if remove the "virtual GPU" an select the "Radeon" one, the system wont start, and i need to make a new XML.

 

Can someone tell me what im doing wrong? there is any chance to fix my issiue for 6.11.5?

Thanks everyone for your work and support.

 

this is the entire schema of my windows 10 machine

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>Windows 10</name>
  <uuid>9f862754-778a-12b8-9566-ec1e1988d38c</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='8'/>
    <vcpupin vcpu='2' cpuset='1'/>
    <vcpupin vcpu='3' cpuset='9'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-q35-6.2'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='3D6L09A17K3O'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='2' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Win10iGPU/vdisk.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </disk>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:b6:00:04'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='it'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x30' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/gpubios/radeonIGPU2.rom'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x30' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x30' slot='0x00' function='0x4'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <qemu:override>
    <qemu:device alias='hostdev0'>
      <qemu:frontend>
        <qemu:property name='x-vga' type='bool' value='true'/>
      </qemu:frontend>
    </qemu:device>
  </qemu:override>
</domain>

 

Edited January 23, 2023 by Goldino
forgot something

[Goldino]

a little update: if i try to boot the machine without the disk and the Windows iso, and the iGPU RUN! But.... it wont boot if i set the disk or the iso.... the Firts core is stuck at 100% and i get only a black screen.

the only think is running on my server is the VM with the iGPU.

Can someone tell me what is going on?

Thanks

[elovsa]

On 1/10/2023 at 9:12 AM, ludaye said:

unraid 6.92版本有问题 ，我的配置是 5700g 32g 3200频率内存条 主板微星b450m 迫击炮 ，6.92时候怎么搞都不行显卡直通后来 直接用 6.11.5 版本直接搞定了 在win10 虚拟机 显示ok  ，方法从官网下载微星主板对应的固件 然后用UBU_v1_79_17 提取vblos 固件 就是 vbios_1638.dat 这个文件 然后其他就是照常配置虚拟机就行了 然后虚拟机里面安装驱动可以正确识别 apu 显卡。我的建议是如果是新的硬件尽量用新的unraid 版本！

兄弟，你这个能直接HDMI接口显示虚拟win画面吗

[Goldino]

On 1/23/2023 at 2:44 AM, Goldino said:

Hi everyone, i have a 5700g, on a msi x470 gaming plus max, with the lates bios.
i used the "UBU" tool to extract the rom, renamed it with "name.rom" and follow several times the step previusly posted.

The unraid version of my server is 6.11.5, my flash setup is:

kernel /bzimage
append pcie_acs_override=downstream,multifunction initrd=/bzroot vfio_iommu_type1.allow_unsafe_interrupts=1 video=efifb:off

and i created the amdgpu.config in /boot/config/modprobe.d

i have other 2 gpu in my server, for other 2 VM (1 gpu each VM), and it works great.

when i try to run my 3rd vm, with the 5700g GPU and few cores, looks like everything is ok, (i follow all step to get this point like:
install windows with no gpu
enable rdp

disable windows update and autoinstall driver

but nothing let me use my onboard graphics (even unplug the HDMI before start and plug after the VM is "ready" works).

The only problem i found is this:

i cannot use the code:

  <qemu:commandline>
    <qemu:arg value='-set'/>
    <qemu:arg value='device.hostdev0.x-vga=on'/>
  </qemu:commandline>

because i get this error:

so, and i use

<qemu:override>
  <qemu:device alias='hostdev0'>
    <qemu:frontend>
      <qemu:property name='x-vga' type='bool' value='true'/>
    </qemu:frontend>
  </qemu:device>
</qemu:override>

and the machine is running, the gpu is in the device list, has no error, latest driver installed but i wont show any video.

 

i almost forgot to say: if remove the "virtual GPU" an select the "Radeon" one, the system wont start, and i need to make a new XML.

 

Can someone tell me what im doing wrong? there is any chance to fix my issiue for 6.11.5?

Thanks everyone for your work and support.

 

this is the entire schema of my windows 10 machine

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>Windows 10</name>
  <uuid>9f862754-778a-12b8-9566-ec1e1988d38c</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='8'/>
    <vcpupin vcpu='2' cpuset='1'/>
    <vcpupin vcpu='3' cpuset='9'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-q35-6.2'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='3D6L09A17K3O'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='2' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Win10iGPU/vdisk.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </disk>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:b6:00:04'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='it'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x30' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/gpubios/radeonIGPU2.rom'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x30' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x30' slot='0x00' function='0x4'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <qemu:override>
    <qemu:device alias='hostdev0'>
      <qemu:frontend>
        <qemu:property name='x-vga' type='bool' value='true'/>
      </qemu:frontend>
    </qemu:device>
  </qemu:override>
</domain>

 

And i finally did it! The only think i change is Machine from Q35-6.2 to i440fx-4.1, i havent try different setting because is WORKS!

 

win10iGPU.xml

[jesta]

Just wanting to add a report to this topic since I mainly bought the 5700G to run a VM off the iGPU:

 

For me it's NOT working.

I can get the VM to POST.

In Ubuntu I can navigate the first install menu but if I select install Ubuntu the screen goes black and nothing comes up. Using safe graphics I can install the OS but after reboot the screen stays black (rebooted server also)

In PopOS I can see the boot messages scrolling and when the UI loads the screen goes black and doesn't come back.

 

Looks like loading the driver makes the iGPU crap the bed.

This feels so close yet so far away.

 

Specs:

5700G

MSI B450m Mortar (Non-Max), BIOS 7B89v1I (AGESA ComboAm4v2PI 1.2.0.7)

3070 as second GPU

VBIOS extracted with UBU (1638)

 

BIOS Settings:

Re-Size BAR Support           --enable

Above 4G memory/Crypto    --enable

Initiate Graphic Adapter       --IGD

Integrated Graphics             --Force

UMA Frame Buffer Size        --4G

IOMMU                                --enable

DMA Protection                   --auto

DMAr Support                     --auto

SVM Mode                          --enable

NX Mode                            --enable

Boot mode                         --UEFI

 

Unraid config:

video=efifb:off

No ACS override needed, GPU and Audio are in their own IOMMU groups

added modprobe blacklist amdgpu

 

VM

OS: Ubuntu 22.10 or PopOS 22.04

Machine: tried both Q35-6.2 and i440fx-4.1

BIOS: SeaBIOS

 

VM XML:

<kvm>
      <hidden state='on'/>
</kvm>

 

<hostdev>

      ... edited the addresses, multifunction etc ...

</hostdev>

 

<qemu:commandline>
     <qemu:arg value='-set'/>
     <qemu:arg value='device.hostdev0.x-vga=on'/>

</qemu:commandline>

Edited March 8, 2023 by jesta

[DocMAX]

No matter what i do, BIOS or UEFI, Windows or Linux, i get pixel junk:

 

Edited March 27, 2023 by DocMAX

[Yellow NL]

I started this route in 2021 withouth succes. 

Now i came back 1.5 year later to see and try it again. 

in the meentime i switched the 5700G for a 5900x and now, switched it back due to insane electrical bills. 

 

And, with the help of the guys here i succeeded! thanks for that!

So the instructions on page 7 and the Quemu lines on page 8 worked for me. 

The bios that worked for me was the i440x on seabios 

 

Then the trick for me for the pixelated screen, and the gpu vendor reset was to add the gpu as 2nd gpu! 

first gpu the virtual screen. seccond gpu the cezanne 5700g gpu.

this way i could install the amd drivers. 

then installed the vendor reset. 

the vendor reset github page specificaly says that you need a seccond gpu driver loaded so it can switch the iGpu and the seccond gpu on and off. 

 

so what this does : i start the vm. it starts with the virtual gpu. then the vendor reset turns off the virtual gpu and turns on the igpu. and boom. there is screen.

 

 

[techhit]

On what version of Unraid?

[TVsIan]

Just wondering if anyone's had any consistent success with this. So far, I've been able to get Device Manager saying the graphics adapter is working, but with the screen still showing the point at the Unraid boot where the graphics & sound devices get bound to VFIO. And that's with the APU as the secondary graphics device and the VNC graphics as primary - if I disable the VNC graphics, Windows will freeze at some point during boot. If I leave it on, I can use Windows indefinitely, just with no display outside VNC or RDP.

The system is ITX and the PCI slot is already in use with an SATA card, so I can't put in a dedicated GPU.
My syslinux config is:

append iommu=soft pci=noats video=efifb:off pcie_acs_override=downstream,multifunction vfio_iommu_type1.allow_unsafe_interrupts=1 initrd=/bzroot

And my XML is:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='3' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>Windows 10</name>
  <uuid>6d499928-104a-0752-7ab8-89914d314d54</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>12</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='10'/>
    <vcpupin vcpu='2' cpuset='3'/>
    <vcpupin vcpu='3' cpuset='11'/>
    <vcpupin vcpu='4' cpuset='4'/>
    <vcpupin vcpu='5' cpuset='12'/>
    <vcpupin vcpu='6' cpuset='5'/>
    <vcpupin vcpu='7' cpuset='13'/>
    <vcpupin vcpu='8' cpuset='6'/>
    <vcpupin vcpu='9' cpuset='14'/>
    <vcpupin vcpu='10' cpuset='7'/>
    <vcpupin vcpu='11' cpuset='15'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-7.1'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='AWDRGYJILPZC'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='6' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Windows.iso' index='3'/>
      <backingStore/>
      <target dev='hda' bus='sata'/>
      <readonly/>
      <boot order='2'/>
      <alias name='sata0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.229-1.iso' index='2'/>
      <backingStore/>
      <target dev='hdb' bus='sata'/>
      <readonly/>
      <alias name='sata0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img' index='1'/>
      <backingStore/>
      <target dev='hdc' bus='sata'/>
      <serial>vdisk1</serial>
      <boot order='1'/>
      <alias name='sata0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x11'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x12'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x13'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x14'/>
      <alias name='pci.6'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:ba:d6:7c'/>
      <source bridge='br0'/>
      <target dev='vnet2'/>
      <model type='rtl8139'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-3-Windows 10/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='vnc' port='5901' autoport='yes' websocket='5701' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/mnt/user/isos/vbios_1638.rom'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
  <qemu:override>
    <qemu:device alias='hostdev0'>
      <qemu:frontend>
        <qemu:property name='x-vga' type='bool' value='true'/>
      </qemu:frontend>
    </qemu:device>
  </qemu:override>
</domain>

With this combo + the current drivers + the reset fix service, it boots with the VNC console, that goes black once the reset service kicks in, then I can load up remote desktop. That shows AMD Radeon Graphics and Microsoft Display Adapter as working, Red Hat QXL controller as disabled, and in System Devices, Microsoft Basic Display Driver as failed. No displays are detected in display settings. The VBIOS was dumped from the download for my current BIOS (asrock B550 phantom ITX), and I tried setting initial graphics to APU in the system BIOS as well.
I've tried different VM BIOS options, at one point the boot text turned from white to blue, another the screen went black, and when forcing a change from SeaBIOS in the XML, I was able to get a Windows install screen up (presumably the boot switched from legacy to UEFI), but the VM froze after that. I haven't been able to get anything usable yet.
Anything I might be missing, or is this just one of those "works for some people but not others" things?

[kurai]

I'm leaning towards the process/functionality being one of those "works for some people but not others" things at this stage.

 

I banged my head against a series of brick walls for an unreasonable amount of time trying all the many and varied approaches I found during the research phase.

 

At the moment I've given up and left it alone, checking back on the few worthwhile information sources now and again to see if anything has substantially improved.

[Mazvydas]

Starting to create parts list for an Unraid build that I plan to center around a 5700G with iGPU passthrough to a Windows 10 VM.

 

If those who have managed to get passthrough working could please share what motherboard they're using and bios version I would be very grateful :) Thanks!