So I decided to try to passthrough an igpu but it causes unraid ui to crash. I'm not able to access vms, dashboard, and a few other areas but ultimately unable to access the unraid gui completely. I'm able to access the actual shares on other PC etc. The VM also doesn't successfully start. The only way to solve the hanging problem is to force restart via holding the power button. (having auto-start vm and server was a nightmare as safemode did not stop vm autostart)

 

Later put in my amd gpu and it does the same thing when trying to passthrough.

I tried to follow spaceinvader's gpu passthrough guide later and found that my gpu (while it shows up in the form view) doesn't show up in the xml view. GPU has 01:00.0 and XML is below (maybe I missed it?). The igpu has 00:02.01 which I can't find in the xml either.

 

I then took a look at the IOMMU group and wasn't sure if the grouping was causing the issue. Take a look please?

 

I enabled in the bios multimonitor so that the igpu and the dedicated gpu are both active and why both are here but it did not make a difference when multimonitor was off.

 

Thank you for reading.

 

 

XML:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Server 2.0</name>
  <uuid>dee16a86-c187-c737-7fb5-aed609f94cf7</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>18</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='11'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='12'/>
    <vcpupin vcpu='4' cpuset='3'/>
    <vcpupin vcpu='5' cpuset='13'/>
    <vcpupin vcpu='6' cpuset='4'/>
    <vcpupin vcpu='7' cpuset='14'/>
    <vcpupin vcpu='8' cpuset='5'/>
    <vcpupin vcpu='9' cpuset='15'/>
    <vcpupin vcpu='10' cpuset='6'/>
    <vcpupin vcpu='11' cpuset='16'/>
    <vcpupin vcpu='12' cpuset='7'/>
    <vcpupin vcpu='13' cpuset='17'/>
    <vcpupin vcpu='14' cpuset='8'/>
    <vcpupin vcpu='15' cpuset='18'/>
    <vcpupin vcpu='16' cpuset='9'/>
    <vcpupin vcpu='17' cpuset='19'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-5.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/dee16a86-c187-c737-7fb5-aed609f94cf7_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='9' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/cache/domains/Server 2.0/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/win10 8-6-2021 use this.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.160-1.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:2e:35:a4'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

 

IOMMU:

IOMMU group 0:[8086:9b33] 00:00.0 Host bridge: Intel Corporation Comet Lake-S 6c Host Bridge/DRAM Controller (rev 05)

 

IOMMU group 1:[8086:1901] 00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 05)

[1002:6810] 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Curacao XT / Trinidad XT [Radeon R7 370 / R9 270X/370X]

[1002:aab0] 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Oland/Hainan/Cape Verde/Pitcairn HDMI Audio [Radeon HD 7000 Series]

 

IOMMU group 2:[8086:9bc5] 00:02.0 VGA compatible controller: Intel Corporation CometLake-S GT2 [UHD Graphics 630] (rev 05)

 

IOMMU group 3:[8086:43ed] 00:14.0 USB controller: Intel Corporation Tiger Lake-H USB 3.2 Gen 2x1 xHCI Host Controller (rev 11)

Bus 001 Device 001 Port 1-0 ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 001 Device 002 Port 1-8 ID 0b05:19af ASUSTek Computer, Inc. AURA LED Controller

Bus 001 Device 003 Port 1-9 ID 05e3:0610 Genesys Logic, Inc. Hub

Bus 001 Device 004 Port 1-9.1 ID 0951:1666 Kingston Technology DataTraveler 100 G3/G4/SE9 G2/50

Bus 002 Device 001 Port 2-0 ID 1d6b:0003 Linux Foundation 3.0 root hub

[8086:43ef] 00:14.2 RAM memory: Intel Corporation Tiger Lake-H Shared SRAM (rev 11)

 

IOMMU group 4:[8086:43e8] 00:15.0 Serial bus controller: Intel Corporation Tiger Lake-H Serial IO I2C Controller #0 (rev 11)

 

IOMMU group 5:[8086:43e0] 00:16.0 Communication controller: Intel Corporation Tiger Lake-H Management Engine Interface (rev 11)

 

IOMMU group 6:[8086:43d2] 00:17.0 SATA controller: Intel Corporation Device 43d2 (rev 11)

[1:0:0:0] disk ATA CT1000BX500SSD1 030 /dev/sdb 1.00TB

[2:0:0:0] disk ATA WDC WD140EDGZ-11 0A85 /dev/sdc 14.0TB

[3:0:0:0] disk ATA WDC WD120EDAZ-11 0A81 /dev/sdd 12.0TB

[4:0:0:0] disk ATA WDC WD120EMFZ-11 0A81 /dev/sde 12.0TB

[6:0:0:0] disk ATA WDC WD20EZRX-00D 0A80 /dev/sdf 2.00TB

 

IOMMU group 7:[8086:43bc] 00:1c.0 PCI bridge: Intel Corporation Tiger Lake-H PCI Express Root Port #5 (rev 11)

 

IOMMU group 8:[8086:43b0] 00:1d.0 PCI bridge: Intel Corporation Tiger Lake-H PCI Express Root Port #9 (rev 11)

 

IOMMU group 9:[8086:4387] 00:1f.0 ISA bridge: Intel Corporation Device 4387 (rev 11)

[8086:f0c8] 00:1f.3 Audio device: Intel Corporation Device f0c8 (rev 11)

[8086:43a3] 00:1f.4 SMBus: Intel Corporation Tiger Lake-H SMBus Controller (rev 11)

[8086:43a4] 00:1f.5 Serial bus controller: Intel Corporation Tiger Lake-H SPI Controller (rev 11)

[8086:15fa] 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (14) I219-V (rev 11)

 

update:

I updated my bios Asus prime b560 plus to bios 1601. It allowed me to throughput he dedicated gpu without crashing unraid when it fails to boot. The igpu still causes unraid to crash.

The dedicated gpu log has the following error.

 

2022-12-07T23:27:22.026621Z qemu-system-x86_64: kvm_set_user_memory_region: KVM_SET_USER_MEMORY_REGION failed, slot=7, start=0xc804300000000000, size=0x10000000: Invalid argument
kvm_set_phys_mem: error registering slot: Invalid argument
2022-12-07 23:27:22.486+0000: shutting down, reason=crashed

unraid-diagnostics-20221207-1827.zip

the igpu when started shows the following.

 

 

river":"virtio-blk-pci","bus":"pci.0","addr":"0x5","drive":"libvirt-1-format","id":"virtio-disk2","bootindex":1,"write-cache":"on"}' \
-netdev tap,fd=37,id=hostnet0 \
-device '{"driver":"virtio-net","netdev":"hostnet0","id":"net0","mac":"52:54:00:2e:35:a4","bus":"pci.0","addr":"0x4"}' \
-chardev pty,id=charserial0 \
-device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
-chardev socket,id=charchannel0,fd=35,server=on,wait=off \
-device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
-device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \
-audiodev '{"id":"audio1","driver":"none"}' \
-device '{"driver":"vfio-pci","host":"0000:00:02.0","id":"hostdev0","bus":"pci.0","addr":"0x2"}' \
-device '{"driver":"vfio-pci","host":"0000:00:1f.3","id":"hostdev1","bus":"pci.0","addr":"0x6"}' \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/6 (label charserial0)
2022-12-07T23:30:31.701076Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:00:02.0","id":"hostdev0","bus":"pci.0","addr":"0x2"}: IGD device 0000:00:02.0 has no ROM, legacy mode disabled
2022-12-07T23:30:31.701300Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:00:1f.3","id":"hostdev1","bus":"pci.0","addr":"0x6"}: vfio 0000:00:1f.3: group 9 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.

I don't think you need to group your PCI Bridge in the IOMMU GPU group.  Try removing that and rebooting

On 12/7/2022 at 7:27 PM, gehenna said:

I don't think you need to group your PCI Bridge in the IOMMU GPU group.  Try removing that and rebooting

I set the PCIe ACS override to downstream and it split the group so the AMD gpu and sound is by itself in it's own IOMMU group now. However, it still does not start up. The server seems to hang after trying to load the rom and since no rom is given it skips. After that it does nothing.
 

also if I set it as multi-function or both it will not boot unless I use csm enable and that disables multi-monitor (setting that allows both igpu and dedicated gpu to be used at same time) which I assume is needed to passthrough igpu to the vm.

 

I was able to passthrough my dedicated gpu via uninstalling the vnc display driver and then passing through the dedicated gpu. That worked without problem now. Apparently there was conflict when having both.

 

For the igpu passthrough I gave up for now. Most likely IOMMU group issue but honestly could be others and as such plex was installed as docker.

 

I couldn't get plex to use my igpu even with it being seen in unraid but was solved finally when having to direct the device path. Put it in the plex preferences.xml

ex: HardwareDevicePath="/dev/dri/renderD129"

 

Hope this helps others who are having issues down the road.

 

Hi,

 

did you get yout vm working with the igpu? 

My new setup don't have a dgpu.

I tried multiple times creating a vm with igpu passthrouth, win 10, ubuntu. But everytimes the ui crashes and I cannot restart over ssh. 

Also installel Intel-GV-T (I have an Intel 10th Gen). Nothing helped...