Kodon

Have you tried deleting the VM and making a new config using the old vDisk?

Maybe your GPU doesn't start up? I'd attach a physical monitor and see what it does. With my 4090 and 4060 I need a (dummy) load so that they fire up.

Both required edits for the RTX 4090 are overwritten when editing the VM in GUI mode On top: <domain type='kvm' id='1'> Needs to be changed to / stay at <domain type='kvm' id='1' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> Before </domain> This needs to be added / stickied <qemu:commandline> <qemu:arg value='-fw_cfg'/> <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/> </qemu:commandline>

Sure, I have everything else at default though so nothing interesting to see. I wish the XML settings would stick and not vanish every time I edit the VM... <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm' id='1' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>Win11</name> <uuid>f7db8eca-8847-8146-dcc3-a645d3c46ea5</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/> </metadata> <memory unit='KiB'>16777216</memory> <currentMemory unit='KiB'>16777216</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>16</vcpu> <cputune> <vcpupin vcpu='0' cpuset='8'/> <vcpupin vcpu='1' cpuset='9'/> <vcpupin vcpu='2' cpuset='10'/> <vcpupin vcpu='3' cpuset='11'/> <vcpupin vcpu='4' cpuset='12'/> <vcpupin vcpu='5' cpuset='13'/> <vcpupin vcpu='6' cpuset='14'/> <vcpupin vcpu='7' cpuset='15'/> <vcpupin vcpu='8' cpuset='16'/> <vcpupin vcpu='9' cpuset='17'/> <vcpupin vcpu='10' cpuset='18'/> <vcpupin vcpu='11' cpuset='19'/> <vcpupin vcpu='12' cpuset='20'/> <vcpupin vcpu='13' cpuset='21'/> <vcpupin vcpu='14' cpuset='22'/> <vcpupin vcpu='15' cpuset='23'/> </cputune> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-q35-7.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader> <nvram>/etc/libvirt/qemu/nvram/f7db8eca-8847-8146-dcc3-a645d3c46ea5_VARS-pure-efi-tpm.fd</nvram> </os> <features> <acpi/> <apic/> <hyperv mode='custom'> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='8' threads='2'/> <cache mode='passthrough'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='writeback'/> <source file='/mnt/user/domains/Win11/vdisk1.img' index='2'/> <backingStore/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk2'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='writeback'/> <source file='/mnt/user/domains/Win11/vdisk2.img' index='1'/> <backingStore/> <target dev='hdd' bus='virtio'/> <alias name='virtio-disk3'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'> <alias name='pcie.0'/> </controller> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x8'/> <alias name='pci.1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x9'/> <alias name='pci.2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0xa'/> <alias name='pci.3'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0xb'/> <alias name='pci.4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0xc'/> <alias name='pci.5'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0xd'/> <alias name='pci.6'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/> </controller> <controller type='pci' index='7' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='7' port='0xe'/> <alias name='pci.7'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <alias name='ide'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <interface type='bridge'> <mac address='52:54:00:00:58:7c'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='virtio-net'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/0'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/0'> <source path='/dev/pts/0'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-Win11/org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'> <alias name='input0'/> </input> <input type='keyboard' bus='ps2'> <alias name='input1'/> </input> <tpm model='tpm-tis'> <backend type='emulator' version='2.0' persistent_state='yes'/> <alias name='tpm0'/> </tpm> <audio id='1' type='none'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </hostdev> <memballoon model='none'/> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+100</label> <imagelabel>+0:+100</imagelabel> </seclabel> <qemu:commandline> <qemu:arg value='-fw_cfg'/> <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/> </qemu:commandline> </domain>

The good new is: I got it working! The bad news is: I have no idea why. I tested if I can safely reproduce the result and yes, it is consistent! What I did: - Flash BIOS to older version (I updated again later, so far no difference so might not have helped at all, but got me started with fresh, stock settings) - Completely started from scratch with Unraid, reflash USB, deleted all drive partitions, only kept key file - Only add one drive to the array at first - Did not install any apps or plugins The above might or might not have helped, I don't know. - Activate IOMMU group for GPU and respective Audio - Setup new VM with Q35, vDisk qcow2, Virtual GPU - Install Windows, install all VirtIO drivers and update OS - Shutdown VM - Delete VM - Setup VM again, choosing the already set up vDisk, Q35 and this time with the physical GPU and respective audio device - Connected a physical monitor, keyboard and mouse, attaching them to the VM - Switching to XML view and added <domain type='kvm' id='1' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> And just before </domain> tag: <qemu:commandline> <qemu:arg value='-fw_cfg'/> <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/> </qemu:commandline> After starting the VM everything worked. I have no idea why but it does. Deleting the VM after having used a virtual GPU might have had the biggest impact. Important: I need to use a monitor or a HDMI dummy dongle to start up my card! Today I added a second GPU (4060ti) and the same steps brought the same success. But I didn't need the XML part. TL;DR for what I think is absolutely necessary for the 4090: - Activate IOMMU group - HDMI Dummy dongle or physical monitor - XML additions I hope this helps someone who runs into the same trouble!

It's sadly not a simple driver issue. I tried your suggestion to set up a Linux VM. This also does not work. I used the newest Ubuntu and installed the Nvidia drivers beforehand, but also no luck. After switching the card to the physical one I don't get any output at all. Then I had the idea to try the physical card as a secondary GPU... This leads to this message in both the Linux and Windows VNC viewer: Guest has not initialized the display (yet) So also no luck there. Deleting and recreating the VM also doesn't change anything for the better. VM Log shows: qxl_send_events: spice-server bug: guest stopped, ignoring Which also didn't bring me closer to a solution.

Thanks for the input. I am using a "real" monitor, so no kickstart required to trick the GPU to fire up. Here are my settings.

I took this to start over from scratch. I reset the BIOS and reformatted the Unraid USB to get rid of all settings. Leaving everything at default and "ticking the boxes" and... still not working :c vfio-pci Log VM Log Unraid Log I really don't know what could be wrong.

Heya! Thanks for the input. I changed the setting in the syslinux config. But it didn't change anything... New Log entries: May 30 12:24:50 ArcticTower kernel: Command line: BOOT_IMAGE=/bzimage vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream,multifunction video=efifb:off initrd=/bzroot [...] May 30 12:24:50 ArcticTower kernel: pci 0000:01:00.0: BAR 1: assigned to efifb

Hey Unraid magicians! I can not get my Gigabyte 4090 Gaming OC passed through to my Windows 11 VM no matter what I try. For the last two days I have been trying everything I could find in what feels like all possible combinations. Any help is greatly appreciated! So far I have tried: - Dumping the vbios and adding it as original or hex edited - Disabling rBAR in Bios - Binding to VFIO - adding this to the VM config: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> [...] <qemu:commandline> <qemu:arg value='-fw_cfg'/> <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/> </qemu:commandline> The display I attached just stays blank. When I connect via TeamViewer I connect to the tinyest resolution possible and can't find the GPU in the device manager, so it's not a driver issue in the VM. I'm really losing my sanity on this. I have been Using a T1000 card without any issues before. Thanks for any input! arctictower-diagnostics-20230528-2042.zip