Everything posted by InfraGerblin
-
VM w/ GPU Passthrough not outputting to display
I now see glorious color on my screen in the form of the fedora installation ui. Thank you so much for the help! Definitely going to need to diff your xml so I can see the changes between the original and the new.
-
VM w/ GPU Passthrough not outputting to display
Here is the output of the terminal command. Attached are the diagnostics file and the vbios file. root@:~# cat /proc/iomem 00000000-00000fff : Reserved 00001000-0009dfff : System RAM 0009e000-0009efff : Reserved 0009f000-0009ffff : System RAM 000a0000-000fffff : Reserved 000a0000-000bffff : PCI Bus 0000:00 000e0000-000e3fff : PCI Bus 0000:00 000e4000-000e7fff : PCI Bus 0000:00 000e8000-000ebfff : PCI Bus 0000:00 000ec000-000effff : PCI Bus 0000:00 000f0000-000fffff : PCI Bus 0000:00 000f0000-000fffff : System ROM 00100000-8a4de017 : System RAM 03000000-03a00816 : Kernel code 03c00000-03e4afff : Kernel rodata 04000000-04127f7f : Kernel data 04471000-045fffff : Kernel bss 8a4de018-8a4eb857 : System RAM 8a4eb858-9466a017 : System RAM 9466a018-94690457 : System RAM 94690458-98dfffff : System RAM 98e00000-98e00fff : Reserved 98e01000-9c1fbfff : System RAM 9c1fc000-9da63fff : Reserved 9da64000-9db93fff : ACPI Tables 9db94000-9dd93fff : ACPI Non-volatile Storage 9dd94000-9ec4dfff : Reserved 9ec4e000-9ec4efff : System RAM 9ec4f000-9fffffff : Reserved a0000000-dfffffff : PCI Bus 0000:00 a0000000-b1ffffff : PCI Bus 0000:02 a0000000-afffffff : 0000:02:00.0 a0000000-a18fffff : efifb b0000000-b1ffffff : 0000:02:00.0 b0000000-b1ffffff : vfio-pci b2000000-b30fffff : PCI Bus 0000:02 b2000000-b2ffffff : 0000:02:00.0 b2000000-b2ffffff : vfio-pci b3000000-b307ffff : 0000:02:00.0 b3080000-b3083fff : 0000:02:00.1 b3100000-b31fffff : 0000:00:1f.3 b3200000-b32fffff : PCI Bus 0000:05 b3200000-b3203fff : 0000:05:00.0 b3200000-b3203fff : nvme b3300000-b33fffff : PCI Bus 0000:04 b3300000-b330ffff : 0000:04:00.0 b3300000-b330ffff : r8169 b3310000-b3313fff : 0000:04:00.0 b3400000-b34fffff : PCI Bus 0000:01 b3400000-b3403fff : 0000:01:00.0 b3400000-b3403fff : nvme b3500000-b350ffff : 0000:00:14.0 b3500000-b350ffff : xhci-hcd b3510000-b3513fff : 0000:00:1f.3 b3514000-b3515fff : 0000:00:17.0 b3514000-b3515fff : ahci b3516000-b3517fff : 0000:00:14.2 b3518000-b35180ff : 0000:00:1f.4 b3519000-b35197ff : 0000:00:17.0 b3519000-b35197ff : ahci b351a000-b351a0ff : 0000:00:17.0 b351a000-b351a0ff : ahci b351b000-b351bfff : 0000:00:16.0 b351c000-b351cfff : 0000:00:14.2 b351d000-b351dfff : 0000:00:12.0 b351d000-b351dfff : Intel PCH thermal driver b351e000-b351efff : 0000:00:08.0 e0000000-efffffff : PCI MMCONFIG 0000 [bus 00-ff] e0000000-efffffff : Reserved e0000000-efffffff : pnp 00:07 fc800000-fe7fffff : PCI Bus 0000:00 fd000000-fd69ffff : pnp 00:04 fd6a0000-fd6affff : pnp 00:06 fd6b0000-fd6bffff : pnp 00:06 fd6c0000-fd6cffff : pnp 00:04 fd6d0000-fd6dffff : pnp 00:06 fd6e0000-fd6effff : pnp 00:06 fd6f0000-fdffffff : pnp 00:04 fe000000-fe010fff : Reserved fe010000-fe010fff : 0000:00:1f.5 fe038000-fe038fff : pnp 00:08 fe200000-fe7fffff : pnp 00:04 fec00000-fec00fff : Reserved fec00000-fec003ff : IOAPIC 0 fed00000-fed03fff : Reserved fed00000-fed003ff : HPET 0 fed00000-fed003ff : PNP0103:00 fed10000-fed17fff : pnp 00:07 fed18000-fed18fff : pnp 00:07 fed19000-fed19fff : pnp 00:07 fed20000-fed3ffff : pnp 00:07 fed40000-fed44fff : MSFT0101:00 fed45000-fed8ffff : pnp 00:07 fed91000-fed91fff : dmar0 fee00000-fee00fff : Local APIC fee00000-fee00fff : Reserved ff000000-ffffffff : Reserved ff000000-ffffffff : pnp 00:04 100000000-85dffffff : System RAM 85e000000-85fffffff : RAM buffer corsair-diagnostics-20210912-1839.zip vbios.zip
-
VM w/ GPU Passthrough not outputting to display
Hi @ghost82, thanks for responding. Long week so I didn't get a chance to revisit your suggestions until today. I moved the gpu to a lower slot and bound it to vfio at boot using the unraid gui (hoping this is what you were talking about for step 1). I also dumped vbios after a lot of googling on how to do so. Still, I am seeing the same blank screen as before. Here are the vfio logs followed by the logs for the fedora vm I am trying to spin up. Looking at the logs I see a lot of these "device or resource busy" warnings. Loading config from /boot/config/vfio-pci.cfg BIND=0000:02:00.0|10de:2206 0000:02:00.1|10de:1aef --- Processing 0000:02:00.0 10de:2206 Vendor:Device 10de:2206 found at 0000:02:00.0 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:02:00.0/iommu_group/devices/0000:02:00.0 /sys/bus/pci/devices/0000:02:00.0/iommu_group/devices/0000:02:00.1 Binding... Successfully bound the device 10de:2206 at 0000:02:00.0 to vfio-pci --- Processing 0000:02:00.1 10de:1aef Vendor:Device 10de:1aef found at 0000:02:00.1 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:02:00.1/iommu_group/devices/0000:02:00.0 /sys/bus/pci/devices/0000:02:00.1/iommu_group/devices/0000:02:00.1 Binding... 0000:02:00.0 already bound to vfio-pci 0000:02:00.1 already bound to vfio-pci Successfully bound the device 10de:1aef at 0000:02:00.1 to vfio-pci --- vfio-pci binding complete Devices listed in /sys/bus/pci/drivers/vfio-pci: lrwxrwxrwx 1 root root 0 Sep 12 02:51 0000:02:00.0 -> ../../../../devices/pci0000:00/0000:00:1b.4/0000:02:00.0 lrwxrwxrwx 1 root root 0 Sep 12 02:51 0000:02:00.1 -> ../../../../devices/pci0000:00/0000:00:1b.4/0000:02:00.1 ls -l /dev/vfio/ total 0 crw------- 1 root root 249, 0 Sep 12 02:51 13 crw-rw-rw- 1 root root 10, 196 Sep 12 02:51 vfio 2021-09-12T07:55:21.967169Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e31d8, 0x0,8) failed: Device or resource busy 2021-09-12T07:55:21.967176Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e41c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967183Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e41c8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967190Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e41d0, 0x0,8) failed: Device or resource busy 2021-09-12T07:55:21.967196Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e41d8, 0x0,8) failed: Device or resource busy 2021-09-12T07:55:21.967203Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e51c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967210Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e51c8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967216Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e51d0, 0x0,8) failed: Device or resource busy 2021-09-12T07:55:21.967223Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e51d8, 0x0,8) failed: Device or resource busy 2021-09-12T07:55:21.967230Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e61c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967237Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e61c8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967244Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e61d0, 0x0,8) failed: Device or resource busy 2021-09-12T07:55:21.967250Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e61d8, 0x0,8) failed: Device or resource busy 2021-09-12T07:55:21.967257Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e71c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967263Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e71c8, 0x98989800989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967270Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e71d0, 0x98989800989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967276Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e71d8, 0x0,8) failed: Device or resource busy 2021-09-12T07:55:21.967283Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e81c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967290Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e81c8, 0x98989800989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967297Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e81d0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967303Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e81d8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967310Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e91c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967317Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e91c8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967324Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e91d0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967330Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1e91d8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967338Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ea1c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967344Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ea1c8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967351Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ea1d0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967358Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ea1d8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967365Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1eb1c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967371Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1eb1c8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967378Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1eb1d0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967385Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1eb1d8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967392Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ec1c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967398Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ec1c8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967405Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ec1d0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967412Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ec1d8, 0x989898,8) failed: Device or resource busy 2021-09-12T07:55:21.967419Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ed1c0, 0x98989800000000,8) failed: Device or resource busy 2021-09-12T07:55:21.967425Z qemu-system-x86_64: vfio_region_write(0000:02:00.0:region1+0x1ed1c8, 0x989898,8) failed: Device or resource busy Any further suggestions are greatly appreciated!
-
VM w/ GPU Passthrough not outputting to display
Hi all! I am testing out unraid as a solution for storage / virtualization / tinkering. I am trying to get a fedora VM w/ my RTX 3080 to output via the displayport connection to my screen. I think something is happening because when my VM starts up and the monitor is off it will turn on as if something is being outputted. However, the screen stays black. When I use VNC instead of the GPU passthrough, I am able to see the fedora UI. The computer works fine when I am booting from bare metal. I did some troubleshooting already from stuff on this forum and from external guides and I realized that I hadn't set up IOMMU, but that should be squared away now. I seem to be pretty stuck on this issue though. Below are my PC specs and my VM xml along with the only suspect item in the logs I noticed. Happy to provide whatever other debug info is helpful and I appreciate all responses. Thanks! Sep 6 18:59:12 Corsair kernel: vfio-pci 0000:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0x8a65 Model: Custom M/B: Micro-Star International Co., Ltd. Z490-A PRO (MS-7C75) Version 1.0 - s/n: 07C7511_K71E628601 BIOS: American Megatrends Inc. Version 2.CO. Dated: 10/07/2020 CPU: Intel® Core™ i9-10850K CPU @ 3.60GHz HVM: Enabled IOMMU: Enabled Cache: 640 KiB, 2560 KiB, 20 MB Memory: 32 GiB DDR4 (max. installable capacity 64 GiB) Network: bond0: fault-tolerance (active-backup), mtu 1500 eth0: 1000 Mbps, full duplex, mtu 1500 Kernel: Linux 5.10.28-Unraid x86_64 <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Fedora</name> <uuid>41ae2ddd-97d3-7eff-a190-8242ffcc94c7</uuid> <metadata> <vmtemplate xmlns="unraid" name="Fedora" icon="fedora.png" os="fedora"/> </metadata> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='6'/> <vcpupin vcpu='1' cpuset='16'/> <vcpupin vcpu='2' cpuset='7'/> <vcpupin vcpu='3' cpuset='17'/> <vcpupin vcpu='4' cpuset='8'/> <vcpupin vcpu='5' cpuset='18'/> <vcpupin vcpu='6' cpuset='9'/> <vcpupin vcpu='7' cpuset='19'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-5.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/41ae2ddd-97d3-7eff-a190-8242ffcc94c7_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='4' threads='2'/> <cache mode='passthrough'/> </cpu> <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/Fedora/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/Fedora-Workstation-Live-x86_64-34-1.2.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <controller type='usb' index='0' model='nec-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <interface type='bridge'> <mac address='52:54:00:31:94:86'/> <source bridge='br0'/> <model type='virtio-net'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <rom file='/mnt/cache/isos/Gigabyte.RTX3080.10240.210608.rom'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc545'/> </source> <address type='usb' bus='0' port='2'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x046d'/> <product id='0xc547'/> </source> <address type='usb' bus='0' port='3'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x8087'/> <product id='0x0029'/> </source> <address type='usb' bus='0' port='4'/> </hostdev> <memballoon model='none'/> </devices> </domain>