PassTheSalt Posted December 7, 2022 Share Posted December 7, 2022 Ok, So i just built a new unraid machine, 13700k, and figured id try out a "gaming vm" to give my fiancé a computer to play some games with me.I have a 1660Ti in slot 1. I watch both of Spaceinvader one's videos on setting up windows VMs and his videos on gpu passthrough. I've been trying for the last few days trying to get this to work. So first I'll throw down what I know. The issue I'm having is the monitor wont display anything. The screen is just black and I can see its in standby. I do have the monitor plugged into a HDMI to VGA adapter, and into the HDMI port on the GPU. I tried setting my igpu as primary. I was able to get the GPU to pass through correctly ONCE on a windows 10 VM. bound the VFIO group and restarted and started the vm and it came up. I shut the vm down to pass through my USB devices and when i starte it back up, it stopped working again and even when i set my settings to be exactly the same as they were before, i couldn't ever get it to display anything. I'm going to start off with a fresh windows 10 VM My VM settings I've changed are:Machine : Q35-7.1Virtio drivers are 1.225-2Primary Vdisk is 100G VNC is my primary display. I can start the VM, Install windows, install drivers, all that works fine. I shut it down and go to add the GPU/sound card IOMMU is enabled in system info. IOMMU group 0: [8086:a703] 00:00.0 Host bridge: Intel Corporation Device a703 (rev 01) IOMMU group 1: [8086:a70d] 00:01.0 PCI bridge: Intel Corporation Device a70d (rev 01) IOMMU group 2: [8086:a780] 00:02.0 VGA compatible controller: Intel Corporation Raptor Lake-S UHD Graphics (rev 04) IOMMU group 3: [8086:7ae0] 00:14.0 USB controller: Intel Corporation Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller (rev 11) Bus 001 Device 001 Port 1-0 ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002 Port 1-2 ID 0930:6545 Toshiba Corp. Kingston DataTraveler 102/2.0 / HEMA Flash Drive 2 GB / PNY Attache 4GB Stick Bus 001 Device 003 Port 1-3 ID 051d:0002 American Power Conversion Uninterruptible Power Supply Bus 001 Device 004 Port 1-6 ID 174c:2074 ASMedia Technology Inc. ASM1074 High-Speed hub Bus 001 Device 005 Port 1-13 ID 26ce:01a2 ASRock LED Controller Bus 001 Device 006 Port 1-6.1 ID 1c4f:0034 SiGma Micro XM102K Optical Wheel Mouse Bus 001 Device 007 Port 1-6.2 ID 413c:2113 Dell Computer Corp. KB216 Wired Keyboard Bus 002 Device 001 Port 2-0 ID 1d6b:0003 Linux Foundation 3.0 root hub [8086:7aa7] 00:14.2 RAM memory: Intel Corporation Alder Lake-S PCH Shared SRAM (rev 11) IOMMU group 4: [8086:7acc] 00:15.0 Serial bus controller: Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #0 (rev 11) IOMMU group 5: [8086:7ae8] 00:16.0 Communication controller: Intel Corporation Alder Lake-S PCH HECI Controller #1 (rev 11) IOMMU group 6: [8086:7ae2] 00:17.0 SATA controller: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] (rev 11) [3:0:0:0] disk ATA ST18000NM000J-2T SN02 /dev/sdb 18.0TB [4:0:0:0] disk ATA ST12000VN0008-2Y SC60 /dev/sdc 12.0TB [5:0:0:0] disk ATA ST12000VN0008-2Y SC60 /dev/sdd 12.0TB [7:0:0:0] disk ATA Samsung SSD 870 2B6Q /dev/sde 500GB [8:0:0:0] disk ATA Samsung SSD 870 2B6Q /dev/sdf 500GB [6:0:0:0] cd/dvd HL-DT-ST BD-RE WH16NS60 1.00 /dev/sr0 34.8GB IOMMU group 7: [8086:7aba] 00:1c.0 PCI bridge: Intel Corporation Device 7aba (rev 11) IOMMU group 8: [8086:7a84] 00:1f.0 ISA bridge: Intel Corporation Z690 Chipset LPC/eSPI Controller (rev 11) [8086:7ad0] 00:1f.3 Audio device: Intel Corporation Alder Lake-S HD Audio Controller (rev 11) [8086:7aa3] 00:1f.4 SMBus: Intel Corporation Alder Lake-S PCH SMBus Controller (rev 11) [8086:7aa4] 00:1f.5 Serial bus controller: Intel Corporation Alder Lake-S PCH SPI Controller (rev 11) IOMMU group 9: [10de:2182] 01:00.0 VGA compatible controller: NVIDIA Corporation TU116 [GeForce GTX 1660 Ti] (rev a1) [10de:1aeb] 01:00.1 Audio device: NVIDIA Corporation TU116 High Definition Audio Controller (rev a1) [10de:1aec] 01:00.2 USB controller: NVIDIA Corporation TU116 USB 3.1 Host Controller (rev a1) Bus 003 Device 001 Port 3-0 ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 004 Device 001 Port 4-0 ID 1d6b:0003 Linux Foundation 3.0 root hub [10de:1aed] 01:00.3 Serial bus controller: NVIDIA Corporation TU116 USB Type-C UCSI Controller (rev a1) IOMMU group 10: [10ec:8125] 02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05) As you can see my IOMMU group is group 9 with 4 items the graphics, the audio, and two USB controllers? so... i edit the vm settings Graphics card : NVIDIA GeForce GTX 1660 Ti (01:00:0) Sound card : NVIDIA TU116 HD Audio Controller (01:00.1) I have USB Mouse and keyboard devices selected. I do not see the other 2 USB devices to pass through. When I try to start the VM i get this error internal error: qemu unexpectedly closed the monitor: 2022-11-29T21:25:57.303884Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:01:00.0","id":"hostdev0","bus":"pci.4","addr":"0x0"}: vfio 0000:01:00.0: group 9 is not viable Please ensure all devices within the iommu_group are bound to their vfio bus driver. So I go and bind the entire group to VFIO. IOMMU group 9: [10de:2182] 01:00.0 VGA compatible controller: NVIDIA Corporation TU116 [GeForce GTX 1660 Ti] (rev a1) [10de:1aeb] 01:00.1 Audio device: NVIDIA Corporation TU116 High Definition Audio Controller (rev a1) [10de:1aec] 01:00.2 USB controller: NVIDIA Corporation TU116 USB 3.1 Host Controller (rev a1) This controller is bound to vfio, connected USB devices are not visible. [10de:1aed] 01:00.3 Serial bus controller: NVIDIA Corporation TU116 USB Type-C UCSI Controller (rev a1) I start the VM and it boots. and it displays! Wow great! my mouse doesn't work, but the keyboard does!. 1660Ti shows in the device manager after awhile. so lets turn it off. and it still works? ____________________________________________________________________________________________________ Ok so everything I've written so far was written about two weeks ago. this time this VM worked fine, for quite awhile, even rebooting it a few times. but today i got a new mouse and added it to the vm and when I tried to start the vm, it didn't start again. the Display doesn't do anything and i don't even see it on parsec. Now, I'm not sure if its me changing the forwarded devices or what. I don't remember if i shutdown the vm yesterady or something else happened making it go down.I can't see anything in the log that tells me why its not starting. does anyone have any idea what's going on? Here is the log for the vm -blockdev '{"driver":"file","filename":"/mnt/user/isos/windows/Windows10.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \ -device '{"driver":"ide-cd","bus":"ide.0","drive":"libvirt-2-format","id":"sata0-0-0","bootindex":2}' \ -blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.225-2.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \ -device '{"driver":"ide-cd","bus":"ide.1","drive":"libvirt-1-format","id":"sata0-0-1"}' \ -netdev tap,fd=37,id=hostnet0 \ -device '{"driver":"virtio-net","netdev":"hostnet0","id":"net0","mac":"52:54:00:78:fe:a1","bus":"pci.1","addr":"0x0"}' \ -chardev pty,id=charserial0 \ -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \ -chardev socket,id=charchannel0,fd=35,server=on,wait=off \ -device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \ -device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -device '{"driver":"vfio-pci","host":"0000:01:00.0","id":"hostdev0","bus":"pci.4","addr":"0x0"}' \ -device '{"driver":"vfio-pci","host":"0000:01:00.1","id":"hostdev1","bus":"pci.5","addr":"0x0"}' \ -device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/013","id":"hostdev2","bus":"usb.0","port":"2"}' \ -device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/007","id":"hostdev3","bus":"usb.0","port":"3"}' \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on char device redirected to /dev/pts/15 (label charserial0) 2022-12-04T00:37:36.431102Z qemu-system-x86_64: libusb_release_interface: -4 [NO_DEVICE] 2022-12-05T21:04:23.878276Z qemu-system-x86_64: terminating on signal 15 from pid 3912 (/usr/sbin/libvirtd) 2022-12-05 21:04:25.517+0000: shutting down, reason=shutdown 2022-12-07 14:35:53.628+0000: starting up libvirt version: 8.7.0, qemu version: 7.1.0, kernel: 5.19.17-Unraid, hostname: Crusty LC_ALL=C \ PATH=/bin:/sbin:/usr/bin:/usr/sbin \ HOME='/var/lib/libvirt/qemu/domain-3-Windows 10' \ XDG_DATA_HOME='/var/lib/libvirt/qemu/domain-3-Windows 10/.local/share' \ XDG_CACHE_HOME='/var/lib/libvirt/qemu/domain-3-Windows 10/.cache' \ XDG_CONFIG_HOME='/var/lib/libvirt/qemu/domain-3-Windows 10/.config' \ /usr/local/sbin/qemu \ -name 'guest=Windows 10,debug-threads=on' \ -S \ -object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-3-Windows 10/master-key.aes"}' \ -blockdev '{"driver":"file","filename":"/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \ -blockdev '{"driver":"file","filename":"/etc/libvirt/qemu/nvram/84fc7bba-4f31-991e-1edd-8a33dab1f502_VARS-pure-efi.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \ -machine pc-q35-7.1,usb=off,dump-guest-core=off,mem-merge=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \ -accel kvm \ -cpu host,migratable=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=none,host-cache-info=on,l3-cache=off \ -m 16384 \ -object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":17179869184}' \ -overcommit mem-lock=off \ -smp 8,sockets=1,dies=1,cores=4,threads=2 \ -uuid 84fc7bba-4f31-991e-1edd-8a33dab1f502 \ -display none \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=36,server=on,wait=off \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=localtime \ -no-hpet \ -no-shutdown \ -boot strict=on \ -device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \ -device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \ -device '{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}' \ -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \ -device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \ -device '{"driver":"ich9-usb-ehci1","id":"usb","bus":"pcie.0","addr":"0x7.0x7"}' \ -device '{"driver":"ich9-usb-uhci1","masterbus":"usb.0","firstport":0,"bus":"pcie.0","multifunction":true,"addr":"0x7"}' \ -device '{"driver":"ich9-usb-uhci2","masterbus":"usb.0","firstport":2,"bus":"pcie.0","multifunction":true,"addr":"0x7.0x1"}' \ -device '{"driver":"ich9-usb-uhci3","masterbus":"usb.0","firstport":4,"bus":"pcie.0","addr":"0x7.0x2"}' \ -device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.2","addr":"0x0"}' \ -blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 10/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.3","addr":"0x0","drive":"libvirt-3-format","id":"virtio-disk2","bootindex":1,"write-cache":"on"}' \ -blockdev '{"driver":"file","filename":"/mnt/user/isos/windows/Windows10.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \ -device '{"driver":"ide-cd","bus":"ide.0","drive":"libvirt-2-format","id":"sata0-0-0","bootindex":2}' \ -blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.225-2.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \ -device '{"driver":"ide-cd","bus":"ide.1","drive":"libvirt-1-format","id":"sata0-0-1"}' \ -netdev tap,fd=37,id=hostnet0 \ -device '{"driver":"virtio-net","netdev":"hostnet0","id":"net0","mac":"52:54:00:78:fe:a1","bus":"pci.1","addr":"0x0"}' \ -chardev pty,id=charserial0 \ -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \ -chardev socket,id=charchannel0,fd=35,server=on,wait=off \ -device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \ -device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -device '{"driver":"vfio-pci","host":"0000:01:00.0","id":"hostdev0","bus":"pci.4","addr":"0x0"}' \ -device '{"driver":"vfio-pci","host":"0000:01:00.1","id":"hostdev1","bus":"pci.5","addr":"0x0"}' \ -device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/018","id":"hostdev2","bus":"usb.0","port":"2"}' \ -device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/007","id":"hostdev3","bus":"usb.0","port":"3"}' \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on char device redirected to /dev/pts/17 (label charserial0) crusty-diagnostics-20221207-1356.zip Quote Link to comment
PassTheSalt Posted December 12, 2022 Author Share Posted December 12, 2022 bump? no one has any idea? Quote Link to comment
PassTheSalt Posted December 14, 2022 Author Share Posted December 14, 2022 (edited) Ok, so I figured this has to be some kind of Unraid Bug. I made a copy of my Vdisk and made a new VM, Set it up exactly the same way, and pointed it at the Vdisk2.img copy i made. And it booted just fine and works!. I'm gonna do some testing to see if i can make it fail and what makes it fail. Edited December 14, 2022 by PassTheSalt Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.