d13f Posted October 8, 2020 Share Posted October 8, 2020 (edited) I've watched SpaceInvaders videos and followed them all the way, but I am still having issues with passing through my GPU (GTX 1050Ti) to my W10 VM. The most success I've had is booting into it, the VM seeing my 1050ti, but it being disabled due to code 43. Now I can't even get into it with RDP (the VM works if I use VNC though.) It simply shows as "started" when I turn it on but there is nothing I can do, it must be stuck before it reaches W10 since I can't RDP into it. My Unraid specs: Mobo: Gigabyte 7PESH2 Mobo Bios: Gigabyte Version R17 RAM: 48GB CPU: Dual Xeon E5-2650L HVM: Enabled IOMMU: Enabled Unraid Version: 6.8.3 My W10 VM XML: <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>CCTV4K</name> <uuid>00965e63-0010-1e0f-db1b-8e0a4d9d5bae</uuid> <metadata> <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/> </metadata> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='12'/> <vcpupin vcpu='1' cpuset='28'/> <vcpupin vcpu='2' cpuset='13'/> <vcpupin vcpu='3' cpuset='29'/> <vcpupin vcpu='4' cpuset='14'/> <vcpupin vcpu='5' cpuset='30'/> <vcpupin vcpu='6' cpuset='15'/> <vcpupin vcpu='7' cpuset='31'/> <emulatorpin cpuset='15'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-4.2'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader> <nvram>/etc/libvirt/qemu/nvram/00965e63-0010-1e0f-db1b-8e0a4d9d5bae_VARS-pure-efi.fd</nvram> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='4' threads='2'/> <cache mode='passthrough'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/CCTV4K/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/isos/virtio-win-0.1.173-2.iso'/> <target dev='hdb' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='usb' index='0' model='nec-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:96:62:3d'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </source> <rom file='/mnt/user/isos/1050ti.rom'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/> </hostdev> <memballoon model='none'/> </devices> </domain> Here is my log to my VM when I try to boot it with the GPU inside: uuid 00965e63-0010-1e0f-db1b-8e0a4d9d5bae \ -display none \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=33,server,nowait \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=localtime,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-hpet \ -no-shutdown \ -boot strict=on \ -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \ -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \ -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \ -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \ -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \ -device nec-usb-xhci,p2=15,p3=15,id=usb,bus=pcie.0,addr=0x7 \ -device virtio-serial-pci,id=virtio-serial0,bus=pci.2,addr=0x0 \ -blockdev '{"driver":"file","filename":"/mnt/user/domains/CCTV4K/vdisk1.img","node-name":"libvirt-2-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-2-storage"}' \ -device virtio-blk-pci,scsi=off,bus=pci.3,addr=0x0,drive=libvirt-2-format,id=virtio-disk2,bootindex=1,write-cache=on \ -blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.173-2.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \ -device ide-cd,bus=ide.1,drive=libvirt-1-format,id=sata0-0-1 \ -netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 \ -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:96:62:3d,bus=pci.1,addr=0x0 \ -chardev pty,id=charserial0 \ -device isa-serial,chardev=charserial0,id=serial0 \ -chardev socket,id=charchannel0,fd=37,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \ -device usb-tablet,id=input0,bus=usb.0,port=1 \ -device vfio-pci,host=0000:03:00.0,id=hostdev0,bus=pci.4,multifunction=on,addr=0x0,romfile=/mnt/user/isos/1050ti.rom \ -device vfio-pci,host=0000:03:00.1,id=hostdev1,bus=pci.4,addr=0x0.0x1 \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2020-10-06 20:08:37.291+0000: Domain id=1 is tainted: high-privileges 2020-10-06 20:08:37.291+0000: Domain id=1 is tainted: host-cpu char device redirected to /dev/pts/0 (label charserial0) 2020-10-06T20:08:44.677245Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3 2020-10-06T20:08:44.681165Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3 I think the problem comes from the last 2 lines of that log: qemu-system-x86_64: vfio: Unable to power on device, stuck in D3 qemu-system-x86_64: vfio: Unable to power on device, stuck in D3 It could be the gpu and vfio-pci.cfg issues, but I have no clue on how to diagnose or fix this. A few other things I should mention: I dumped the exact bios of the 1050ti from another PC. Did not download it from a website. (And it is edited to remove the nvidia heading like SpaceInvader told me) I only have 1 GPU in my system. (Thinking this could be the issue) My mobo has onboard graphics via VGA which is fine for unraid console. I want to passthrough the 1050ti GPU just for my VM. Don't need it for dockers or anything else (maybe Plex in the future *if possible using 1 GPU* but not worried about that at all right now). Unfortunately, I have only one available slot in my mobo for the GPU, because the top PCIe slot is occupied with an NVME adapter that I use for cache. So I cannot relocate the GPU and it is stuck in the third/fourth PCIe slot which is a 3.0 8x slot (which should be fine right?) Already have a dummy HDMI plugged in. I added to my Syslinux config: append video=efifb:off isolcpus=2-15,18-31 vfio-pci.ids=10de:1c82,10de:0fb9 (my 1050ti) pcie_acs_override=downstream,multifunction vfio_iommu_type1.allow_unsafe_interrupts=1 I have already tried the VM with i440fx, but that didn't work. HyperV is also off. I am booting in Legacy mode. PCIe ACS Override is enabled. Here are my IOMMU groups. I have to Force Stop the VM to turn it off, then when I try to start it again, it spits out Execution error internal error: Unknown PCI header type '127' for device '0000:03:00.0'. I reboot my server after this and am back to step 1. I have been reading that 'Stuck in D3' is possibly a BIOS issue with the mobo, and it happens often with AMD chips, but I think it's unlikely that it is a BIOS issue and I am not running an AMD chip. I appreciate any & all help, thank you guys. Edited October 8, 2020 by d13f Quote Link to comment
Spazhead Posted October 10, 2020 Share Posted October 10, 2020 correct me if i'm wrong but it could be this line <rom file='/mnt/user/isos/1050ti.rom'/> should be <rom bar='on' file='/mnt/user/isos/1050ti.rom'/> 1 Quote Link to comment
d13f Posted October 12, 2020 Author Share Posted October 12, 2020 On 10/9/2020 at 10:40 PM, Spazhead said: correct me if i'm wrong but it could be this line <rom file='/mnt/user/isos/1050ti.rom'/> should be <rom bar='on' file='/mnt/user/isos/1050ti.rom'/> Oh my goodness, thank you sooooo much!!! I had no idea this was causing it and after your fix, it seems to run perfectly! I greatly appreciate it man seriously, you have no idea how much I struggled with this!! Thank you!! 1 Quote Link to comment
bigbangus Posted October 13, 2020 Share Posted October 13, 2020 On 10/10/2020 at 1:40 AM, Spazhead said: correct me if i'm wrong but it could be this line <rom file='/mnt/user/isos/1050ti.rom'/> should be <rom bar='on' file='/mnt/user/isos/1050ti.rom'/> Curious what this does? I'm currently running with just <rom file='...> and I don't have any issues. Is the bar='on' necessary in all cases and what is it for? Quote Link to comment
Spazhead Posted October 13, 2020 Share Posted October 13, 2020 i believe it just forces it to load rom file, otherwise it will determine if it needs it or not. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.