GPU not recognized on passthrough

Elmojo · March 17, 2023

Hi all,

So I have a bit of an odd issue. I'm trying to pass my nvidia GT 710 to an existing Win10 VM. I'm on unRAID 6.11.5.

I'm generally following this guide, along with some additional tips by SpaceInvaderOne, and some other things I've found here and there.

The VM itself is running just fine, and I can RDP into it with no issues.

The trouble is that the VM doesn't see the GPU at all.

I'm unable to find a vBIOS for my exact card (It's a Zotac GT 710 1GB silent). I've tried SIO's script to dump the vBIOS from unRAID directly, but it just throws an error, even after stubbing through the System Devices tab. I don't know what else to try on that front, so I'm having to try it without a vBIOS file.

My XML is posted below, and I've added “video=efifb:off” to my syslinux.cfg.

When I boot the VM, it all works just fine, except that Device Manager doesn't show any nvidia hardware, just the regular virtual display device.

What else could it be? I'm at a loss here.

VM Device Manager attached as well.

XML:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='4'>
  <name>Sandbox (W10 )</name>
  <uuid>6eee4b38-2964-616c-20e2-b7ce678ba076</uuid>
  <description>Windows 10 isolated Sandbox</description>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>12</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='16'/>
    <vcpupin vcpu='1' cpuset='52'/>
    <vcpupin vcpu='2' cpuset='17'/>
    <vcpupin vcpu='3' cpuset='53'/>
    <vcpupin vcpu='4' cpuset='18'/>
    <vcpupin vcpu='5' cpuset='54'/>
    <vcpupin vcpu='6' cpuset='19'/>
    <vcpupin vcpu='7' cpuset='55'/>
    <vcpupin vcpu='8' cpuset='20'/>
    <vcpupin vcpu='9' cpuset='56'/>
    <vcpupin vcpu='10' cpuset='21'/>
    <vcpupin vcpu='11' cpuset='57'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-7.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/6eee4b38-2964-616c-20e2-b7ce678ba076_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='6' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
      <source file='/mnt/cache/domains/Win10 Sandbox/vdisk1.img' index='1'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='nec-xhci' ports='15'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:3c:3f:ce'/>
      <source bridge='br0'/>
      <target dev='vnet2'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-4-Sandbox (W10 )/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x1'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

SimonF · March 17, 2023

36 minutes ago, Elmojo said:

Hi all,

So I have a bit of an odd issue. I'm trying to pass my nvidia GT 710 to an existing Win10 VM. I'm on unRAID 6.11.5.

I'm generally following this guide, along with some additional tips by SpaceInvaderOne, and some other things I've found here and there.

The VM itself is running just fine, and I can RDP into it with no issues.

The trouble is that the VM doesn't see the GPU at all.

I'm unable to find a vBIOS for my exact card (It's a Zotac GT 710 1GB silent). I've tried SIO's script to dump the vBIOS from unRAID directly, but it just throws an error, even after stubbing through the System Devices tab. I don't know what else to try on that front, so I'm having to try it without a vBIOS file.

My XML is posted below, and I've added “video=efifb:off” to my syslinux.cfg.

When I boot the VM, it all works just fine, except that Device Manager doesn't show any nvidia hardware, just the regular virtual display device.

What else could it be? I'm at a loss here.

VM Device Manager attached as well.

XML:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='4'>
  <name>Sandbox (W10 )</name>
  <uuid>6eee4b38-2964-616c-20e2-b7ce678ba076</uuid>
  <description>Windows 10 isolated Sandbox</description>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>12</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='16'/>
    <vcpupin vcpu='1' cpuset='52'/>
    <vcpupin vcpu='2' cpuset='17'/>
    <vcpupin vcpu='3' cpuset='53'/>
    <vcpupin vcpu='4' cpuset='18'/>
    <vcpupin vcpu='5' cpuset='54'/>
    <vcpupin vcpu='6' cpuset='19'/>
    <vcpupin vcpu='7' cpuset='55'/>
    <vcpupin vcpu='8' cpuset='20'/>
    <vcpupin vcpu='9' cpuset='56'/>
    <vcpupin vcpu='10' cpuset='21'/>
    <vcpupin vcpu='11' cpuset='57'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-7.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/6eee4b38-2964-616c-20e2-b7ce678ba076_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='6' threads='2'/>
    <cache mode='passthrough'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
      <source file='/mnt/cache/domains/Win10 Sandbox/vdisk1.img' index='1'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='nec-xhci' ports='15'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:3c:3f:ce'/>
      <source bridge='br0'/>
      <target dev='vnet2'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/1'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-4-Sandbox (W10 )/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x1'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

Have you tried binding card to vfio via system devices? Any errors in the VM log? post diagnostics.

Elmojo · March 17, 2023

13 minutes ago, SimonF said:

but it just throws an error, even after stubbing through the System Devices tab

I have.

When I try to view the VM logs, I just get a "502 Bad Gateway" page. It's always done this, so I assumed it was just an unRAID bug, since this happens for any of my VMs.

Diagnostics attached.

Thanks!

tower-diagnostics-20230317-1323.zip

SimonF · March 17, 2023

Cannot see any thing jumping out in the logs. But will keep looking.

Elmojo · March 18, 2023

3 hours ago, SimonF said:

Cannot see any thing jumping out in the logs. But will keep looking.

Ok, thanks. At least I'm not losing my mind here. lol

Elmojo · March 19, 2023

Any new thoughts on this? @SimonF or anyone else? I'm at a loss here, and I really need to get this working so I can determine if this might be a viable option as a remote workstation for our office. My boss is waiting for an answer...

SimonF · March 19, 2023

16 minutes ago, Elmojo said:

Any new thoughts on this? @SimonF or anyone else? I'm at a loss here, and I really need to get this working so I can determine if this might be a viable option as a remote workstation for our office. My boss is waiting for an answer...

Just looking at the XML, you have set multifunction on the first part, but you have not changed the second part to be the same bus you have 05 not 04. so that needs to be updated or remove multifunction-='on'

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x1'/>
    </hostdev>

Elmojo · March 19, 2023

53 minutes ago, SimonF said:

ust looking at the XML, you have set multifunction on the first part, but you have not changed the second part to be the same bus you have 05 not 04. so that needs to be updated or remove multifunction-='on'

Thanks! Not sure how I missed that. 🤦‍♂️

Unfortunately, that made no difference. It's still not showing either the GPU or sound in Device Manager.

EDIT: I also plugged in an HDMI dummy plug that just came in the mail to see if that helped, but no change. I had previously had it connected to a physical monitor. Neither made any difference.

Edited March 19, 2023 by Elmojo
added info

SimonF · March 19, 2023

48 minutes ago, Elmojo said:

Thanks! Not sure how I missed that. 🤦‍♂️

Unfortunately, that made no difference. It's still not showing either the GPU or sound in Device Manager.

EDIT: I also plugged in an HDMI dummy plug that just came in the mail to see if that helped, but no change. I had previously had it connected to a physical monitor. Neither made any difference.

I would remove the XML lines for the gpu and just trying adding GPU back in + sound card via the GUI and dont add the Multifunction and see if that works.

Edited March 19, 2023 by SimonF

Elmojo · March 19, 2023

44 minutes ago, SimonF said:

I would remove the XML lines for the gpu and just trying adding GPU back in + sound card via the GUI and dont add the Multifunction and see if that works.

I believe I did that once before, but it can't hurt to try again, thanks. Standby...

Elmojo · March 19, 2023

So I tried something....

I attached the GPU to a different VM entirely, a Win11 machine. I used your suggested method above, of only selecting it via the GUI.

It's still not recognized at all via Device manager in the Win11 machine.

This is making me wonder if perhaps the card itself might be faulty? It was sitting in my box of PCIe cards, so it's possible that it's just no good. I'll try a different card in the next day or so and report back. It would be silly if this whole thing was faulty hardware! 🙄

The other thing is that I can't dump the vBOIS using the user script from SIO. I get an error, even after stubbing. This kinda leads me to believe that hardware failure might be the culprit indeed....

SimonF · March 19, 2023

4 minutes ago, Elmojo said:

So I tried something....

I attached the GPU to a different VM entirely, a Win11 machine. I used your suggested method above, of only selecting it via the GUI.

It's still not recognized at all via Device manager in the Win11 machine.

This is making me wonder if perhaps the card itself might be faulty? It was sitting in my box of PCIe cards, so it's possible that it's just no good. I'll try a different card in the next day or so and report back. It would be silly if this whole thing was faulty hardware! 🙄

The other thing is that I can't dump the vBOIS using the user script from SIO. I get an error, even after stubbing. This kinda leads me to believe that hardware failure might be the culprit indeed....

Have you tried SeaBIOS instead of OVMF?

Elmojo · March 19, 2023

25 minutes ago, SimonF said:

Have you tried SeaBIOS instead of OVMF?

I haven't, because everything I've read says that SeaBIOS isn't suitable for PCIe devices. I guess it wouldn't hurt to try...

Elmojo · March 19, 2023

I can't select SeaBIOS now, I guess because the VM is already created? 🤷‍♂️

SimonF · March 19, 2023

24 minutes ago, Elmojo said:

I can't select SeaBIOS now, I guess because the VM is already created? 🤷‍♂️

Yes you cannot change are you able to build a test vm?

Elmojo · March 20, 2023

I feel stupid. I've created a new VM, using SeaBIOS. I attached the GPU and started it, but now I have no way to connect to the VM to view it, as I don't know the IP or credentials for RDP, and there is no option for VNC obviously. How do I connect to this machine? 🤨

EDIT: So I edited the VM to remove the GPU temporarily and switched back to the virtual VNC driver, so I could get the IP and such. When I start it, I get "Booting from Hard Disk.... No bootable device."

It seems that it doesn't like that BIOS switch.

Edited March 20, 2023 by Elmojo
update

Elmojo · March 20, 2023

So my next option I guess is to start a new VM from scratch (I was reusing the disk images on that last one), but I'm still not sure how I'll connect to it after creation, if I'm passing through the video from the GPU, assuming it even works. Do I need to use virtual hardware at first, then switch it over to the physical GPU after things are set up? That strikes me as being the same boat I'm in now, and not really testing any new conditions. I'm sure I'm missing something....

JonathanM · March 21, 2023

If you are passing through a physical GPU it's assumed you will have a monitor connected to that GPU to see the output. Some cards even require some sort of monitor to be detected to even work at all, in those cases if you are using some flavor of remote desktop software to access it you still need a monitor or a monitor emulation (dummy) plug.

Elmojo · March 21, 2023

Thanks @JonathanM. As noted previously, I have both a dummy plug, and a physical monitor installed, and have since the beginning.

Whatever is going on, it's not that.

Elmojo · April 2, 2023

Any further thoughts on this?

I'd really like to get this working if possible. I've tried all as noted above, but I'm kinda stuck...

Any assistance appreciated. If I've failed to try something that's been suggested, please point it out and I'll get right on it!

SimonF · April 2, 2023

5 hours ago, Elmojo said:

Any further thoughts on this?

I'd really like to get this working if possible. I've tried all as noted above, but I'm kinda stuck...

Any assistance appreciated. If I've failed to try something that's been suggested, please point it out and I'll get right on it!

Do you have both a VNC and physical GPU configured? You need both to boot so you can load drivers.

Elmojo · April 2, 2023

5 hours ago, SimonF said:

Do you have both a VNC and physical GPU configured? You need both to boot so you can load drivers

Okaay... I'm not sure how I'd do that. I've never seen that mentioned in any of the tutorials or videos about passthrough, and I only have the option to select VNC or physical GPU when creating a VM. Do you have any info about how to proceed? This may be the whole issue...

SimonF · April 2, 2023

46 minutes ago, Elmojo said:

Okaay... I'm not sure how I'd do that. I've never seen that mentioned in any of the tutorials or videos about passthrough, and I only have the option to select VNC or physical GPU when creating a VM. Do you have any info about how to proceed? This may be the whole issue...

In the templete leave the first gpu as virtual. Click on the little green circle with + and add another

Elmojo · April 3, 2023

Ok, awesome! I'm trying that now.... Bummer, no-go.

I created a brand new VM with the following specs:

4 logical CPUs

4GB RAM

Machine: Q35-7.1

SeaBIOS

Hyper-V: Yes

USB: 2.0 (EHCI)

Graphics: Virtual (VNC)

Graphics2: GT710 GPU (no BIOS file)

Sound: GK208 HDMI (on GPU)

All of this was selected in the GUI.

I fired up the VM, installed Windows, and once I got to the desktop, opened Device Manager.

It shows only the "Microsoft Basic Display Adapter". There's no mention of the nVidia GPU at all, just like before.

The dummy plug and the physical monitor are still connected to the GPU.

I'm stumped.

ghost82 · April 3, 2023

Hi, try these steps:

General considerations:

1. first you have to pass a vbios for the gpu, because, assuming nothing changed from the logs you attached, the GT 710 is set as boot vga by the system:

Mar 17 12:15:50 Tower kernel: pci 0000:04:00.0: vgaarb: setting as boot VGA device

2. I would prefer ovmf (uefi) instead of seabios, so you need to unbind efifb; you already modified the syslinux config, so nothing to do here:

Mar 17 12:15:50 Tower kernel: Command line: BOOT_IMAGE=/bzimage video=efifb:off initrd=/bzroot

3. If I remember well the GT 710 may require unsafe interrupts to be enabled, just do it from the unraid gui

-------------------

Steps:
1. Dump your vbios file (it's highly the preferred way, but if you have no way to dump it go to step 2 and try to use the vbios file attached in the next post, I downloaded and modified one with the same device id (10DE 128B) and same subsystem id (19DA 6326) of your gpu, so I'm confident it should work)

Open an unraid terminal and type the following commands, after each line press enter:

echo "0000:04:00.0" > /sys/bus/pci/drivers/vfio-pci/unbind
cd /sys/bus/pci/devices/0000:04:00.0/
echo 1 > rom
cat rom > /mnt/user/yoursharename/gt710rom.dump
echo 0 > rom
echo "0000:04:00.0" > /sys/bus/pci/drivers/vfio-pci/bind

Here I'm assuming the address of the vga, 04:00.0, didn't change from your latest logs

First command could return a 'no such device', if it is, nothing wrong, just proceed

On the fourth command change the target directory to point to a share you have on your system

Last command only if you did not receive an error ('no such device') for the first command

Since the gpu is set as boot vga you could not be able to dump the gpu rom easily, in this case you have 2 choices:
a) boot windows bare metal and dump the rom with gpu-z, then strip with an hex editor the nvdia flash header (preferred)

b) download a rom from techpowerup and strip with an hex editor the nvdia flash header (not preferred)

2. Double check the syslinux config to make sure that video=efifb:off is still there

3. Enable unsafe interrupts for vfio in unraid gui: Settings -> VM -> change "VFIO allow unsafe interrupts" to Yes

4. Double check both audio and video are attached to vfio at boot (you already set them, so there should nothing to do here)

5. Reboot unraid

6. Create a new windows vm with ovmf, set the gpu as vnc, do not pass the gpu yet

7. Install windows, finally install a vnc server inside the vm, like realvnc or similar, or if it's available in your windows distro enable remote desktop; make sure the vm has internet access, change the ip address of the vm to a static ip so it will be easier to connect to it later. Download the nvidia webdrivers for your gt 710 and save them somewhere in the vm. Shutdown the vm

8. Change the settings of the vm, remove the primary vnc gpu, passthrough the gt 710 (video and audio) and choose the vbios file you saved before;

Switch to xml view and manually change the gpu block (audio and video) to enable multifunction,to:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <rom file='/path-to-vbios/gt710rom.dump'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
    </hostdev>

Note1: target addresses may be different for your case (04:00.0/1); just add multifunction='on' and make sure that video and audio are on same bus, same slot and different function in the target addresses

Note2: path to vbios (/path-to-vbios/) is obviously different in your case

9. Connect a monitor to the gpu and then start the vm; here there are 2 possibilities:

9a- you have video output: install inside the vm the nvidia drivers and uninstall (if you want) the vnc server/disable remote desktop

9b- you don't have a video output on the monitor: connect directly to the vm from a second device in the lan, pointing to the static ip you set before, install nvidia drivers and see if you have video output

Edited April 3, 2023 by ghost82

GPU not recognized on passthrough

Recommended Posts

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation