GPU Passthrough With GB 3070 Gaming OC

RJ83

By RJ83
in VM Templates

Recommended Posts

RJ83 Noob

RJ83

Posted
Posted (edited)

I've worked through spaceinvaderone's videos on TS and still can't get past the black screen.  I've downloaded the appropriate rom, edited the nvidia header out, confirmed both the audio and video are on the same virtual slot and still no joy.  Has anyone been able to successfully get the card working on a W10 VM?  Full XML attached and I've tried loading the rom as both a .dump and .rom file.

 

edited for clarification: I should also note I was able to get the VM running prior to passing through the GPU and installed splashtop.  I have a monitor plugged into the card via hdmi (and a loopback dongle when it's not plugged in) and still cannot get this thing working.  Please help!

  

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>GamingVM</name>
  <uuid>1e489a41-f454-ed0d-8beb-543e2ece4491</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='4'/>
    <vcpupin vcpu='1' cpuset='12'/>
    <vcpupin vcpu='2' cpuset='5'/>
    <vcpupin vcpu='3' cpuset='13'/>
    <vcpupin vcpu='4' cpuset='6'/>
    <vcpupin vcpu='5' cpuset='14'/>
    <vcpupin vcpu='6' cpuset='7'/>
    <vcpupin vcpu='7' cpuset='15'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-4.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/1e489a41-f454-ed0d-8beb-543e2ece4491_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/GamingVM/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:a3:4b:60'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/blahblahblah/Gigabyte.RTX3070.8192.200917.dump'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

 

Edited by RJ83
Clarification
Link to comment
  • 3 months later...
Valiran Rookie

Valiran

Posted
Posted

Hello!

 

This is the part of my xml for my RTX3080:

  

<hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x21' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/mnt/user/isos/BIOS/Gigabyte.RTX3080.10240.200903.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x21' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x1'/>
    </hostdev>

 

For passing throught the GPU to the VM, I've just ticked the boxes of my card:

  

IOMMU group 28:		[10de:2206] 21:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3080] (rev a1)
 			[10de:1aef] 21:00.1 Audio device: NVIDIA Corporation GA102 High Definition Audio Controller (rev a1)

 

And I have this script running at startup of array (settings/user script):

  

#!/bin/bash
echo 0 > /sys/class/vtconsole/vtcon0/bind
echo 0 > /sys/class/vtconsole/vtcon1/bind
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

 

Have you tried other bios rom files? Without any bios rom?

Link to comment
  • 4 months later...
gray squirrel Rookie

gray squirrel

Posted
Posted (edited)

I have a 3080 and the only thing that worked was dumping my own V-BIOS. I found the old method using a temporary GPU the most reliable.

 


But this is the newer version. (This didn’t work for me)

 


edit: don’t put the loop back dongle in until you get it working with the monitor as it might be grabbing that as the primary monitor and is set to display on it only.

 

I have one and if I need to reconfigure unraid or my VM there is a risk it grabs it first

Edited by gray squirrel
Link to comment
cristiano1492 Noob

cristiano1492

Posted
Posted

Hi guys, I have the same issue with a new brand 3060Ti :  Blank screen and/or at least seems that the VideoCard didn't work. I just tried to create the script mentioned bu Spaceinvaders, but, at least for me, this script didn,t work as it might be.

- It's stuck to the line where it says to restart the server. I did it...the server re started, but the script didn't go over

Two more questions:

- in the tutorial the script recognize which kind of card is installed, in my case the system didn't show any card ( just a general nividia card ). What I did worng ? How can I let the system to recognize which card is installed ?

- Did the script create on his own the dump file ? Or do we have to tell the system where the .rom file is located ?

 

I would really love to understand why my brand new card can't work and/or is not recognized.

 

THX

 

Any supoort is more than welcome

script.png

Link to comment
  • 3 months later...
runamuk Rookie

runamuk

Posted
Posted (edited) 
achine='pc-i440fx-4.2'>hvm</type>

Why you using i440fx this always has issues with gpu passthrough.

 

multifunction is not in the correct spot

 

Passthrough all the other function of your video card then edit the multifuction . (usb port and other)
If you passthrough everything correctly you wont even need the bios. 

Edited by runamuk
Link to comment
runamuk Rookie

runamuk

Posted
Posted (edited) 
<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>GamingVM</name>
  <uuid>1e489a41-f454-ed0d-8beb-543e2ece4491</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='4'/>
    <vcpupin vcpu='1' cpuset='12'/>
    <vcpupin vcpu='2' cpuset='5'/>
    <vcpupin vcpu='3' cpuset='13'/>
    <vcpupin vcpu='4' cpuset='6'/>
    <vcpupin vcpu='5' cpuset='14'/>
    <vcpupin vcpu='6' cpuset='7'/>
    <vcpupin vcpu='7' cpuset='15'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-q35-6.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/1e489a41-f454-ed0d-8beb-543e2ece4491_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/GamingVM/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:a3:4b:60'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x0' multifunction='on'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>



Not sure if you are running the latest beta of Unraid; If you are make a new VM and paste in this XML. I also fixed the fact you have you multifunction on the wrong line. 

Edited by runamuk
Link to comment
  • 10 months later...
guayocampo Noob

guayocampo

Posted
Posted
On 8/15/2021 at 7:07 AM, cristiano1492 said:

Hi guys, I have the same issue with a new brand 3060Ti :  Blank screen and/or at least seems that the VideoCard didn't work. I just tried to create the script mentioned bu Spaceinvaders, but, at least for me, this script didn,t work as it might be.

- It's stuck to the line where it says to restart the server. I did it...the server re started, but the script didn't go over

Two more questions:

- in the tutorial the script recognize which kind of card is installed, in my case the system didn't show any card ( just a general nividia card ). What I did worng ? How can I let the system to recognize which card is installed ?

- Did the script create on his own the dump file ? Or do we have to tell the system where the .rom file is located ?

 

I would really love to understand why my brand new card can't work and/or is not recognized.

 

THX

 

Any supoort is more than welcome

script.png

 

Hey I have the same card with the same problem were you able to resolve it I already have 2 weeks and I try already all the videos from spaceinveder one. I really apreciated if you can provide me a hint or the answer for the problem with the new cards.

 

In advance thank you very much, at least for reading the post.

Link to comment
  • 2 months later...
nnescorpionn Noob

nnescorpionn

Posted
Posted
On 10/2/2022 at 5:40 PM, guayocampo said:

 

Hey I have the same card with the same problem were you able to resolve it I already have 2 weeks and I try already all the videos from spaceinveder one. I really apreciated if you can provide me a hint or the answer for the problem with the new cards.

 

In advance thank you very much, at least for reading the post.

did you manage to solve it? I also have 3060 ti and always have black screen.

Link to comment
ghost82 Proficient

ghost82

Posted
Posted
On 11/23/2021 at 7:19 AM, runamuk said: 
    <type arch='x86_64' machine='pc-q35-6.1'>hvm</type>

 

This xml wont work, you can't change only the machine type to q35 and use pci-root.

q35 has a different layout, based on pcie-root on bus 0, with pcie-root-port (s) attached to bus 0, with devices attached to either pcie-root (built-in devices) or pcie-root-port (s).

Fastest way is to re-create the vm with q35 machine type.

 

Link to comment
ghost82 Proficient

ghost82

Posted
Posted (edited)
On 11/23/2021 at 7:19 AM, runamuk said: 
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x0' multifunction='on'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>

 

From a general point of view this is also wrong.

multifunction tag should be not in the source block but in the target address block.

Obviously, for a pcie gpu, in a q35 machine type, target address bus should be other than 0, the gpu is not built-in, it should be attached to a pcie-root-port (index=1...n), i.e. bus=1...n

Edited by ghost82
Link to comment
Mrtj18 Apprentice

Mrtj18

Posted
Posted

I will admit I had a few issues getting my rtx 3080 to boot correctly within windows 11 VM. I had to:

 

1. Delete my windows 11 VM and vdisk from the VM menu and start all the way over.

 

2. Pass through my windows 11 nvme drive that already had windows 11 ruining. ( had issues getting it to run properly by installing it on a vdisk from my cache drive)

 

3. I used spaceinvadors GPU dumping tool found in his video. ( I only have 1 3080 in my primary GPU slot)

 

4. I had to set up the VM with the virtual machine GPU first, on first boot. Attempting to use the dumped GPU BIOS and passed through video and sound of my rtx 3080 would give me a black screen on first boot and would hang.

 

Hopefully my experience may help with installing windows 10 VM with your GPUs.

 

Link to comment
  • 1 month later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
Go to topic listing