Not Able to Passthrough 3090

[antwick]

Having some issues i could use some help with.  I have created a win10 VM using spaceinvader's tutorial and it works perfectly using VNC  as the graphics card.   Then following his next tutorial i am attempting to passthrough my 3090 (which is my secondary card p2200 in first slot) but cannot get it to work. When i try and passthrough the 3090 the VM does not appear to actually boot - i cannot access it on splashtop and nothing displays on my monitor hooked to the 3090. When i start the VM with the 3090 passed through the logo on the 3090 will flash once and then stay lit - the monitor will flicker and stay on a black screen.  I have tried messing around with various settings but nothing seems to do the trick.  For Reference I am on 6.9.0-rc2 and i had installed the nvidia driver plugin to pass the p2200 through to plex (i have since deleted the plugin and am not passing through the p2200 to see if that would fix the vm issue, which it has not) . I wasnt sure what all would be needed to diagnose this so i thought i'd upload anything that might be of assistance.  Here is the XML of the VNC - VM that will work flawlessly VNC-VM-XML.txt  and the XML after i have passed through the 3090 3090-VM-XML.txt lastly here is a snipped of the GUI when i am attempting to passthrough the 3090.   

 

 

 

I have a friend that has the same server as me except he didnt install the nvidia driver plugin and he is not on 6.9 --- his works flawlessly which leads me to believe it has something to do with that.

Edited February 8, 2021 by antwick

[antwick]

Bump.... Anyone?  Someone?   Would really like to get this VM situation sorted out.

 

[Xionous]

Couple of things, if your 3090 is your second GPU you do not need to use gpu bios. I do not use a bios on my 3090 which is second card and it works fine for me. Secondly did you edit the xml to set the GPU sound card as the second function of your GPU? I Just looked at your XML for the 3090 and I see that you did not do this. Watch this video 

 

Thirdly have you tried starting the VM with only core 0 selected?

Edited February 8, 2021 by Xionous

[tjb_altf4]

Try dropping machine type back from i440fx-5.1 to i440fx-5.0

I've had a couple windows installs that wouldn't boot on 5.1, but would on 5.0

[antwick]

3 hours ago, Xionous said:

Couple of things, if your 3090 is your second GPU you do not need to use gpu bios. I do not use a bios on my 3090 which is second card and it works fine for me. Secondly did you edit the xml to set the GPU sound card as the second function of your GPU? I Just looked at your XML for the 3090 and I see that you did not do this. Watch this video 

 

Thirdly have you tried starting the VM with only core 0 selected?

I knew i was forgetting something when passing the GPU through again.  Watched the video and edited the XML - Still no luck. Nothing shows on attached monitor and cannot connect remotely.   Here is my current XML:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>3090-2</name>
  <uuid>e58ff718-cdb4-fa70-181c-c3eec10b82e2</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>13107200</memory>
  <currentMemory unit='KiB'>13107200</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>1</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-5.0'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/e58ff718-cdb4-fa70-181c-c3eec10b82e2_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='1' threads='1'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/3090-2/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Windows.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.173-2.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:53:94:26'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='2'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x4e' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x4e' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0b05'/>
        <product id='0x1915'/>
      </source>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x413c'/>
        <product id='0x301d'/>
      </source>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

[happyUnraid]

Do you passthrougt other device (like nvme) with your GPU

[antwick]

8 minutes ago, happyUnraid said:

Do you passthrougt other device (like nvme) with your GPU

No I did not

[happyUnraid]

Ok maybe try use multifunction for you GPU.

Do you have error in the VM manager log?

[domrockt]

Hello there, icant boot the VM with pc-i440fx-5.0 but with pc-q35-5.1

I did the xml edit

multifunction='on' and Soundcard 0x1 

i do not use any VBios

 

the only downside i experience is that when i reboot my VM it gets stuck on boot, pins one CPU to 100% and blacks out after Tiano spashscreen. 

I have to restart my whole server.

 

Mainboard: Asus Sage c621e 

CPU: Intel ES Platinum 26Core

GPU: EVGA 3090

iam using the Onboard Aspeed VGA as Unraid output by booting in legacy mode.

Edited March 2, 2021 by domrockt