GPU Passthrough Nvidia RTX 3700


Recommended Posts

Linux 5.10.28-Unraid x86_64

 

 

I try and set it using spaceinvaderones guide and the monitor is trying to connect but ends up without signal

 

 

ErrorWarningSystemArrayLogin


-nodefaults \
-chardev socket,id=charmonitor,fd=31,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
-blockdev '{"driver":"file","filename":"/mnt/user/domains/Windows 10/vdisk1.img","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \
-device virtio-blk-pci,bus=pci.0,addr=0x4,drive=libvirt-3-format,id=virtio-disk2,bootindex=1,write-cache=on \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/Windows.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \
-device ide-cd,bus=ide.0,unit=0,drive=libvirt-2-format,id=ide0-0-0,bootindex=2 \
-blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.190-1.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \
-device ide-cd,bus=ide.0,unit=1,drive=libvirt-1-format,id=ide0-0-1 \
-netdev tap,fd=33,id=hostnet0 \
-device virtio-net,netdev=hostnet0,id=net0,mac=52:54:00:a2:91:b5,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=34,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device usb-tablet,id=input0,bus=usb.0,port=1 \
-device vfio-pci,host=0000:06:00.0,id=hostdev0,bus=pci.0,multifunction=on,addr=0x5,romfile=/mnt/disk3/VM-shared/System/nvidia.rom \
-device vfio-pci,host=0000:06:00.1,id=hostdev1,bus=pci.0,addr=0x5.0x1 \
-device vfio-pci,host=0000:08:00.4,id=hostdev2,bus=pci.0,addr=0x8 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2022-01-21 07:17:55.342+0000: Domain id=3 is tainted: high-privileges
2022-01-21 07:17:55.342+0000: Domain id=3 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)
2022-01-21T07:17:57.472389Z qemu-system-x86_64: vfio: Cannot reset device 0000:08:00.4, depends on group 17 which is not owned.
2022-01-21T07:17:58.518687Z qemu-system-x86_64: vfio: Cannot reset device 0000:08:00.4, depends on group 17 which is not owned.
2022-01-21T07:18:06.894483Z qemu-system-x86_64: vfio: Cannot reset device 0000:08:00.4, depends on group 17 which is not owned.
2022-01-21T07:18:07.927438Z qemu-system-x86_64: vfio: Cannot reset device 0000:08:00.4, depends on group 17 which is not owned.

 

 

in the log I only see this line that might be revelant

 

Jan 20 22:33:59 Maincore kernel: vfio-pci 0000:06:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem

Link to comment
2 hours ago, binaryhamster said:

2022-01-21T07:17:57.472389Z qemu-system-x86_64: vfio: Cannot reset device 0000:08:00.4, depends on group 17 which is not owned.

You didn't setup correctly the passthrough.

Why you didn't attach diagnostics?

How one can know what it is that device?

 

Anyway, 08:00.4 is in a iommu group together with other devices, that are not attached to vfio, so 08:00.4 can't be resetted.

Link to comment
20 minutes ago, ghost82 said:

You didn't setup correctly the passthrough.

Why you didn't attach diagnostics?

How one can know what it is that device?

I am new to unraid. I will do that now. 

 

Anyway, 08:00.4 is in a iommu group together with other devices, that are not attached to vfio, so 08:00.4 can't be resetted.

 

Thank you. How to resolve it?

 

maincore-diagnostics-20220121-1206.zip

Link to comment

This:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/disk3/VM-shared/System/rtx3700.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x08' slot='0x00' function='0x4'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>

is wrong.

you are passing the video part of the gpu, putting it in a multifunction device at 00:05.0 and then you are passing the analogic built in audio and assign it to the multifunction device.

 

1. Replace this:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/disk3/VM-shared/System/rtx3700.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x08' slot='0x00' function='0x4'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>

 

with this:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/disk3/VM-shared/System/rtx3700.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x06' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x08' slot='0x00' function='0x4'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

 

This will pass both digital and analogic audio.

If you use displayport or hdmi audio only you can delete this part:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x08' slot='0x00' function='0x4'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>

 

2. You should attach to vfio at boot iommu groups 15 and 20: note: you won't have video output for unraid, the system will seem frozen, but it's not, connect to the unraid gui from an external device in the lan.

 

3. Ensure that rtx3700.rom is compatible with your gpu; if you are unsure, dump it from your card, if it's already dumped from your gpu you are ok.

 

4. reboot unraid and try.

 

If it doesn't work attach new diagnostics with all these changes applied.

Link to comment

No idea..nothing in the logs as far as I can see. I would try to switch to pc-q35 machine type instead of i440fx and reapply all the fixes.

Maybe you can run it first with vnc/emulated graphics, once booted you can enable remote desktop, check that it works with an external device, switch to gpu passthrough; if the vm boots but has no video output you can connect to it with remote desktop and check if the gpu is detected in windows but throws errors or if it's not detected at all.

If it's detected with errors, maybe you need to install drivers to have a display output.

  • Like 1
Link to comment

You are a bloddy genius... I installed Teamviewer because thats the only one I knew for windows. Then I just installed the nvidia drivers and boom...It appeared. Ive been obsessed about this issue for 3 days now. I bought the 2000$ setup just for this purpose and the thought of not being able because of bad choise of hardware would have been devastating.

 

Thanx again man... Your awsome... Next step...passthrough keyboard and mouse.

  • Like 2
Link to comment
On 1/21/2022 at 9:17 AM, binaryhamster said:

You are a bloddy genius... I installed Teamviewer because thats the only one I knew for windows. Then I just installed the nvidia drivers and boom...It appeared. Ive been obsessed about this issue for 3 days now. I bought the 2000$ setup just for this purpose and the thought of not being able because of bad choise of hardware would have been devastating.


 

Hi, im having the same problem that you experienced. Been trying all sorts of "fixes" but no luck. Did you changed the to q35 or left it as i440fx?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.