Trying to passtrough my rtx 3080


Recommended Posts

Hi, i am trying to passtrough my rtx 3080, un-suprisingly its not working i am getting this error when creating the vm

Unable to complete install: 'internal error: qemu unexpectedly closed the monitor: 2020-10-02T00:01:15.125028Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T00:01:15.127699Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-10-02T00:01:15.127739Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T00:01:15.130352Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-10-02T00:01:15.130391Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T00:01:15.202958Z qemu-system-x86_64: -device vfio-pci,host=0000:01:00.0,id=hostdev0,bus=pci.4,addr=0x0: vfio 0000:01:00.0: group 1 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.'

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/createvm.py", line 2089, in _do_async_install
    guest.installer_instance.start_install(guest, meter=meter)
  File "/usr/share/virt-manager/virtinst/install/installer.py", line 542, in start_install
    domain = self._create_guest(
  File "/usr/share/virt-manager/virtinst/install/installer.py", line 491, in _create_guest
    domain = self.conn.createXML(install_xml or final_xml, 0)
  File "/usr/lib/python3.8/site-packages/libvirt.py", line 4035, in createXML
    if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
libvirt.libvirtError: internal error: qemu unexpectedly closed the monitor: 2020-10-02T00:01:15.125028Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T00:01:15.127699Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-10-02T00:01:15.127739Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T00:01:15.130352Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-10-02T00:01:15.130391Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T00:01:15.202958Z qemu-system-x86_64: -device vfio-pci,host=0000:01:00.0,id=hostdev0,bus=pci.4,addr=0x0: vfio 0000:01:00.0: group 1 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.

 

Some more info

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2206] (rev a1) <- this is my rtx 3080 vga
	Subsystem: eVga.com. Corp. Device [3842:3885]
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau, nvidia_drm, nvidia
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aef] (rev a1) <- this is my rtx 3080 audio
	Subsystem: eVga.com. Corp. Device [3842:3885]
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

 

Here is the list of devices in my iommu group 1

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) [8086:1905] (rev 07)

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2206] (rev a1) <- vfio
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aef] (rev a1) <- vfio

02:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU106 [GeForce RTX 2070] [10de:1f02] (rev a1)
02:00.1 Audio device [0403]: NVIDIA Corporation TU106 High Definition Audio Controller [10de:10f9] (rev a1)
02:00.2 USB controller [0c03]: NVIDIA Corporation TU106 USB 3.1 Host Controller [10de:1ada] (rev a1)
02:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU106 USB Type-C UCSI Controller [10de:1adb] (rev a1)

 

Here is information about the PC


Motherboard: Z390 AORUS ULTRA (VT-d is set to enabled in the bios, i am also on the latest firmware version which is F10b)

CPU: Intel core i7 8700
GPUS: (Both plugged straight in the motherboard)
   - rtx 3080 (pcie x16 top slot)

   - rtx 2070 (pcie x8 2nd slot)

Memory: 16gb dual channel

Edited by Cobiyyy
Link to comment

I don't know if maybe this could be your error:

RTX 2XXX also have a usb device, maybe is the same with RTX 3XXX if this is the case ensure to add it, also remember to use the "multifunction" there is a video from spaceinvader one that explains how to passthrough correctly RTX 2xxx.

Link to comment
7 hours ago, Decto said:

Do you have VT-D or the equivalent for AMD (AMD-V?) fully enabled in the BIOS

In my experience it's often switched off to allow Ryzen master to run

 

 

image.png.bf1ab9fbbb9306be356f3ed0bf774556.png

VT-D is set to Enabled in my bios.

 

4 hours ago, segator said:

I don't know if maybe this could be your error:

RTX 2XXX also have a usb device, maybe is the same with RTX 3XXX if this is the case ensure to add it, also remember to use the "multifunction" there is a video from spaceinvader one that explains how to passthrough correctly RTX 2xxx.

My rtx 3080 does not seem to have a usb device 

Here is the list of my iommu group 1

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) [8086:1905] (rev 07)

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2206] (rev a1) <- vfio
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aef] (rev a1) <- vfio

02:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU106 [GeForce RTX 2070] [10de:1f02] (rev a1)
02:00.1 Audio device [0403]: NVIDIA Corporation TU106 High Definition Audio Controller [10de:10f9] (rev a1)
02:00.2 USB controller [0c03]: NVIDIA Corporation TU106 USB 3.1 Host Controller [10de:1ada] (rev a1)
02:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU106 USB Type-C UCSI Controller [10de:1adb] (rev a1)

 

Link to comment

I get the same error when i try to pass my rtx 2070 with all of its usb controllers
 

Error starting domain: internal error: qemu unexpectedly closed the monitor: 2020-10-02T14:43:22.346905Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T14:43:22.349563Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-10-02T14:43:22.349601Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T14:43:22.352260Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-10-02T14:43:22.352307Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T14:43:22.488021Z qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0,id=hostdev0,bus=pci.6,addr=0x0: vfio 0000:02:00.0: group 1 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 66, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 1279, in startup
    self._backend.create()
  File "/usr/lib/python3.8/site-packages/libvirt.py", line 1234, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirt.libvirtError: internal error: qemu unexpectedly closed the monitor: 2020-10-02T14:43:22.346905Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T14:43:22.349563Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-10-02T14:43:22.349601Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T14:43:22.352260Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(48FH).vmx-exit-load-perf-global-ctrl [bit 12]
2020-10-02T14:43:22.352307Z qemu-system-x86_64: warning: host doesn't support requested feature: MSR(490H).vmx-entry-load-perf-global-ctrl [bit 13]
2020-10-02T14:43:22.488021Z qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0,id=hostdev0,bus=pci.6,addr=0x0: vfio 0000:02:00.0: group 1 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.

 

Edited by Cobiyyy
Link to comment

I'm not sure why I read this as AMD earlier.

 

Anyhow, I note you say that both GPU's and some PCI-E bus slots are in IOMMU group 1. 

Devices really need to be seperated into different IOMMU groups to pass through

 

Eg. from my server, 

image.thumb.png.2ce727d3cae71857611ef1ce254e0ccb.png

 

Have you tried PCIe ACS overide in Settings > VM Manager ?

This may break the devices into more manageable groups although it is a bit of a hack which tells the OS to assume it can talk to devices independently when they aren't actually reporting that as possible so your milage may vary. 

 

You may need to check the BIOS for any other functions in addition to VT-d which support virtualisation as some BIOS need multiple settings to fully support VM's.

 

'IOAPIC 24-119 Entries' - 'Enable' may help as it uses an alternative to IRQ for multifunction devices... but that's a long shot, I suspect it is more linked to the cards sharing the same IOMMU.

 

 

Link to comment
  • 1 year later...

Hi have same problem with my RTX3080
Here are logs from VM
https://pastebin.com/shUSQwmY
Here my VM Config

https://pastebin.com/EJE8AsPf

And my IOMMU
https://pastebin.com/Azy0scLs

I want to notice, that I already pass succesfuly Radeon R9 Nano, (also many problems, also inside of VM, so I changed for this RTX...)
Can somone just give me some tip where to look to fix it?
Reagrds
L.

Link to comment
  • 2 weeks later...

I guess I will hold off on buying a RTX card.

 

I love my unRaid server and have no regrets purchasing and using it. My over all feature request (not a complaint) is to make GPU passthrough much more plug-n-play. Getting my GTX card to work took 2+ hours to track down the pass-through config info to make it work. Thank You Space Invader One for providing much of the info needed. It still took more meta data mining  to fix the very bad audio  quality coming out the HDMI port on the GTX card. 

Edited by JackSafari
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.