RTX2080 passthrough surprise


11 posts in this topic Last Reply

Recommended Posts

So, 

I received an RTX2080 today (took advantage of the EVGA step up programme as I got a 1080 in July).

This is what hardware it presents:

IOMMU group 20:
[10de:1e87] 03:00.0 VGA compatible controller: NVIDIA Corporation TU104 [GeForce RTX 2080 Rev. A] (rev a1)
[10de:10f8] 03:00.1 Audio device: NVIDIA Corporation Device 10f8 (rev a1)
[10de:1ad8] 03:00.2 USB controller: NVIDIA Corporation Device 1ad8 (rev a1)
[10de:1ad9] 03:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1ad9 (rev a1)

 

So.. Previous Nvidia cards I've had presented as 2 devices. One graphics device and one audio device. The 'new' extra two are the serial bus (which I assume is the RGB controller), and a usb controller.

To my surprise, the usb type-c port on the back of the card actually functions as a full fledged usb port, so I'm able to connect a usb3 hub to it using a 'type-c to a' adapter and no longer need to pass through an additional pci-e usb card! 

The hub is being powered by the usb port, and has a keyboard, mouse and usb DAC connected with zero issues. 

 

Seeing as these cards are quite new and virtualization is a bit niche, I thought I'd put this down in a post for people to see.

Edited by billington.mark
Link to post
On 1/16/2019 at 4:22 PM, billington.mark said:

So, 

I received an RTX2080 today (took advantage of the EVGA step up programme as I got a 1080 in July).

This is what hardware it presents:


IOMMU group 20:
[10de:1e87] 03:00.0 VGA compatible controller: NVIDIA Corporation TU104 [GeForce RTX 2080 Rev. A] (rev a1)
[10de:10f8] 03:00.1 Audio device: NVIDIA Corporation Device 10f8 (rev a1)
[10de:1ad8] 03:00.2 USB controller: NVIDIA Corporation Device 1ad8 (rev a1)
[10de:1ad9] 03:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 1ad9 (rev a1)

 

So.. Previous Nvidia cards I've had presented as 2 devices. One graphics device and one audio device. The 'new' extra two are the serial bus (which I assume is the RGB controller), and a usb controller.

To my surprise, the usb type-c port on the back of the card actually functions as a full fledged usb port, so I'm able to connect a usb3 hub to it using a 'type-c to a' adapter and no longer need to pass through an additional pci-e usb card! 

The hub is being powered by the usb port, and has a keyboard, mouse and usb DAC connected with zero issues. 

 

Seeing as these cards are quite new and virtualization is a bit niche, I thought I'd put this down in a post for people to see.

Hey Thanks for the post.  

 

I am looking at getting an RTX 2080 but wasn't sure how the support with virtualization looked. 

 

Did you have to do anything in particular to get pass through to work with the USB and control on card.  ? ie.  any changes to vfio passthrough etc.. ?

 

Thanks in advance. 

Link to post
14 hours ago, billington.mark said:

Having a look, it doesnt seem its standard across the board on all 20series. might only be on the higher end SKUs? (2070/2080/2080Ti).

EVGA doesnt seem to have it on their 2060's

I just had a quick read around and it's inclusion is aimed at VR headsets for power/video/data instead of a birds nest of cables, dubbed VirtualLink.

Being aimed at VR probably explains why it's only on the higher SKUs.

 

https://hothardware.com/news/virtuallink-vr-headset-usb-c-connector-spec-backed-amd-nvidia-microsoft-oculus

Quote

VirtualLink uses an Alternate Mode of USB-C and connects with VR headsets to deliver four high-speed HBR3 DisplayPort lanes, a USB 3.1 data channel for supporting high-resolution cameras and sensors, and up to 27 watts of power. It's basically a nod to the future as VR headsets become more capable and demanding.

Pretty exciting bonus for virtualisation users, hopefully AMD picks up the standard for it's cards to!

Edited by tjb_altf4
Link to post
7 hours ago, Aceriz said:

Hey Thanks for the post.  

 

I am looking at getting an RTX 2080 but wasn't sure how the support with virtualization looked. 

 

Did you have to do anything in particular to get pass through to work with the USB and control on card.  ? ie.  any changes to vfio passthrough etc.. ?

 

Thanks in advance. 

Nothing special to get it to work... Stubbed the device like you usually would, and passed through like any other device. No issues at all! 

No extra config in Windows needed either. Just plugged in and it worked instantly like any normal usb port.

Link to post
  • 11 months later...
On 1/18/2019 at 9:14 AM, billington.mark said:

Nothing special to get it to work... Stubbed the device like you usually would, and passed through like any other device. No issues at all! 

No extra config in Windows needed either. Just plugged in and it worked instantly like any normal usb port.

Can you explain this a bit further please?

I want to do the same thing using a Quadro RTX 4000 but the USB / Serial bus controller is not visible in the VM settings (no other device is in the IOMMU group):

image.thumb.png.72d9b3696cd8b12b27234c09f3b51eae.png

 

image.thumb.png.c1043cda0b62dedbaf7f9d266b12686e.png

 

How to passtrough Nvidia USB / Serial bus controller ?

Link to post
  • 1 month later...

Sorry, ive not been online here in a while and only just seen that ive been tagged in a few posts. 

I no longer use a VM for my workstation for various reasons which are off topic!

(Also I cant remember all the commands, or fully up to speed with the best practice for stubbing and passing devices through in 2020, so im happy to be corrected if any of this advice is out of date!)
I know there's a new way of passing devices to VMs nowadays, but the premise that you have to pass all devices in an IOMMU group will still apply. 

 

You need to stub all the devices associcate with the card, so 2c:00.0 (GPU), 2c:00.0 (Audio), 2c:00.0 (USB) ,2c:00.0 (Serial).

The GPU and Audio are probaby stubbed by unraid in the background automagically (so they'll not have a driver assigned on boot) which is why they're selectable. The serial and USB wont have been, so unraid will be assigning a driver and grabbing the device as its own. 

Stubbing then rebooting should be enough for the USB and Serial device to show up in the VM config GUI for you to select. After that, it should pass through and just work. 

 

If they're not selectable, you need to check if a driver has been applied to the device (lspci should tell you). I think a virtio driver is assigned to stubbed devices. If anything else has grabbed it, youve not stubbed it properly.

If they have a virtio driver assigned, its probably just a GUI limitation. you'll need to go into the XML, find where the GPU and Audio device is assigned, and duplicate whats been set up automatically for the GPU and Audio, for the USB and Serial device.

 

Heopefully thats enough to work with. 

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.