NVIDIA RTX GPU Passthrough


FCruz2489

Recommended Posts

Hey Guys,

 

I just received my new NVIDIA RTX 2080 with which I was hoping to do GPU pass-through with to a Windows 10 VM and unfortunately it doesn't appear that UNRAID even recognizes it. I am running UNRAID 6.6 final on a Asrock - Z370 Gaming-itx/ac, with an Intel i7-8700, I have VT-d and IOMMU enabled. In the bios I set the integrated Intel GPU to "ON" and I also set that as the primary (although I also tried with PCI-e GPU as the primary and got the same results). I also tried with the integrated graphics set to auto. 

 

I don't know if its a matter of the card being too new for unraid to recognize it or if I am missing something else. From all the guides I have read, it seems that unraid should at least see the card and the trickery comes in when trying to do the passthrough but I am not seeing the card under system devices and it is not an option for the GPU within the edit VM settings. I have already tried the basic hardware troubleshooting of re-seating the card and power cables, and i dont have a second pci-e slot to test on so I came here.

 

I realize the cards were just released yesterday but I am hoping my diagnostics can help someone with more knowledge explain the issue or that its a simple matter of the hardware is unsupported for now. Any help is greatly appreciated, thank you!!!

fc-unraid-diagnostics-20180921-0827.zip

SystemDevices.png

Link to comment

Hi @testdasi thanks for the reply! Ok that makes sense, I did something similar to pass through my NVME drive, but how do I know the address of the RTX card if it doesn't show up anywhere within UNRAID? Is there a default address for the first PCI-E slot or something like that or should I be looking somewhere else to find the address tab besides the system devices? 

Link to comment

Hi @david279 thanks for the reply! Hmmm that is a very interesting factor that I didnt even consider. I will remove the NVME drive and try booting without it to see if that makes a difference, that should free up the 4 lanes which would hopefully allow the 16 needed for the GPU. Unfortunately I dont have another GPU to test with, maybe I can borrow one from work just to see if i get different results. PSU is a Corsiar SF600 - 600 Watt 80 PLUS Gold so I dont think that is a limiting factor, the GPU lights up and the fans spin, the PSU fan doesn't even kick in so that means the load is less than 50% if i am not mistaken.  

Link to comment

Ok so PCI lanes are not the problem, I was actually just doing research and confirmed the GPU is typically connected directly to the CPU and NVME drives are connected to the chipset so thats exactly what you just mentioned. This is a mini-itx board with a single pci-e slot so i theoretically should never run out of PCI-e lanes but it was a valid concern to mention. Thanks @johnnie.black! I think trying another GPU will be the best bet for now but the only real test would be another RTX GPU because i assume an older NVIDIA card would be ok. 

Link to comment

I guess its possible the card is DOA - which would be my luck. The motherboard is recent and running the latest 3.1 bios which is dated 7/10/2018 but anything is possible from a hardware standpoint and there's not much out there since the card is only a day or two old. I have also never tried any other card in the pci-e x16 slot so i definitely want to try something else to see if that works.

 

When i set the PCI-E RTX GPU as the primary GPU in the bios i just get a black screen but unraid eventually boots. In the web interface there is no change to the system devices or IOMMU groups. Dead card is a real possibility, will have to do some further testing to confirm. Thanks for the reply! 

 

Link to comment

I was able to get my RTX 2080 passthrough working on a Threadripper system with a Gigabyte X399 Designare motherboard. Initially I got an 'IOMMU group is not viable' error, which was due to the card having an internal USB controller in the same IOMMU group. After changing PCIe ACS override to 'both' under VM Settings, IOMMU groups were split up and I was then able to boot the VM with the RTX 2080. Performance has been great. I plan to try passing through the USB controller at some point in the future as well.

  • Like 1
Link to comment

Interesting, that is sort of the experience I was hoping for. So you didn't have to do anything special when you first popped it in right? It just showed up and then the IOMMU group issue is a different matter.

 

Did you get an NVIDIA Founder's edition or one of the AIB models? Just curious if our cards match since our systems are so different. 

 

Thanks for the reply and enjoy your new toy! :) 

Link to comment
23 hours ago, FCruz2489 said:

Interesting, that is sort of the experience I was hoping for. So you didn't have to do anything special when you first popped it in right? It just showed up and then the IOMMU group issue is a different matter.

 

Did you get an NVIDIA Founder's edition or one of the AIB models? Just curious if our cards match since our systems are so different. 

 

Thanks for the reply and enjoy your new toy! :) 

 

No, I didn't have to do anything special. I have the Founder's Edition card. It's definitely odd that it isn't showing up in lspci for you, so I'd have to guess that it's motherboard/chipset/CPU related. Also wanted to mention that I am running in UEFI mode.

 

image.thumb.png.b0398726748c0434332b5964b80d84e6.png

 

An update on my previous comment, I have now passed through the NVIDIA USB and Serial bus controller devices. I added vfio-pci.ids=10de:1ad8,10de:1ad9 to my syslinux config for them to show up in the VM template in Unraid. Seems to pass through fine, but no use for it it yet. I tried to plug in a flash drive but it didn't show up in the OS, so not sure exactly how this port is configured to function. I know it's meant for things like VR headsets but was hopeful I could also use it for regular USB devices. I'll have to test a keyboard or something basic like that.

 

Edited by rinseaid
  • Upvote 1
Link to comment

The behavior described sounds like you don't have the power connectors connected properly or insufficient power is being supplied. What's your PSU and what GPU did you have previously.

Bottom line: if the card doesn't show up on system devices it is 100% a hardware or configuration issue.

Sent from my Nexus 6 using Tapatalk

  • Upvote 1
Link to comment

Thanks for the suggestions guys. I have a Corsair SF600 600W 80+ Gold certified PSU with both the 8pin and 6pin connected to the card - but I’m going to pull the card and check all the cables to make sure they are inserted properly (they’re modular) and give this another go. My system is only supposed to consume around 450w at peak usage without overclocking so the PSU should be more than enough, especially for just booting. 

 

I wasn’t able to get a second card to test with but once I recheck everything, grabbing some kind of second card to check is my next step. I’ll update on status or progress, appreciate all the helpful suggestions! 

Link to comment

So to update this, I reseated the card and the PSU cables on both end 3 separate times. I saw no change at all so I am just going to assume the card arrived DOA since I dont have another system to test the card in, or another card to test in my unRAID build. I have initiated an RMA with NVIDIA so when I get a replacement card, if i still have this issue then at least I will know it is not the card. Thanks again to everyone for the help and suggestions, Ill be sure to check back once there is a change, hope everyone is having a great weekend! 

 

Link to comment
So to update this, I reseated the card and the PSU cables on both end 3 separate times. I saw no change at all so I am just going to assume the card arrived DOA since I dont have another system to test the card in, or another card to test in my unRAID build. I have initiated an RMA with NVIDIA so when I get a replacement card, if i still have this issue then at least I will know it is not the card. Thanks again to everyone for the help and suggestions, Ill be sure to check back once there is a change, hope everyone is having a great weekend! 
 
As an FYI, I have a 2080Ti on pre order as well (should be arriving this week). Once received, I'll also be able to perform a quick test.

Sent from my Nexus 6 using Tapatalk

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.