RTX 3090 passthrough with Windows 10 VM


Juxsta

Recommended Posts

I've tried just about everything and I can't get the passthrough to work. 

Specs: 

Motherboard: Asus rog strix x570i

CPU: 3900x

GPU: PNY XLR8 RTX 3090 UPRISING EPIC-X Triple Fan

RAM: G.SKILL TridentZ RGB Series 64GB (2 x 32GB) 288-Pin DDR4 SDRAM DDR4 3600 (PC4 28800) Intel XMP 2.0 Desktop Memory Model F4-3600C18D-64GTZR

 

I've followed SpaceInvader One's videos to get the vm started. The card is outputting to a monitor but I am receiving error 43. I tried to follow this guide but this method seems dated. [GUIDE] Fix Nvidia Code 43 Issue on Nvidia GPU - VMs - Unraid

The kvm patcher is no longer being maintained and is not compatible with newer nvidia drivers, this is a problem with the new set of cards.

 

I tried booting for UEFI but it stopped my card from outputting completely. 

image.png.7f9b950676699b4180f4eaf65c02cef0.png

 

I've tried turning hyper-v on and off. This is my config

image.thumb.png.d3e66f3809859448659cb5b03a6914e9.png

I've been battling with this for a while and could really use some help!

Edited by Juxsta
Link to comment

I don’t remember editing the vbios when I dumped my own. I don’t think you are supposed to do that.

 

did you follow this guide?

 

 

edit: just re watched this and you need the 3090 as the secondary GPU.

 

when I tried dumping via GPUZ and editing it still don’t work for me. So I followed this guide.


do you have another system with two PCIE slots. Or somebody you trust to dump the bios via this method?

 

 

 

Edited by gray squirrel
Link to comment
On 12/5/2020 at 2:01 AM, gray squirrel said:

I don’t remember editing the vbios when I dumped my own. I don’t think you are supposed to do that.

 

did you follow this guide?

 

 

edit: just re watched this and you need the 3090 as the secondary GPU.

 

when I tried dumping via GPUZ and editing it still don’t work for me. So I followed this guide.


do you have another system with two PCIE slots. Or somebody you trust to dump the bios via this method?

 

 

 

I actually moved the card to another machine running windows and I was able to successfully dump the bios on a different computer. It seems like spaceinvader used the card as a secondary gpu so that he can successfully pass it into the virtual machine. 

I've also referenced this video as it seems to be a bit more updated than the one you posted. 

https://www.youtube.com/watch?v=1IP-h9IKof0&ab_channel=SpaceinvaderOne

Edited by Juxsta
Link to comment
10 hours ago, turnipisum said:

Have you got all parts of the 3090 card selected in iommu/vfio sound any usb etc...

I've selected the sound and video controllers. The new cards don't seem to have usb. 

image.thumb.png.5378f67b72218735ef864660a1c8e743.png

10 hours ago, turnipisum said:

Also maybe try machine as i440fx

I've tried both i440fx and Q35

 

9 hours ago, turnipisum said:

Just took a look at this and am resetting my server now. Do you happen to have any recommended configuration for the vm. I've seen a lot of varying information, such as whether hyper-v is enabled, Q-35 vs 1440fx, booting from UEFI vs legacy, or even in the order of how to setup the vm. Such as vs setup

1. booting with vnc, installing all the drivers THEN rebooting with gpu passthrough

2. booting and setting up with vnc enabled and gpu passthrough
3. booting and setting up with only graphics card

I've tried just about every variation and am pretty much at my wits end. I'm hoping that modifying the boot menu alone will work.

Link to comment
21 hours ago, Juxsta said:

I've selected the sound and video controllers. The new cards don't seem to have usb. 

image.thumb.png.5378f67b72218735ef864660a1c8e743.png

I've tried both i440fx and Q35

 

Just took a look at this and am resetting my server now. Do you happen to have any recommended configuration for the vm. I've seen a lot of varying information, such as whether hyper-v is enabled, Q-35 vs 1440fx, booting from UEFI vs legacy, or even in the order of how to setup the vm. Such as vs setup

1. booting with vnc, installing all the drivers THEN rebooting with gpu passthrough

2. booting and setting up with vnc enabled and gpu passthrough
3. booting and setting up with only graphics card

I've tried just about every variation and am pretty much at my wits end. I'm hoping that modifying the boot menu alone will work.

VM setup it's been a while lol. As far as i can remember i passed 2070 super, audio and usb through when i created the vm's. I had to mess about a bit to find the correct usb's for each vm as i have 2 vm's with 2070 supers.

 

Make sure you have enabled all the options in BIOS for virtual machine, iommu etc...

 

My setup is as below if it helps

 

Unraid boot: Legacy

 

VM Manager

Libvirt version: 6.5.0

QEMU version: 5.1.0

Libvirt storage location: /mnt/user/system/libvirt/libvirt.img

Default VM storage path: /mnt/user/domains/

Default ISO storage path: /mnt/user/isos/

Default Windows VirtIO driver ISO (optional): virtio-win-0.1.189-1.iso

Default network bridge: br0

Upon host shutdown: Shutdown

PCIe ACS override: Multi-function

VFIO allow unsafe interrupts: No

 

VM Windows 10

CPU Mode: Host Passthrough (AMD Ryzen Threadripper 3970X 32-Core)

Logical CPUs: 

Initial Memory:  32256MB 

Machine: i440fx-5.1

BIOS: OVMF

Hyper-V: Yes

USB Controller: 3.0 (qemu XHCI)

OS Install ISO: /mnt/user/isos/Windows.iso

OS Install CDRom Bus: IDE

VirtIO Drivers ISO: /mnt/user/isos/virtio-win-0.1.189-1.iso

VirtIO Drivers CDRom Bus: IDE

Primary vDisk Location: Manual /mnt/disks/Force_MP600_XXXX/Windows 10/vdisk1.img

Primary vDisk Bus: VirtIO

Graphics Card: NVIDIA GeForce RTX 2070 SUPER (4d:00.0)

Graphics ROM BIOS:

Sound Card: NVIDIA TU104 HD Audio Controller (4d:00.1)

Network MAC: XXXXXXX

Network Bridge: br0

Network Model: virtio-netvirtio

Other PCI Devices:

 AMD Starship USB 3.0 Host Controller | USB controller (03:00.3)
 NVIDIA TU104 USB 3.1 Host Controller | USB controller (4d:00.2)
 NVIDIA TU104 USB Type-C UCSI Controller | Serial bus controller (4d:00.3)
 

 

Hope it helps

 

Edited by turnipisum
Link to comment
16 hours ago, KptnKMan said:

 @Juxsta

 You might benefit from the HOWTO I wrote over here:

 

I spent a lot of time putting this HOWTO together, and I usually refer to it when I see others struggling with the same.

 

Let me know if you found it useful. 👍

I tried this out but unfortunately I'm still getting the code 43

 

Link to comment
50 minutes ago, KptnKMan said:

You checked both the videos, followed all the instructions and made sure to pass through the sound devices exactly as advised also?

Yeah.I’ve dumped the v-bios using a different computer and passed in the rom file via the GUI. (Looks like the translated XML is the same as in the video). I setup the virtual machine using the most up to date options posted on your guide. I passed in both the Audio and video devices and made the respective changes to the xml as noted in the second spaceinvader video and your guide. I then continued to install all the drivers using virtio, but once the gpu installs its drivers, it immediately registers error 43.

I could definitely be missing something so please let me know if there’s anything you can think of that I missed.

Link to comment
2 hours ago, turnipisum said:

Is slot='0x00 being used more than once? Virtio-net, virtio-serial and gpu looking at it.

Ahh I didn’t realize that virtio devices also need a slot. Would I move it to 0x01?

Looks like you can't modify the slot number of a pci address past 0x00 

image.png.bb408298c7eaf8063198dbb00efbb9da.png

Edited by Juxsta
Link to comment
1 hour ago, Juxsta said:

Ahh I didn’t realize that virtio devices also need a slot. Would I move it to 0x01?

Looks like you can't modify the slot number of a pci address past 0x00 

image.png.bb408298c7eaf8063198dbb00efbb9da.png

Just try putting the 3090 on 0x05 or something not in use, make sure not to change the part that is physical slot, first part i think it is. only change the virtual slot. check video space invader explains the difference.

Link to comment
17 minutes ago, Juxsta said:

I get this error when Trying to use that file.

image.png.d28ed960c02ce81cfbcd1bb0f9d86b1a.png

Looks like it wants the GPU to stay on slot 0. I'm getting similar errors when changing the slots for the virtio devices

Hmm ok. getting thin on idea's now. Have you tried working config without a vbios rom and see if the drivers install/load without error.

Edit: I say that can't remember if you even got windows setup or not?

Edited by turnipisum
Link to comment
13 minutes ago, turnipisum said:

Hmm ok. getting thin on idea's now. Have you tried working config without a vbios rom and see if the drivers install/load without error.

Edit: I say that can't remember if you even got windows setup or not?

There is no video output when loading up the gpu without a V-bios. And my windows VM is working, it's just throwing error 43. Saying the device was turned off because it wasn't working properly.

Yeah, I've been working on this for a while now. I've also messaged limetech support directly but they were unable to help me as they have been unable to get any 3090's on hand. The rep did say that it could also be a brand specific problem. If all else fails I do have an rtx 2080 evga card in my other rig, but i'd really rather not swap them.

Edited by Juxsta
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.