Passthrough & VBios Dump Trouble


Resistor

Recommended Posts

Hey guys,

i'm trying to switch to Unraid with my Main PC, cause i wanted to have more possibilities to play with my Hardware.
I already have a running Unraid System, but without VM's, so my i'm pretty new to the "kinks" of VM's under Unraid.

I'm currently running with
 

Spoiler

Mainboard: ASUS ROG Strix X570-F Gaming (latest Bios)
CPU: Ryzen 7 3700X
GPU Main: ZOTAC GAMING GeForce RTX 2070 SUPER AMP Extreme (latest VBios)
GPU Second: MSI GTX 960


So i installed Unraid on another Stick and give it a go. Unraid booted fine, but when it comes to VM's, i'm pretty much at the end of my possibilities.

To start i'm having huge problems dumping a VBios from and passing through my 2070 to a W10 VM.
At least the File which comes out of the process, doesn't help with VM Blackscreen.
So i assume the VBios wasn't dumped correctly.

I'm using the latest @SpaceInvaderOne Script on a fresh Unraid 6.9.2 following his latest Video.

*Tried as Primary and Secondary GPU => Same results
 

This what happens when i try to dump my VBios.

//Without VFIO Bind//

Spoiler


Script location: /tmp/user.scripts/tmpScripts/Dump VBios/script
Note that closing this window will abort the execution of this script
You have selected this device to dump the vbios from
09:00.0 VGA compatible controller: NVIDIA Corporation TU104 [GeForce RTX 2070 SUPER] (rev a1)

This does look like a valid GPU to me. Continuing .........


Checking if location to put vbios file exists
Vbios folder already exists

I will try and dump the vbios without disconnecting and reconnecting the GPU
This normally only works if the GPU is NOT the Primary or the only GPU
I will check the vbios at the end. If it seems wrong I will then retry after disconnecting the GPU

Defining temp vm with gpu attached
Domain dumpvbios defined from /tmp/dumpvbios.xml

Starting the temp vm to allow dump
error: Failed to start domain dumpvbios
error: internal error: qemu unexpectedly closed the monitor: 2021-09-19T19:51:19.219455Z qemu-system-x86_64: -device vfio-pci,host=0000:09:00.0,id=hostdev0,x-vga=on,bus=pci.2,addr=0x0: vfio 0000:09:00.0: group 23 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.

Waiting for a few seconds .....

Stopping the temp vm
error: Failed to destroy domain dumpvbios
error: Requested operation is not valid: domain is not running

Removing the temp vm
Domain dumpvbios has been undefined

Okay dumping vbios file named RTX 2070 SUPER.rom to the location /mnt/user/isos/vbios/

vbios seems to be correct. All done :)

---------------------------------------------------------------------------------------------------------------------------------
 

The dumped VBios is around 128kb in size, which feels way to small considerung techpowerups VBios Dump for my Card is around 1MB.
I still tried it, but it does not work either.


 

//With VFIO Bind//

Spoiler

IOMMU group 23:
[10de:1e84] 09:00.0 VGA compatible controller: NVIDIA Corporation TU104 [GeForce RTX 2070 SUPER] (rev a1)
[10de:10f8] 09:00.1 Audio device: NVIDIA Corporation TU104 HD Audio Controller (rev a1)
[10de:1ad8] 09:00.2 USB controller: NVIDIA Corporation TU104 USB 3.1 Host Controller (rev a1)
This controller is bound to vfio, connected USB devices are not visible.
[10de:1ad9] 09:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU104 USB Type-C UCSI Controller (rev a1)

---------------------------------------------------------------------------------
Script location: /tmp/user.scripts/tmpScripts/Dump VBios/script
Note that closing this window will abort the execution of this script
You have selected this device to dump the vbios from
09:00.0 VGA compatible controller: NVIDIA Corporation TU104 [GeForce RTX 2070 SUPER] (rev a1)

This does look like a valid GPU to me. Continuing .........


Checking if location to put vbios file exists
Vbios folder already exists

I will try and dump the vbios without disconnecting and reconnecting the GPU
This normally only works if the GPU is NOT the Primary or the only GPU
I will check the vbios at the end. If it seems wrong I will then retry after disconnecting the GPU

Defining temp vm with gpu attached
Domain dumpvbios defined from /tmp/dumpvbios.xml

Starting the temp vm to allow dump
Domain dumpvbios started

Waiting for a few seconds .....

Stopping the temp vm
Domain dumpvbios destroyed

Removing the temp vm
Domain dumpvbios has been undefined

Okay dumping vbios file named RTX 2070 SUPER.rom to the location /mnt/user/isos/vbios/
cat: rom: Input/output error

Um.... somethings gone wrong and I couldn't dump the vbios for some reason
Sometimes when this happens all we need to do to fix this is 'stub' or 'bind to the vfio' the gpu and reboot the server

This can be done in Unraid 6.8.3 with the use of the vfio config plugin or if you are on Unraid 6.9 or above it can be done
directly from the gui in Tools/System Devices .....So please do this and run the script again

--------------------------------------------------------------------------------------------------------------------------
 

The Monitor connected to the Card shows a Failed Boot Screen while the script is running.

SeaBios
Booting from Hard Disk -> failed
Boot from Floppy -> failed

No bootable device


 

When it comes to Passthrough it looks like this:

Passing through my secondary GPU 2 (GTX 960) even without a VBios File works.
Passing through my 2070 as first or secondary GPU (Bound or Unbound) does not work.

When i try to start the VM when bound the screen stays black this shows up in the Logs.

 

Spoiler

2021-09-19T20:49:35.843125Z qemu-system-x86_64: vfio_region_write(0000:08:00.0:region1+0x11f880, 0xff000000ff000000,8)
failed: Device or resource busy

 

This also happens when i try to pass through my GTX 960, when it's in the first Slot.

Even completely removing my secondary GPU from the System does not help.


Anyone got a hint what could be the problem ?

Edited by Resistor
Link to comment
  • 1 month later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.