How to passthrough primary nVidia GPU with UEFI (without legacy/CSM boot)

[Matesaktesak]

Hi, so i had big trouble passing my nVidia GTX 1070Ti to a Debian VM, but got it resolved, so here is my solution.

The problem:

I'm running a consumer motherboard, consumer CPU and GPU. The only servergrade component in my setup is the HBA... This brings up some problems.

1. My system won't boot with legacy, it only works with UEFI. Besides that, i like using UEFI, because it makes sense for the usecase.

2. I only have a single GPU with no integrated GPU.

 

The solution:

1. Check that you have IOMMU enabled in the BIOS (while you're at it, also enable SR-IOV) – this makes it able, to pass individual components/cores to VMs

 

1.5 If your BIOS has the option, set VGA to IGNORE – this is important

 

2. In unRAID go to Tools → System Devices → PCI Devices and IOMMU Groups and check the tick boxes for your GPU and its integrated sound card – they should line on a single bus and slot, but have different functions. E.g. 26:00.0 VGA compatible controller and 26:00.1 Audio device

 

Press the "Bind selected to VFIO at boot” button.

 

3. Open the terminal and:

cd /boot/syslinux/

when you ls, this should be showing the content of your flesh drive

 

Use your editor of choice to edit the syslinux.cfg file:

nano syslinux.cfg

 

You will have to either modify the default boot option or make a new one, which is recommended. Insert this into the file:

label Unraid OS (GPUs left alone)
  menu default
  kernel /bzimage
  append initrd=/bzroot earlymodules=vfio-pci video=efifb:off,vesafb:off gfxpayload=text

The first line makes a new option called "Unraid OS (GPUs left alone)"

The second line makes it the default option – you have to remove this line from whichever option was default previously!

The third line sets the kernel dir

The last one sets the entry and some arguments:

 - earlymodules=vfio-pci - tells the system to load vfio-pci early in the bootup process, so it can take the GPUs

 - video=efifb:off,vesafb:off - turns off unraid console display

 - gfxpayload=text - disables graphics modes, limiting the attempts to use the GPU

 

Save the file (in nano its Ctrl+X than Y and than ENTER)

 

4. Reboot the server

 

Now try to create a VM with an image ideally containing the proper driver (eg. pop-os_22.04_amd64_nvidia_24.iso)

and assign the GPU and soundcard! If the GPU has a USB port, assign it too!

Also, if you're not using VNC or SSH, pass-through a keyboard...

 

Create the VM without running it, open the editor again and switch to the Form view. Edit the Soundcard address tag to mimic the HW GPU—so that the port is the same for both and the function is 0 and 1 respectively. 

 

I hope this is helpful!

 

 

Edited March 25, 2023 by Matesaktesak

[digital_billy]

thanks im going to give this a try hopefully it fixes my black screen issue