Intel iGPU SR-IOV on 12th Gen Alderlake UHD770 /GT1


Recommended Posts

Hi all,

 

I'm trying to use the SR-IOV feature of the iGPU of the intel 12600k (VGA function is device 0000:00:02.0).

 

I'm on 6.10RC2 using an intel 12600k on a MSI Z690 PRO A

 

This is my /boot/config/go

#!/bin/bash
# Start the Management Utility
/usr/local/sbin/emhttp &
echo 3 > /sys/bus/pci/devices/0000:00:02.0/sriov_numvfs
# Relaunch vfio-pci script to bind virtual function adapters that didn't exist at boot time
/usr/local/sbin/vfio-pci >>/var/log/vfio-pci

 

This is my /boot/syslinux/syslinux.cfg

default menu.c32
menu title Lime Technology, Inc.
prompt 0
timeout 50
label Unraid OS
  menu default
  kernel /bzimage i915.force_probe=4680 vfio-pci.enable_sriov=1
  append isolcpus=0-11 video=efifb:off initrd=/bzroot intel_iommu=on
label Unraid OS GUI Mode
  kernel /bzimage
  append isolcpus=0-11 initrd=/bzroot,/bzroot-gui
label Unraid OS Safe Mode (no plugins, no GUI)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Unraid OS GUI Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot,/bzroot-gui unraidsafemode
label Memtest86+
  kernel /memtest

 

 

The log from the vfio-pci binding 

Loading config from /boot/config/vfio-pci.cfg
BIND=0000:00:02.0|8086:4680 0000:00:02.1|8086:4680 0000:00:02.2|8086:4680 0000:00:02.3|8086:4680
---
Processing 0000:00:02.0 8086:4680
Vendor:Device 8086:4680 found at 0000:00:02.0

IOMMU group members (sans bridges):
/sys/bus/pci/devices/0000:00:02.0/iommu_group/devices/0000:00:02.0

Binding...
Successfully bound the device 8086:4680 at 0000:00:02.0 to vfio-pci
---
Processing 0000:00:02.1 8086:4680
Error: Device 0000:00:02.1 does not exist, unable to bind device
---
Processing 0000:00:02.2 8086:4680
Error: Device 0000:00:02.2 does not exist, unable to bind device
---
Processing 0000:00:02.3 8086:4680
Error: Device 0000:00:02.3 does not exist, unable to bind device
---
vfio-pci binding complete

Devices listed in /sys/bus/pci/drivers/vfio-pci:
lrwxrwxrwx 1 root root 0 Jan 12 21:51 0000:00:02.0 -> ../../../../devices/pci0000:00/0000:00:02.0

Loading config from /boot/config/vfio-pci.cfg
BIND=0000:00:02.0|8086:4680 0000:00:02.1|8086:4680 0000:00:02.2|8086:4680 0000:00:02.3|8086:4680
---
Processing 0000:00:02.0 8086:4680
Vendor:Device 8086:4680 found at 0000:00:02.0

IOMMU group members (sans bridges):
/sys/bus/pci/devices/0000:00:02.0/iommu_group/devices/0000:00:02.0

Binding...
0000:00:02.0 already bound to vfio-pci
Successfully bound the device 8086:4680 at 0000:00:02.0 to vfio-pci
---
Processing 0000:00:02.1 8086:4680
Vendor:Device 8086:4680 found at 0000:00:02.1

IOMMU group members (sans bridges):
/sys/bus/pci/devices/0000:00:02.1/iommu_group/devices/0000:00:02.1

Binding...
0000:00:02.1 already bound to vfio-pci
Successfully bound the device 8086:4680 at 0000:00:02.1 to vfio-pci
---
Processing 0000:00:02.2 8086:4680
Vendor:Device 8086:4680 found at 0000:00:02.2

IOMMU group members (sans bridges):
/sys/bus/pci/devices/0000:00:02.2/iommu_group/devices/0000:00:02.2

Binding...
0000:00:02.2 already bound to vfio-pci
Successfully bound the device 8086:4680 at 0000:00:02.2 to vfio-pci
---
Processing 0000:00:02.3 8086:4680
Vendor:Device 8086:4680 found at 0000:00:02.3

IOMMU group members (sans bridges):
/sys/bus/pci/devices/0000:00:02.3/iommu_group/devices/0000:00:02.3

Binding...
0000:00:02.3 already bound to vfio-pci
Successfully bound the device 8086:4680 at 0000:00:02.3 to vfio-pci
---
vfio-pci binding complete

Devices listed in /sys/bus/pci/drivers/vfio-pci:
lrwxrwxrwx 1 root root 0 Jan 12 13:51 0000:00:02.0 -> ../../../../devices/pci0000:00/0000:00:02.0
lrwxrwxrwx 1 root root 0 Jan 12 13:52 0000:00:02.1 -> ../../../../devices/pci0000:00/0000:00:02.1
lrwxrwxrwx 1 root root 0 Jan 12 13:52 0000:00:02.2 -> ../../../../devices/pci0000:00/0000:00:02.2
lrwxrwxrwx 1 root root 0 Jan 12 13:52 0000:00:02.3 -> ../../../../devices/pci0000:00/0000:00:02.3

 

So the virtual function seem to be established end binded to vfio-pci (Kernel driver in use is vfio-pci):

IOMMU.thumb.PNG.64f6885b63446b8d4df86ff5b29660c1.PNG

 

The devices are also showing in the GPU DropDown in the VM-Config:

VMconf.PNG.e25ceb44be3b93cf7afc22c0c4dfa675.PNG

 

When I try to start the VM with any VF other than 00:02.0 I get the following error-message:

internal error: qemu unexpectedly closed the monitor: 2022-01-12T22:18:26.174166Z qemu-system-x86_64: -device vfio-pci,host=0000:00:02.1,id=hostdev0,bus=pci.0,addr=0x2.0x1: vfio 0000:00:02.1: error getting device from group 14: Permission denied Verify all devices in group 14 are bound to vfio- or pci-stub and not already in use

 

But I guess this error message is generic - in DMESG I get the following error after starting the VM, which I think is the real error: 

[  640.917885] vfio-pci 0000:00:02.3: VF token required to access device

 

By god I tried my best to find anything about that "VF token" thing, but I wasn't able to find much.

Does anyone have a clue how get around that error?

Edited by Acidcliff
Forgot to mention my Unraid Version
Link to comment

The kernel driver isnt ready for 12th gen at least as far as gpu partitioning is concerned. Haven't tried it with bleeding edge kernels yet or anything, but at least as of 5.13, the latest "clean/stable" iGPU supported is 10th gen.

Link to comment
  • 1 month later...
On 3/12/2022 at 2:04 PM, monkiq said:

unraid 6.10 rc3 use the new linux kernel 5.15.27, will the new version support the sr-iov ?

 

Would love to know this. I have a 11th gen on the way, and it would be awesome to be able to use the HDMI output for a VM, but still use the GPU for hardware transcoding in Plex.

 

I'm running RC3 currently, but on an AMD system

Link to comment
  • 2 months later...

Looking at the code and commits to me it seems that Intel's LTS Kernel 5.10 has added SR-IOV support for the i915 driver. (Although nothing happened to this after the commit in on 24th Feb). Intels 5.15 LTS Kernel does not have it though. It's also not in the 5.18 Linux Kernel.

 

Not sure in what state the updated SR-IOV driver in 5.10 is.

 

I'm no Linux Pro so I'm also not sure if and how it is possible to compile the adapted driver and add it to the unraid kernel. But if it would be possible it would definitelly be cool :D

  • Upvote 2
Link to comment
  • 1 month later...
On 5/31/2022 at 5:15 PM, Acidcliff said:

Looking at the code and commits to me it seems that Intel's LTS Kernel 5.10 has added SR-IOV support for the i915 driver. (Although nothing happened to this after the commit in on 24th Feb). Intels 5.15 LTS Kernel does not have it though. It's also not in the 5.18 Linux Kernel.

 

Not sure in what state the updated SR-IOV driver in 5.10 is.

 

I'm no Linux Pro so I'm also not sure if and how it is possible to compile the adapted driver and add it to the unraid kernel. But if it would be possible it would definitelly be cool :D

Hello Acidcliff, I built a kernel with the IOV code in intel LTS Kernel 5.15. It looks fine to run but I have no 12th CPU to test the i915 SR-IOV ability.

Would you please try it? It was built for unraid 6.10.3, please just overwrite these files to the flash drive.

Please backup original 4 files, copying them back will revert the change.

https://drive.google.com/drive/folders/1bAnedRHWaz7QGQAkO7xWohhnjDxSfbqS?usp=sharing

It is just experiment, may not work. Maybe the code is not the only condition.

Edited by jinlife
Link to comment
On 7/2/2022 at 1:06 PM, jinlife said:

Hello Acidcliff, I built a kernel with the IOV code in intel LTS Kernel 5.15. It looks fine to run but I have no 12th CPU to test the i915 SR-IOV ability.

Would you please try it? It was built for unraid 6.10.3, please just overwrite these files to the flash drive.

Please backup original 4 files, copying them back will revert the change.

https://drive.google.com/drive/folders/1bAnedRHWaz7QGQAkO7xWohhnjDxSfbqS?usp=sharing

It is just experiment, may not work. Maybe the code is not the only condition.

Anyone has tried ?

Link to comment
On 7/2/2022 at 7:06 AM, jinlife said:

Hello Acidcliff, I built a kernel with the IOV code in intel LTS Kernel 5.15. It looks fine to run but I have no 12th CPU to test the i915 SR-IOV ability.

Would you please try it? It was built for unraid 6.10.3, please just overwrite these files to the flash drive.

Please backup original 4 files, copying them back will revert the change.

https://drive.google.com/drive/folders/1bAnedRHWaz7QGQAkO7xWohhnjDxSfbqS?usp=sharing

It is just experiment, may not work. Maybe the code is not the only condition.

I think I will be able to test it on the weekend!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.