January 12, 20224 yr Hi all, I'm trying to use the SR-IOV feature of the iGPU of the intel 12600k (VGA function is device 0000:00:02.0). I'm on 6.10RC2 using an intel 12600k on a MSI Z690 PRO A This is my /boot/config/go #!/bin/bash # Start the Management Utility /usr/local/sbin/emhttp & echo 3 > /sys/bus/pci/devices/0000:00:02.0/sriov_numvfs # Relaunch vfio-pci script to bind virtual function adapters that didn't exist at boot time /usr/local/sbin/vfio-pci >>/var/log/vfio-pci This is my /boot/syslinux/syslinux.cfg default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label Unraid OS menu default kernel /bzimage i915.force_probe=4680 vfio-pci.enable_sriov=1 append isolcpus=0-11 video=efifb:off initrd=/bzroot intel_iommu=on label Unraid OS GUI Mode kernel /bzimage append isolcpus=0-11 initrd=/bzroot,/bzroot-gui label Unraid OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label Unraid OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest The log from the vfio-pci binding Loading config from /boot/config/vfio-pci.cfg BIND=0000:00:02.0|8086:4680 0000:00:02.1|8086:4680 0000:00:02.2|8086:4680 0000:00:02.3|8086:4680 --- Processing 0000:00:02.0 8086:4680 Vendor:Device 8086:4680 found at 0000:00:02.0 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:00:02.0/iommu_group/devices/0000:00:02.0 Binding... Successfully bound the device 8086:4680 at 0000:00:02.0 to vfio-pci --- Processing 0000:00:02.1 8086:4680 Error: Device 0000:00:02.1 does not exist, unable to bind device --- Processing 0000:00:02.2 8086:4680 Error: Device 0000:00:02.2 does not exist, unable to bind device --- Processing 0000:00:02.3 8086:4680 Error: Device 0000:00:02.3 does not exist, unable to bind device --- vfio-pci binding complete Devices listed in /sys/bus/pci/drivers/vfio-pci: lrwxrwxrwx 1 root root 0 Jan 12 21:51 0000:00:02.0 -> ../../../../devices/pci0000:00/0000:00:02.0 Loading config from /boot/config/vfio-pci.cfg BIND=0000:00:02.0|8086:4680 0000:00:02.1|8086:4680 0000:00:02.2|8086:4680 0000:00:02.3|8086:4680 --- Processing 0000:00:02.0 8086:4680 Vendor:Device 8086:4680 found at 0000:00:02.0 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:00:02.0/iommu_group/devices/0000:00:02.0 Binding... 0000:00:02.0 already bound to vfio-pci Successfully bound the device 8086:4680 at 0000:00:02.0 to vfio-pci --- Processing 0000:00:02.1 8086:4680 Vendor:Device 8086:4680 found at 0000:00:02.1 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:00:02.1/iommu_group/devices/0000:00:02.1 Binding... 0000:00:02.1 already bound to vfio-pci Successfully bound the device 8086:4680 at 0000:00:02.1 to vfio-pci --- Processing 0000:00:02.2 8086:4680 Vendor:Device 8086:4680 found at 0000:00:02.2 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:00:02.2/iommu_group/devices/0000:00:02.2 Binding... 0000:00:02.2 already bound to vfio-pci Successfully bound the device 8086:4680 at 0000:00:02.2 to vfio-pci --- Processing 0000:00:02.3 8086:4680 Vendor:Device 8086:4680 found at 0000:00:02.3 IOMMU group members (sans bridges): /sys/bus/pci/devices/0000:00:02.3/iommu_group/devices/0000:00:02.3 Binding... 0000:00:02.3 already bound to vfio-pci Successfully bound the device 8086:4680 at 0000:00:02.3 to vfio-pci --- vfio-pci binding complete Devices listed in /sys/bus/pci/drivers/vfio-pci: lrwxrwxrwx 1 root root 0 Jan 12 13:51 0000:00:02.0 -> ../../../../devices/pci0000:00/0000:00:02.0 lrwxrwxrwx 1 root root 0 Jan 12 13:52 0000:00:02.1 -> ../../../../devices/pci0000:00/0000:00:02.1 lrwxrwxrwx 1 root root 0 Jan 12 13:52 0000:00:02.2 -> ../../../../devices/pci0000:00/0000:00:02.2 lrwxrwxrwx 1 root root 0 Jan 12 13:52 0000:00:02.3 -> ../../../../devices/pci0000:00/0000:00:02.3 So the virtual function seem to be established end binded to vfio-pci (Kernel driver in use is vfio-pci): The devices are also showing in the GPU DropDown in the VM-Config: When I try to start the VM with any VF other than 00:02.0 I get the following error-message: internal error: qemu unexpectedly closed the monitor: 2022-01-12T22:18:26.174166Z qemu-system-x86_64: -device vfio-pci,host=0000:00:02.1,id=hostdev0,bus=pci.0,addr=0x2.0x1: vfio 0000:00:02.1: error getting device from group 14: Permission denied Verify all devices in group 14 are bound to vfio- or pci-stub and not already in use But I guess this error message is generic - in DMESG I get the following error after starting the VM, which I think is the real error: [ 640.917885] vfio-pci 0000:00:02.3: VF token required to access device By god I tried my best to find anything about that "VF token" thing, but I wasn't able to find much. Does anyone have a clue how get around that error? Edited January 12, 20224 yr by Acidcliff Forgot to mention my Unraid Version
January 13, 20224 yr The kernel driver isnt ready for 12th gen at least as far as gpu partitioning is concerned. Haven't tried it with bleeding edge kernels yet or anything, but at least as of 5.13, the latest "clean/stable" iGPU supported is 10th gen.
March 12, 20224 yr unraid 6.10 rc3 use the new linux kernel 5.15.27, will the new version support the sr-iov ?
March 18, 20224 yr On 3/12/2022 at 2:04 PM, monkiq said: unraid 6.10 rc3 use the new linux kernel 5.15.27, will the new version support the sr-iov ? Would love to know this. I have a 11th gen on the way, and it would be awesome to be able to use the HDMI output for a VM, but still use the GPU for hardware transcoding in Plex. I'm running RC3 currently, but on an AMD system
May 30, 20224 yr On 3/18/2022 at 4:03 AM, eatoff said: Would love to know this. I have a 11th gen on the way [...] I have an 11600K sitting in my system right now and just realized I can't use GVT-g. I'm really interested in this too.
May 30, 20224 yr What do you guys make of this comment? https://github.com/intel/gvt-linux/issues/190#issuecomment-1139698936 Quote You can check the linux-intel-lts repo, it is also maintained by Intel, You can check this tag, https://github.com/intel/linux-intel-lts/releases/tag/lts-v5.10.100-yocto-220223T232711Z which is support Alder Lake platform GPU SRIOV. Edited August 1, 20223 yr by smurofdiarnu
May 31, 20224 yr Author Looking at the code and commits to me it seems that Intel's LTS Kernel 5.10 has added SR-IOV support for the i915 driver. (Although nothing happened to this after the commit in on 24th Feb). Intels 5.15 LTS Kernel does not have it though. It's also not in the 5.18 Linux Kernel. Not sure in what state the updated SR-IOV driver in 5.10 is. I'm no Linux Pro so I'm also not sure if and how it is possible to compile the adapted driver and add it to the unraid kernel. But if it would be possible it would definitelly be cool
July 2, 20224 yr On 5/31/2022 at 5:15 PM, Acidcliff said: Looking at the code and commits to me it seems that Intel's LTS Kernel 5.10 has added SR-IOV support for the i915 driver. (Although nothing happened to this after the commit in on 24th Feb). Intels 5.15 LTS Kernel does not have it though. It's also not in the 5.18 Linux Kernel. Not sure in what state the updated SR-IOV driver in 5.10 is. I'm no Linux Pro so I'm also not sure if and how it is possible to compile the adapted driver and add it to the unraid kernel. But if it would be possible it would definitelly be cool Hello Acidcliff, I built a kernel with the IOV code in intel LTS Kernel 5.15. It looks fine to run but I have no 12th CPU to test the i915 SR-IOV ability. Would you please try it? It was built for unraid 6.10.3, please just overwrite these files to the flash drive. Please backup original 4 files, copying them back will revert the change. https://drive.google.com/drive/folders/1bAnedRHWaz7QGQAkO7xWohhnjDxSfbqS?usp=sharing It is just experiment, may not work. Maybe the code is not the only condition. Not working!!! Edited December 14, 20223 yr by jinlife
July 5, 20224 yr On 7/2/2022 at 1:06 PM, jinlife said: Hello Acidcliff, I built a kernel with the IOV code in intel LTS Kernel 5.15. It looks fine to run but I have no 12th CPU to test the i915 SR-IOV ability. Would you please try it? It was built for unraid 6.10.3, please just overwrite these files to the flash drive. Please backup original 4 files, copying them back will revert the change. https://drive.google.com/drive/folders/1bAnedRHWaz7QGQAkO7xWohhnjDxSfbqS?usp=sharing It is just experiment, may not work. Maybe the code is not the only condition. Anyone has tried ？
July 5, 20224 yr Author On 7/2/2022 at 7:06 AM, jinlife said: Hello Acidcliff, I built a kernel with the IOV code in intel LTS Kernel 5.15. It looks fine to run but I have no 12th CPU to test the i915 SR-IOV ability. Would you please try it? It was built for unraid 6.10.3, please just overwrite these files to the flash drive. Please backup original 4 files, copying them back will revert the change. https://drive.google.com/drive/folders/1bAnedRHWaz7QGQAkO7xWohhnjDxSfbqS?usp=sharing It is just experiment, may not work. Maybe the code is not the only condition. I think I will be able to test it on the weekend!
July 8, 20224 yr On 7/6/2022 at 4:06 AM, Acidcliff said: I think I will be able to test it on the weekend! Awesome, thanks for trying. Looking forward to the result, otherwise, there is still long way to go.
July 8, 20224 yr Quote krispan-intel commented yesterday >SRIOV upstream task is on-going, ETA Q4'23 https://github.com/intel/linux-intel-lts/issues/33#issuecomment-1176996341 Wow so far away... kinda makes we wish I went for AMD instead now since IIRC I saw people passing through their 5600G iGPUs etc.
July 8, 20224 yr Community Expert On 7/2/2022 at 6:06 AM, jinlife said: Hello Acidcliff, I built a kernel with the IOV code in intel LTS Kernel 5.15. It looks fine to run but I have no 12th CPU to test the i915 SR-IOV ability. Would you please try it? It was built for unraid 6.10.3, please just overwrite these files to the flash drive. Please backup original 4 files, copying them back will revert the change. https://drive.google.com/drive/folders/1bAnedRHWaz7QGQAkO7xWohhnjDxSfbqS?usp=sharing It is just experiment, may not work. Maybe the code is not the only condition. Are you able to advise which kernel values you may have changed?
July 9, 20224 yr 14 hours ago, SimonF said: Are you able to advise which kernel values you may have changed? I just built the whole kernel from intel LTS Kernel 5.15 which has the SR-IOV code. It may be not stable, but we could find out whether the SR-IOV code works fine or it is just earlier stage code.
July 10, 20224 yr On 7/2/2022 at 1:06 PM, jinlife said: Hello Acidcliff, I built a kernel with the IOV code in intel LTS Kernel 5.15. It looks fine to run but I have no 12th CPU to test the i915 SR-IOV ability. Would you please try it? It was built for unraid 6.10.3, please just overwrite these files to the flash drive. Please backup original 4 files, copying them back will revert the change. https://drive.google.com/drive/folders/1bAnedRHWaz7QGQAkO7xWohhnjDxSfbqS?usp=sharing It is just experiment, may not work. Maybe the code is not the only condition. It seems that only branches with yocto fields in the tags have SR-IOV related code, but I checked and it seems that only linux510 tags have yocto fields, did you port the i915 SR-IOV patch for linux510 to linux515?
July 26, 20223 yr On 7/10/2022 at 6:13 PM, Da_iaji said: It seems that only branches with yocto fields in the tags have SR-IOV related code, but I checked and it seems that only linux510 tags have yocto fields, did you port the i915 SR-IOV patch for linux510 to linux515? There are 2 branches of 5.15 have the SR-IOV code. I guess you need to check again. Anyway, no one test it yet. I guess not many people need it. Edited July 26, 20223 yr by jinlife
July 27, 20223 yr I'm not that knowledgeable about Linux, but is the SR-IOV compatibility built into the new 6.11 RC1?
July 30, 20223 yr On 7/26/2022 at 1:00 PM, jinlife said: There are 2 branches of 5.15 have the SR-IOV code. I guess you need to check again. Anyway, no one test it yet. I guess not many people need it. I think people are scared to test, I know I am. I have 1 Unraid server and if it goes down then that's my 1 server. Still want SR-IOV to work though
July 30, 20223 yr Short question? When i try this could i use one of the vigpus for docker? And the rest for vm‘s?
August 1, 20223 yr On 7/10/2022 at 7:13 PM, Da_iaji said: It seems that only branches with yocto fields in the tags have SR-IOV related code, but I checked and it seems that only linux510 tags have yocto fields, did you port the i915 SR-IOV patch for linux510 to linux515? If you checkout the 5.15/linux branch then it does show some SRIOV commits related to i915: git log --author='intel.com' --all --grep='SRIOV' but then again it's hard to tell whether it includes the latest work or not... Edited August 1, 20223 yr by smurofdiarnu
August 1, 20223 yr On 7/28/2022 at 6:36 AM, eatoff said: I'm not that knowledgeable about Linux, but is the SR-IOV compatibility built into the new 6.11 RC1? Unraid 6.11 RC1 is based on Linux 5.18 branch but intel only has 5.15 as the latest one for now. There are many customized codes in intel's repository, I would say let's wait for intel to update it. Edited August 18, 20223 yr by jinlife
September 26, 20223 yr Hi, I managed to assign SR-IOV VFs to VM using Intel's WIP driver mentioned above, though I'm testing with Arch Linux. More about it here: https://www.reddit.com/r/VFIO/comments/xoeika/sriov_on_intel_uhd_graphics_770_xe_12th_gen/ https://github.com/strongtz/i915-sriov-dkms Edited September 26, 20223 yr by Sophon
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.