unRAID plugin for iGPU SR-IOV support

[Tower_trial]

On 4/10/2023 at 4:07 PM, Cheater said:

When I use SR-IOV the Intel XE graphics card is correctly detected. I'm using Teamviewer to view the screen, but the resolution is just 640x480. I assume this is because no physical screen is attached to adapt the resolution to. Any ideas how to resolve this?

 

 

I have the same problem . i testet also with hdmi dongel in hdmi port but i cant set high resolution.

 

any tipps ?

[shpitz461]

Hi, I'm confused, do I still need to follow the installation steps shown here:

Quote

Installation (Assuming running on the Unraid web terminal):

Backup original bzimage, bzimage.sha256, bzmodules, bzmodules.sha256 in /boot/;

Use bzimage, bzimage.sha256, bzmodules, bzmodules.sha256 from this release to replace the original ones;

Add module option for i915:
echo "options i915 enable_guc=7" >> /boot/config/modprobe.d/i915.conf

Make i915 module load on boot:
echo "modprobe i915 >> /boot/config/go"
echo "echo 2 > /sys/devices/pci0000:00/0000:00:02.0/sriov_numvfs" >> /boot/config/go

Fix PCI address error when iGPU VF assigned to VM
echo "sed -i \"s/\\(strSpecialAddress.*\\)\\\$gpu_function/\\1\\\"0\\\"/\" /usr/local/emhttp/plugins/dynamix.vm.manager/include/libvirt.php" >> /boot/config/go

I just don't need to replace the kernel files on the usb stick?

 

Does this work on AMD/Epyc or just Intel?

 

Does this work on Unraid 6.12.0-rc3?

 

Does this work for any gpu or only on Intel iGPU?

 

Thanks

[zhtengw]

28 minutes ago, shpitz461 said:

Hi, I'm confused, do I still need to follow the installation steps shown here:

I just don't need to replace the kernel files on the usb stick?

 

Does this work on AMD/Epyc or just Intel?

 

Does this work on Unraid 6.12.0-rc3?

 

Does this work for any gpu or only on Intel iGPU?

 

Thanks

Hi, this works only on tiger-lake and later Intel iGPU. And if you have a supported hardware, you are not need to replace the kernel image, installing the i915-sriov is enough.

6.12.0-rc3 is supported.

[FEILUAN]

The plugin cannot work on 6.12.0-rc5

The installation run failed

----------Downloading i915-sriov module  for kernel v6.1.27---------- -

 

--------This could take some time, please don't close this window!-----------

--------Can't download i915-sriov module Package for kernel v6.1.27----------

- plugin: run failed: '/bin/bash' returned 1 Executing hook script: post_plugin_checks

Edited May 2, 2023 by FEILUAN

[dirkinthedark]

Hello, I'm looking to upgrade my current setup.

 

I would really like a virtual gpu to free up my pci slots and save power.

 

Need transcoding in a docker and passthrough vm (Linux or Windows 10 or 11 (Jellyfin)) for home theatre. 

 

Is the sriov working yet?  Should I wait? Is there a more stable AMD solution? 

 

Thank you  

[riduxd]

On 5/1/2023 at 8:57 PM, FEILUAN said:

The plugin cannot work on 6.12.0-rc5

The installation run failed

----------Downloading i915-sriov module  for kernel v6.1.27---------- -

 

--------This could take some time, please don't close this window!-----------

--------Can't download i915-sriov module Package for kernel v6.1.27----------

- plugin: run failed: '/bin/bash' returned 1 Executing hook script: post_plugin_checks

Mine installed just fine, but this plugin is causing incompatibility issues with intel_gpu_top

[shpitz461]

On 5/3/2023 at 11:06 AM, dirkinthedark said:

Hello, I'm looking to upgrade my current setup.

 

I would really like a virtual gpu to free up my pci slots and save power.

 

Need transcoding in a docker and passthrough vm (Linux or Windows 10 or 11 (Jellyfin)) for home theatre. 

 

Is the sriov working yet?  Should I wait? Is there a more stable AMD solution? 

 

Thank you  

Two posts up the OP said this only works on Intel, I have an AMD cpu as well, so no-go for us...

[Duglim]

I want to thank you for providing this solution and information in this thread!

Was reading for a while now and fully managed to get iGPU passthrouh working yesterday.

 

My used Hardware:

- Intel Core-i5-13600 CPU with Alder Lake-S GT1 iGPU
- Gigabyte Aorus B760i DDR4 Mainboard

- NO dedicated GPU

- Unraid 6.11.5

 

I will tell you my journey and steps i took:

1. I started with not having an SR-IOV Option in my BIOS. So I contacted Gigabyte Support and asked them about SR-IOV enabled Bios. Honestley, I assumed, my journey would end here, but after 7 days, the Gigabayte Support send me a custom BIOS with SR-IOV Option. I'm still impressed with such an incredible support, simply wow!
2. After flashing the BIOS and enabling SR-IOV, I installed this SR-IOV-Plugin and 2 VF iGPUs came up. Device 2.0 for host, 2.1 and 2.2 for VMs.
3. I was able to bind the 2 VFs to VFIO without any issue.
4. The GPU-Statistics-Plugin could not show any usage data anymore. I figured out, this was due to using logical iGPU adresses instead of physical. Also the Plugin was just looking for one iGPU. But there is a fork plugin from SimonF in the Unraid Apps which does the trick. So installed SimonFs GPU-Statistics-Plugin and iGPU usage data for the host iGPU was available again.
   Please note, that this only worked after bindung all VF iGPUs to VFIO. As soon as there are more than 1 iGPU visible to the host, usage data in GPU-Statistics breaks and only Core Freqency works.
5. Edited my existing Windows 11 VM template to add the VFIO iGPU 2.1 as 2nd VGA device. I did NOT add a VGA bios.
6. Booted the VM and installed the Intel GPU drivers. The GPU was detected correctly and was already working. I also set up Remote Desktop and Parsec (host mode with virtual display driver)
7. Since I dont need the VNC Driver and Virtal Adapter, I removed both from the VM XML (needed to do this in XML-view, since editing the VM in Simple View Mode messed up some PCI-devices.
8. Last but not least I also bound the Onboard Soundcard to VFIO and passed to to the VM (again in XML-mode) and finally had sound in Parsec.

So, please keep up this good work and discussion here, it was very helpful and provided fresh knowledge to me.

 

Here are both XML snippets I added to the VM-Template:

 

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1f' function='0x3'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x00‘ function='0x0'/>
    </hostdev>

 

First hostdev is the iGPU. second the onboard soundcard.

 

Greetz, Duglim

 

 

Edited May 10, 2023 by Duglim

[Cheater]

About the physical outputs a video card has... Intel describes SR-IOV like this: (source)

Quote

Intel Graphics SR-IOV Technology

Intel Graphics SR-IOV is Intel's latest virtualization technology for graphics. Single Root I/O Virtualization (SR-IOV) defines a standard method for sharing a physical device function by partitioning the device into multiple virtual functions. Each virtual function is directly assigned to a virtual machine, thereby achieving near-native performance for the virtual machine.

The key benefits of Intel Graphics SR-IOV are:

 

- A standard method of sharing a physical GPU with virtual machines, thus allowing efficient use of GPU resources in a virtual system.

- Improved video transcode, media AI analytics, and virtual desktop infrastructure (VDI) workloads performance in virtual machine.

- Support up to four independent display output and seven virtualized functions (12th generation Intel® Core™ embedded processors).

- Support for multiple guest operating systems.

 

Related to item 4:

"- Support up to four independent display output and seven virtualized functions (12th generation Intel® Core™ embedded processors)." comes the question:

 

Can we expect this functionality (four independent display outputs) anytime, from the SR-IOV implementation within the Linux Kernel or is it already possible to configure this?

Edited May 11, 2023 by Cheater

[asjk169]

 

My hardware platform is ERYING 12500H+G660, and the plugin isn't working on my platform.

 

During the installation, the system log shows 

May 10 19:53:33 Tower root: plugin: creating: /boot/config/plugins/i915-sriov//unraid-i915-sriov-2023.04.06.txz - downloading from URL https://raw.githubusercontent.com/zhtengw/unraid-i915-sriov/master/packages/unraid-i915-sriov-2023.04.06.txz
May 10 19:53:34 Tower root: plugin: running: /boot/config/plugins/i915-sriov//unraid-i915-sriov-2023.04.06.txz
May 10 19:53:34 Tower root: plugin: running: anonymous
May 10 19:53:34 Tower root: plugin: running: anonymous
May 10 19:53:40 Tower kernel: i915 0000:00:02.0: driver does not support SR-IOV configuration via sysfs
May 10 19:53:40 Tower root: plugin: i915-sriov.plg installed

 

before reboot, the plugin page shows

 

after reboot, the plugin page shows

 

 

I paste the system diagnostics before reboot and after reboot here. Does this is the problem of drive not supporting my devices?However, hardware decode works fine in jellyfin docker container. Thanks for paying attention.

 

tower-diagnostics-20230511-1451.zip tower-diagnostics-20230511-1455.zip

[dirkinthedark]

On 5/9/2023 at 6:24 PM, shpitz461 said:

Two posts up the OP said this only works on Intel, I have an AMD cpu as well, so no-go for us...

Amd uses a different solution.  I was wondering which one is best currently.

[asjk169]

On 5/11/2023 at 4:45 PM, asjk169 said:

 

My hardware platform is ERYING 12500H+G660, and the plugin isn't working on my platform.

 

During the installation, the system log shows 

May 10 19:53:33 Tower root: plugin: creating: /boot/config/plugins/i915-sriov//unraid-i915-sriov-2023.04.06.txz - downloading from URL https://raw.githubusercontent.com/zhtengw/unraid-i915-sriov/master/packages/unraid-i915-sriov-2023.04.06.txz
May 10 19:53:34 Tower root: plugin: running: /boot/config/plugins/i915-sriov//unraid-i915-sriov-2023.04.06.txz
May 10 19:53:34 Tower root: plugin: running: anonymous
May 10 19:53:34 Tower root: plugin: running: anonymous
May 10 19:53:40 Tower kernel: i915 0000:00:02.0: driver does not support SR-IOV configuration via sysfs
May 10 19:53:40 Tower root: plugin: i915-sriov.plg installed

 

before reboot, the plugin page shows

 

after reboot, the plugin page shows

 

 

I paste the system diagnostics before reboot and after reboot here. Does this is the problem of drive not supporting my devices?However, hardware decode works fine in jellyfin docker container. Thanks for paying attention.

 

tower-diagnostics-20230511-1451.zip 64.04 kB · 0 downloads tower-diagnostics-20230511-1455.zip 64.51 kB · 0 downloads

 

Problem solved. It needs enable "Above 4G MMIO BIOS assignment" in BIOS. 

[jyueh]

Jellyfin and Win10 are both working properly, but this plugin may cause Intel GPU top to fail to display call situations and have more serious compatibility issues, resulting in the Dynamix S3 Sleep plugin not being able to sleep. thank.

[Hawxxer]

While the Intel 770 from my 13600k is being recognized in Windows 11 pro and the newest arc drivers are installed, parsec won't launch because of error -15000 - no hardware video encoding supported. Sunshine works kind of if you install a virtual display called IDDSampledriver from github, but is a hazzle to maintain, because it somehow breakes the vnc driver and if sunshine is not started correctly the vm is unusable as the vnc (if it works) is the second monitor where you cant really do anything. Also duplicating display does not work.

[domrockt]

Hello there, FYI 

 

6.12.0-rc7 broke the Plugin, idk why and reverted to 6.12.0-rc6. 

 

thank you much fpr this Plugin iam having a blast with sriov and Parsec

Edited June 7, 2023 by domrockt

[Vipa]

On 6/7/2023 at 10:37 AM, domrockt said:

Hello there, FYI 

 

6.12.0-rc7 broke the Plugin, idk why and reverted to 6.12.0-rc6. 

 

thank you much fpr this Plugin iam having a blast with sriov and Parsec

I set up my new intel 13700 box with rc7 over the weekend because I wanted zfs pools and the plugin didn't work for me either. RC8 didn't even have a compatible version at first (duh), but as I looked for it just now, it got added and seems to run fine.

 

 

 

Thank you for the plugin!

Edited June 12, 2023 by Vipa

[domrockt]

@zhtengw

 

FYI

 

i have this error this Morning after the update to 12.1

 

----------Downloading i915-sriov module Package for kernel v6.1.34---------- ---------This could take some time, please don't close this window!----------- --------Can't download i915-sriov module Package for kernel v6.1.34----------- plugin: run failed: '/bin/bash' returned 1 Executing hook script: post_plugin_checks

Edited June 26, 2023 by domrockt

[Shine]

大佬好，更新6.12.1后，只要启动虚拟机就会报下面这个错误，导致虚拟机没办法自动启动。6.11.5正常运行

VM Autostart disabled: 27-06-2023 22:06

vfio-pci-errors
VM Autostart disabled due to vfio-bind error

Edited June 27, 2023 by Shine

[zhtengw]

7 minutes ago, Shine said:

大佬好，更新6.12.1后，只要启动虚拟机就会报下面这个错误，导致虚拟机没办法自动启动。6.11.5正常运行

VM Autostart disabled: 27-06-2023 22:06

vfio-pci-errors
VM Autostart disabled due to vfio-bind error

这是因为系统在加载插件驱动之前先执行了检查vfio绑定的脚本，在设备页面取消绑定就好了

[Shine]

11 minutes ago, zhtengw said:

这是因为系统在加载插件之前驱动先执行了检查vfio绑定的脚本，在设备页面取消绑定就可以了

好的 谢谢大佬

[Crunch41]

Hi, Running unraid version 6.12.2 with an Intel® Core™ i5-13600K but I'm not having any luck since installing the plugin. The plugin settings list SR-IOV as supported but I don't have any vf's available to use. I've installed the plugin and rebooted with no luck.

 

Here's my output of lspci -vs 00:02.0

00:02.0 VGA compatible controller: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] (rev 04) (prog-if 00 [VGA controller])
        DeviceName: Onboard - Video
        Subsystem: Gigabyte Technology Co., Ltd Device d000
        Flags: bus master, fast devsel, latency 0, IRQ 151, IOMMU group 0
        Memory at 41000000 (64-bit, non-prefetchable)
        Memory at 50000000 (64-bit, prefetchable)
        I/O ports at 5000
        Expansion ROM at 000c0000 [virtual] [disabled]
        Capabilities: [40] Vendor Specific Information: Len=0c <?>
        Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
        Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit-
        Capabilities: [d0] Power Management version 2
        Capabilities: [100] Process Address Space ID (PASID)
        Capabilities: [200] Address Translation Service (ATS)
        Capabilities: [300] Page Request Interface (PRI)
        Capabilities: [320] Single Root I/O Virtualization (SR-IOV)
        Kernel driver in use: i915
        Kernel modules: i915

 

 

 

here's the output of dmesg | grep i915

[   70.146936] i915: loading out-of-tree module taints kernel.
[   70.228321] i915 0000:00:02.0: [drm] VT-d active for gfx access
[   70.228338] i915 0000:00:02.0: vgaarb: deactivate vga console
[   70.228519] i915 0000:00:02.0: [drm] Using Transparent Hugepages
[   70.229118] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[   70.230389] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915])
[   70.231056] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adls_dmc_ver2_01.bin (v2.1)
[   70.322006] i915 0000:00:02.0: [drm] Missing GuC-Err-Cap reglist Class(1):Compute(4)!
[   70.322008] i915 0000:00:02.0: [drm] Missing GuC-Err-Cap reglist Instance(2):Compute(4)!
[   70.323120] i915 0000:00:02.0: [drm] Missing GuC-Err-Cap reglist Class(1):Compute(4)!
[   70.323121] i915 0000:00:02.0: [drm] Missing GuC-Err-Cap reglist Instance(2):Compute(4)!
[   70.323521] i915 0000:00:02.0: [drm] GuC firmware i915/tgl_guc_70.bin version 70.5.1
[   70.323522] i915 0000:00:02.0: [drm] HuC firmware i915/tgl_huc.bin version 7.9.3
[   70.323954] i915 0000:00:02.0: [drm] Missing GuC-Err-Cap reglist Class(1):Compute(4)!
[   70.323955] i915 0000:00:02.0: [drm] Missing GuC-Err-Cap reglist Instance(2):Compute(4)!
[   70.326961] i915 0000:00:02.0: [drm] HuC authenticated
[   70.327409] i915 0000:00:02.0: [drm] GuC submission enabled
[   70.327410] i915 0000:00:02.0: [drm] GuC SLPC enabled
[   70.327779] i915 0000:00:02.0: [drm] GuC RC: enabled
[   70.328522] mei_pxp 0000:00:16.0-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:00:02.0 (ops i915_pxp_tee_component_ops [i915])
[   70.328578] i915 0000:00:02.0: [drm] Protected Xe Path (PXP) protected content support initialized
[   70.329367] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
[   70.341265] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[   70.341387] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes

 

and the output of cat /sys/devices/pci0000:00/0000:00:02.0/sriov_totalvfs just returns 0.

 

Any idea what might be wrong here?

 

Thanks!

 

 

Edited July 13, 2023 by Crunch41
added more info

[letrain]

unraid 6.12.3 was just released. it broke sr-iov. worked fine in 6.12.2. 

6.12.3 - 

"Docker

This release resolves an issue where Docker does not properly stop when the array Stops, which can result in an unclean shutdown.

If Docker containers have issues starting after a while, and you are running Plex, go to your Plex Docker container settings, switch to advanced view, and add this to the Extra Params: --no-healthcheck"

 

I had the this problem with dockers and unclean shutdown, and disks not unmounting so upgraded. after upgrading sr-iov was missing. after trying to reinstall it this is the dialog. it appears in settings again, but as soon as i try to save config, and enable it doesn't work. 

 

thanks

 

EDIT:

found this in the logs

"Jul 15 12:17:33 Tower kernel: i915 0000:00:02.0: driver does not support SR-IOV configuration via sysfs"

not sure what happened between 6.12.2 and 6.12.3. its weird that sr-iov is in settings, but not on the plugin page. and "apps" still gives me the option to install, but fails.

Edited July 15, 2023 by letrain
log located

[mkkk]

What is VF numbers? What should I select?

 

 

I can output video with my Ubuntu VM, but not Windows

 

Unraid 6.12.2

 

Edited July 18, 2023 by mkkk

[Cheater]

On 7/18/2023 at 1:11 AM, mkkk said:

What is VF numbers? What should I select?

 

I have always assumed that it is the number of how many Virtual Functions you want to split your video card into. In other words, you will divide your graphics card into 1 to 7 individual graphic cards, however, you loose the ability to output video through a physical connector** that is attached to that video card.  So you can have graphical computing power in 1 to 7 Virtual Machines, which you use in a particular way that makes sense to not use a video output over a cable.  

I'm not sure if the computing power is evenly distributed over the amount of VF's you select. (I think) It should be evenly distributed, but it depends on the implementation of the manufacturer. Maybe you can use a benchmark tool to know for sure.

 

** The official specification of sr-iov supports up to 4 physical outputs (according to Intel), but I think it's not implemented yet.

I've used sr-iov but since I loose the complete physical output, it seems useless to me.

[mikeyosm]

Just stumbled across this thread and having come from a Nvidia Tesla P4 on ESXi I am keen to test vf sr-iov.

Does this plugin work with 13th gen Intel CPUs and latest UNRAID build 6.12.3 ? I plan to split the iGPU so I can use it in VMs and Dockers, is it possible to do this or must the igpu be divided across docker containers or VMs but not both?