[Plugin] Intel-GVT-g

StatMatt · July 6, 2022

No PCIe slots available in the server to add a GPU, I'm afraid. But I have a i5-6500 computer around that isn't getting much use so I'll just try going the bare-metal route. Thanks so much for all of the help!

m4rcus · August 8, 2022

quick question, would this work on windows 11 as well? didnt find anything in that regard yet

alturismo · August 8, 2022

5 minutes ago, m4rcus said:

quick question, would this work on windows 11 as well? didnt find anything in that regard yet

yes

ApfelBirneKreis · August 17, 2022

Can we also add SR-IOV Support forr 11th and 12th gen igpus? I dont know how to do sr-iov on unraid.

ich777 · August 17, 2022

1 minute ago, ApfelBirneKreis said:

Can we also add SR-IOV Support forr 11th and 12th gen igpus? I dont know how to do sr-iov on unraid.

This has nothing to do with GVT-g...

This needs to be supported by Unraid or better speaking QEMU.

I would recommend that you create a post on the Feature Request subforums.

Andrea3000 · August 22, 2022

Hi @ich777

Thank you for having provided this plug-in.

I have a Core i3-9100 on a Gigabyte C246M-WU4 with 64GB of ram.

In the BIOS I’ve set the “Aperture Size” to 2048MB and vram allocation set to MAX.

In Unraid I can confirm that my aperture size is indeed set to 2GB

Memory at 6013000000 (64-bit, non-prefetchable) [size=16M]
Memory at 4000000000 (64-bit, prefetchable) [size=2G]

Based on this table from Intel:

y	aperture memory size		hidden memory size		fence size		weight

-------------------------------------------------------------------------------------------------------------

8	64				384				4			2

4	128				512				4			4

2	256				1024				4			8

1	512				2048				4			16

with an aperture size of 2GB I should be able to create at least 3 vGPUs with GVTg_V5_1, as each one should require 512MB of aperture size.

However, with no other VM or Docker container running, I can’t start even a single VM because gvt complains that it requires 2048MB of vram and I only have about 1600MB free.

Do you know why gvt is asking for so much vram?

If I reduce the weight of the vGPU to GVTg_V5_2 I can start the VM correctly and Windows 10 reports 4GB of vram available in total.

Am I missing something?

ich777 · August 23, 2022

9 hours ago, Andrea3000 said:

Am I missing something?

The lack of interest by most Motherboard manufacturers and the maybe not quite right implementation from the aperture size (even Intel boards have that issue) in the BIOS is most certainly what is causing this on your system.

GVT-g is not very popular by any means and most manufacturers are not really interested in supporting it properly or updating their BIOS so that everything is working.

@alturismo do you have any other thoughts about this?

Andrea3000 · August 23, 2022

3 minutes ago, ich777 said:

The lack of interest by most Motherboard manufacturers and the maybe not quite right implementation from the aperture size (even Intel boards have that issue) in the BIOS is most certainly what is causing this on your system.

GVT-g is not very popular by any means and most manufacturers are not really interested in supporting it properly or updating their BIOS so that everything is working.

@alturismo do you have any other thoughts about this?

Thanks for the reply.

I don’t know enough about this to say whether my motherboard implements the aperture size correctly or not.

But when I changed the aperture size from its default value of 256MB to 2048MB I got confirmation from Unraid that the value has changed correctly.

Having approx 1600MB of vram available (as my system does) seems to be in the right ballpark.

What is strange is that the requirement to assign a GVTg_V5_1 is 4 times what it should be (2048MB instead of 512MB)

ich777 · August 23, 2022

5 minutes ago, Andrea3000 said:

What is strange is that the requirement to assign a GVTg_V5_1 is 4 times what it should be (2048MB instead of 512MB)

Please also look at this GitHub Issue here: Click

Do you boot Unraid in GUI Mode?

alturismo · August 23, 2022

24 minutes ago, ich777 said:

@alturismo do you have any other thoughts about this?

16 minutes ago, Andrea3000 said:

What is strange is that the requirement to assign a GVTg_V5_1 is 4 times what it should be (2048MB instead of 512MB)

from what i see its all looking good on his side with aperture size, and as its also showing up he should be capable todo what he wants to.

im here sadly stuck on 256mb as asrock and msi cant handle it properly on linux on the boards im using, even with 1GB assigned in bios but only 256 coming on ...

may worth a try, play with the GUC/HUC firmware loadings, but be aware it could freeze the system if the VM crashes

image.png.bc561c489ef30d695095c4f16793f99a.png

Andrea3000 · August 23, 2022

13 hours ago, alturismo said:

from what i see its all looking good on his side with aperture size, and as its also showing up he should be capable todo what he wants to.

im here sadly stuck on 256mb as asrock and msi cant handle it properly on linux on the boards im using, even with 1GB assigned in bios but only 256 coming on ...

may worth a try, play with the GUC/HUC firmware loadings, but be aware it could freeze the system if the VM crashes

Thanks @alturismo,

I tried with both Guc/Huc disabled (mode 0) and mode 3 but it makes no difference.

In both cases I get the same error when I try to launch the VM with a GVTg_V5_1:

gvt: fail to allocate resource high GM space
gvt: request 2048MB avail 1664MB max 1664MB taken 0MB
gvt: failed to create intel vgpu: -28

Based on the Intel spec, the GVTg_V5_1 mode requires 512MB of low GM space and 2048MB of high GM space.

I have no idea what the low and high GM spaces mean, but I seem to remember having read that the low GM space has to be less than the aperture size and that the host alone will use 128MB of the available aperture size.

As the error clearly states, my system doesn't seem to have enough memory for 2048MB of high GM space. This raises 2 questions:

1) Where is that 1664MB coming from? The only settings in the bios related to VRAM are:

- Aperture size (set to 2048MB)

- DVMT Pre-allocated memory (set to 1024MB which is the maximum)

- DVMT total Gfx Mem (set to MAX). Note that other options are 128MB and 256MB. I tried to set it to 256MB but the error message still reports 1664MB available, which means that this parameter isn't linked with that memory.

2) How can someone create a GVTg_V5_4 vGPU with only 256MB of aperture size if the high GM space is 512MB? Does it mean that the high GM space has nothing to do with aperture size?

EDIT:

I was inspecting the portion of the log file that relates to GVT-g

Aug 23 21:49:38 Jarvis root: -----------Found Intel GVT-g Plugin!-----------
Aug 23 21:49:38 Jarvis root: ---GVT-g Plugin will enable the i915 Module!---
Aug 23 21:49:38 Jarvis root: 
Aug 23 21:49:38 Jarvis root: ----Installation of Intel GPU TOP complete-----
Aug 23 21:49:38 Jarvis root: plugin: intel-gpu-top.plg installed
Aug 23 21:49:38 Jarvis root: plugin: installing: /boot/config/plugins/intel-gvt-g.plg
Aug 23 21:49:38 Jarvis root: plugin: running: anonymous
Aug 23 21:49:38 Jarvis root: plugin: skipping: /boot/config/plugins/intel-gvt-g/gvt.g-2022.05.02.txz already exists
Aug 23 21:49:38 Jarvis root: plugin: running: /boot/config/plugins/intel-gvt-g/gvt.g-2022.05.02.txz
Aug 23 21:49:38 Jarvis root: 
Aug 23 21:49:38 Jarvis root: +==============================================================================
Aug 23 21:49:38 Jarvis root: | Installing new package /boot/config/plugins/intel-gvt-g/gvt.g-2022.05.02.txz
Aug 23 21:49:38 Jarvis root: +==============================================================================
Aug 23 21:49:38 Jarvis root: 
Aug 23 21:49:38 Jarvis root: Verifying package gvt.g-2022.05.02.txz.
Aug 23 21:49:38 Jarvis root: Installing package gvt.g-2022.05.02.txz:
Aug 23 21:49:38 Jarvis root: PACKAGE DESCRIPTION:
Aug 23 21:49:38 Jarvis root: Package gvt.g-2022.05.02.txz installed.
Aug 23 21:49:38 Jarvis root: plugin: creating: /usr/local/emhttp/plugins/intel-gvt-g/README.md - from INLINE content
Aug 23 21:49:38 Jarvis root: plugin: running: anonymous
Aug 23 21:49:38 Jarvis root: 
Aug 23 21:49:38 Jarvis root: -----------Intel GVT-g already found, nothing to do------------
Aug 23 21:49:38 Jarvis root: 
Aug 23 21:49:38 Jarvis root: -----------------Loading Kernel Module 'i915'------------------
Aug 23 21:49:38 Jarvis kernel: Linux agpgart interface v0.103
Aug 23 21:49:38 Jarvis kernel: Setting dangerous option force_probe - tainting kernel
Aug 23 21:49:38 Jarvis kernel: i915 0000:00:02.0: [drm] VT-d active for gfx access
Aug 23 21:49:38 Jarvis kernel: checking generic (4000000000 300000) vs hw (6001000000 1000000)
Aug 23 21:49:38 Jarvis kernel: checking generic (4000000000 300000) vs hw (4000000000 80000000)
Aug 23 21:49:38 Jarvis kernel: fb0: switching to i915 from EFI VGA
Aug 23 21:49:38 Jarvis kernel: Console: switching to colour dummy device 80x25
Aug 23 21:49:38 Jarvis kernel: i915 0000:00:02.0: vgaarb: deactivate vga console
Aug 23 21:49:38 Jarvis kernel: i915 0000:00:02.0: Direct firmware load for i915/gvt/vid_0x8086_did_0x3e91_rid_0x00.golden_hw_state failed with error -2
Aug 23 21:49:38 Jarvis kernel: i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
Aug 23 21:49:38 Jarvis kernel: i915 0000:00:02.0: [drm] [ENCODER:110:DDI C/PHY C] is disabled/in DSI mode with an ungated DDI clock, gate it
Aug 23 21:49:38 Jarvis kernel: i915 0000:00:02.0: [drm] [ENCODER:114:DDI E/PHY E] is disabled/in DSI mode with an ungated DDI clock, gate it
Aug 23 21:49:38 Jarvis kernel: i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/kbl_dmc_ver1_04.bin (v1.4)
Aug 23 21:49:38 Jarvis kernel: [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
Aug 23 21:49:38 Jarvis kernel: ACPI: video: Video Device [GFX0] (multi-head: yes  rom: no  post: no)
Aug 23 21:49:38 Jarvis kernel: input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00/input/input10
Aug 23 21:49:39 Jarvis kernel: fbcon: i915drmfb (fb0) is primary device
Aug 23 21:49:39 Jarvis kernel: Console: switching to colour frame buffer device 240x75
Aug 23 21:49:39 Jarvis kernel: i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
Aug 23 21:49:39 Jarvis kernel: i915 0000:00:02.0: MDEV: Registered
Aug 23 21:49:40 Jarvis root: 
Aug 23 21:49:40 Jarvis root: -------------Installation of Intel GVT-g successful--------------
Aug 23 21:49:40 Jarvis root: 
Aug 23 21:49:40 Jarvis root: 
Aug 23 21:49:40 Jarvis root: -----------------------------------------------------------------
Aug 23 21:49:40 Jarvis root: -----------------Intel GVT-g plugin installed--------------------
Aug 23 21:49:40 Jarvis root: -----------------------------------------------------------------
Aug 23 21:49:40 Jarvis root: plugin: intel-gvt-g.plg installed

and I saw this error:

i915 0000:00:02.0: Direct firmware load for i915/gvt/vid_0x8086_did_0x3e91_rid_0x00.golden_hw_state failed with error -2

Do you know what it means? Could this be linked with the issue I'm seeing? At the moment the GuC/Huc firmware loading is disabled.

EDIT2:

Apparently that error message isn't an issue: https://github.com/intel/gvt-linux/pull/215

Edited August 23, 2022 by Andrea3000

ich777 · August 24, 2022

10 hours ago, Andrea3000 said:

Apparently that error message isn't an issue: https://github.com/intel/gvt-linux/pull/215

I think Intel has lost slowly but surely interest in GVT-g anyways because it is too complicated to support and also depends on the Motherboard manufacturers that everything is implemented correctly and will silently let it die in favor to SR-IOV which is supported by newer platforms.

10 hours ago, Andrea3000 said:

This raises 2 questions

I think it would be best to ask on the GitHub Issue tracker from GVT-g over here: Click

10 hours ago, Andrea3000 said:

- DVMT Pre-allocated memory (set to 1024MB which is the maximum)

Isn't this the cause of the issue because you reserve 1024MB of available VRAM to the iGPU or do I get this wrong?

It would be best to let this sit at 256 or the default value for your Motherboard because this is Pre-allocated memory and it will allocated more if it needs to - of course if it's available.

10 hours ago, Andrea3000 said:

DVMT total Gfx Mem

Also see here:

Quote

When DVMT Memory Size is set to 0MB and this BIOS feature set to 128 MB, a total of 128 MB of system memory is allocated as graphics memory in the Fixed mode. If DVMT Memory Size is set to any value other than 0 MB, then the onboard graphics processor will operate in the Fixed + DVMT Memory mode.

Source

Andrea3000 · August 24, 2022

14 hours ago, ich777 said:

I think Intel has lost slowly but surely interest in GVT-g anyways because it is too complicated to support and also depends on the Motherboard manufacturers that everything is implemented correctly and will silently let it die in favor to SR-IOV which is supported by newer platforms.

I think it would be best to ask on the GitHub Issue tracker from GVT-g over here: Click

Isn't this the cause of the issue because you reserve 1024MB of available VRAM to the iGPU or do I get this wrong?

It would be best to let this sit at 256 or the default value for your Motherboard because this is Pre-allocated memory and it will allocated more if it needs to - of course if it's available.

Also see here:

Source

Ok, I managed to make it work.

I found this paper: https://ieeexplore.ieee.org/ielaam/71/8410754/8247267-aam.pdf

that shows this diagram:

together with this description:

Quote

Graphics translation table (GTT), sometimes known as global graphics translation table, is a page table providing the translations from logical graphics memory address to physical memory address, as Figure 1 shows. The physical memory space served by GTT is called global graphics memory space, which is used by all GPU components, such as the render engine and the display engine. According to the architecture of Intel GPU, GTT is a unique global register of GPU which resides in MMIO (memory map- ping input/output) range. The CPU does not access the global graphics memory space directly. However, through the Aperture [8], which is an MMIO range, CPU could also access the global graphics memory space. This CPU’s visible part of global graphics memory is called low global graphics memory, while the rest part is called high global graphics memory (high GM) or hidden global graphics memory (hidden GM). Note that the mapping between low GM and aperture is setup by the hardware directly. To be specific, Intel GPU has a 2MB GTT which maps to a 2GB graphics memory space. The aperture range could maximally be 512KB which maps to 512MB graphics memory space visible by CPU. Accordingly, the low GM is 512MB, while the high GM is 1536MB. Besides the graphics translation table, there is another type of GPU page tables called per-process graphics translation table (PPGTT) which provides each process its own local graphics memory space. Unlike GTT, PPGTT resides in the main memory. Sometimes, GPU also uses a group of fence registers to access global graphics memory. Each Intel GPU only has 32 fence registers.

This got me thinking. If the VRAM that the GPU can address is the sum of low + high memory, if I set an aperture size that is too large, there might not be enough high memory left.

I tried to reduce the aperture size in the BIOS to 1GB (from 2GB) and the VM started without issues with a GVTg_V5_1 vGPU.

I don't have any other VM set at this time therefore I don't know how many more vGPU I can create. Considering that the host occupies 128MB of low memory/aperture, there should be 384MB left.

Depending on how much high memory I got left, I might be able to create also a GVTg_V5_2 vGPU and a GVTg_V5_4.

I haven't found anywhere a command in linux that can show the VRAM usage therefore it is going to be trial and error I suppose.

PS: I tried to play with DVMT pre-allocation but it didn't make a difference.

Thank you very much for your help.

Andrea3000 · August 25, 2022

Quick update on some tests I've done, in case it is of use to somebody else.

It appears that the total GPU memory that GVT-g can address on my system is 3712MB.

This gets split between low GM space (a.k.a. Aperture Size) and high GM space.

With Aperture Size set to 1024MB, the system has the remaining 2688MB as high GM space.

I tried successfully multiple combinations of VMs with different vGPU modes.

Combo 1:

- 1x GVTg_V5_1

- 1x GVTg_V5_4

Low GM size used: 512MB + 128MB + 128MB (host) = 768MB

High GM size used: 2048MB + 512MB = 2560MB

Combo 2:

- 2x GVTg_V5_2

- 1x GVTg_V5_4

Low GM size used: 2x256MB + 128MB + 128MB (host) = 768MB

High GM size used: 2x1024MB + 512MB = 2560MB

I haven't tried more than 3 VMs in parallel but I'm expecting that 5x GVTg_V5_4 should also be possible.

For my particular system, setting an Aperture Size larger than 1GB not only is useless but hurts because it leaves so little high GM space that I can't even run a single GVTg_V5_1

Thanks again to @ich777 and @alturismo for your help!

Edited September 3, 2022 by Andrea3000

reinv · August 27, 2022

Hi there, I have an GK50 minisforum computer with an Pentium N5030. The plugins says "GVT-g not supported" whilst the cpu is a 9th generation CPU. Any thoughts?

ich777 · August 27, 2022

20 minutes ago, reinv said:

Any thoughts?

Only i-Series and Xeon series are supported like mentioned on the plugin page.

reinv · August 28, 2022

Ah ok, check, thanks.

xmoffice · September 12, 2022

@ich777

Hello, I'm sorry to disturb you. But I really need Intel gvt-g, a plug-in that can share GPU. Unfortunately, I have been unable to install. Please give me guidance or solutions.

Hello, I'm sorry to disturb you. But I really need Intel gvt-g, a plug-in that can share GPU. Unfortunately, I have been unable to install. Please give me guidance or solutions

My unread hardware configuration: Lenovo ib250mh motherboard, CPU g4560.

Vtx, VTD, core display IgD have been set in BIOS, and 1024m video memory has been allocated.

Model: Custom
M/B: LENOVO 3102 Version SDK0L77767 WIN 3423478701079 - s/n:
BIOS: LENOVO Version M16KT68A. Dated: 12/27/2021
CPU: Intel® Pentium® CPU G4560 @ 3.50GHz
HVM: Enabled
IOMMU: Enabled
Cache: 128 KiB, 512 KiB, 3 MB
Memory: 20 GiB DDR4 (max. installable capacity 32 GiB)
Network: bond0: fault-tolerance (active-backup), mtu 1500 
 eth0: 1000 Mbps, full duplex, mtu 1500
Kernel: Linux 5.10.28-Unraid x86_64
OpenSSL: 1.1.1j
Uptime:

The installation results are as follows:

plugin: installing: https://raw.githubusercontent.com/ich777/intel-gvt-g/master/intel-gvt-g.plg
plugin: downloading https://raw.githubusercontent.com/ich777/intel-gvt-g/master/intel-gvt-g.plg
plugin: downloading: https://raw.githubusercontent.com/ich777/intel-gvt-g/master/intel-gvt-g.plg ... done
plugin: downloading: https://raw.githubusercontent.com/ich777/intel-gvt-g/master/packages/gvt.g-2022.05.02.txz ... done

+==============================================================================
| Skipping package gvt.g-2022.05.02 (already installed)
+==============================================================================


--------------------Downloading Intel GVT-g----------------------
---This could take some time, please don't close this window!----

-------------------Can't download Intel GVT-g---------------------
plugin: run failed: /bin/bash retval: 1

Updating Support Links

Run Intel gvt-g plug-in in the settings, and the prompt is as follows

GVT-g not enabled in Kernel Module i915
Please make sure that the file '/boot/config/modprobe.d/i915.conf' exist and contains the line 'blacklist i915' (without quotes).
After ensuring that the file exist and contains the above content reboot your server!

I have tried to restart or delete i915.conf or add blacklist i915, but I still can't install it successfully.

After restarting, the plug-in will not be displayed in the settings.

ich777 · September 12, 2022

16 minutes ago, xmoffice said:

I have been unable to install.

Seems like the plugin isn't able to download the package for your Unraid version.

On what Unraid version are you? Can you please upload your Diagnostics?

xmoffice · September 12, 2022

7 minutes ago, ich777 said:

Seems like the plugin isn't able to download the package for your Unraid version.

On what Unraid version are you? Can you please upload your Diagnostics?

Thank you very much for your prompt reply. This is my diagnosis. Waiting for your reply online, thank you

tower-diagnostics-20220912-1519.zip

xmoffice · September 12, 2022

2 hours ago, ich777 said:

Seems like the plugin isn't able to download the package for your Unraid version.

On what Unraid version are you? Can you please upload your Diagnostics?

@ich777

My diagnosis record has been uploaded. Is there a solution?

Edited September 12, 2022 by xmoffice

ich777 · September 12, 2022

7 hours ago, xmoffice said:

My diagnosis record has been uploaded. Is there a solution?

https://unraid.net/buy-genuine-license

oldAzzBstrd · September 12, 2022

Hi ! Is there a way to get it work on an i7 4970 ?

alturismo · September 13, 2022

6 hours ago, oldAzzBstrd said:

Hi ! Is there a way to get it work on an i7 4970 ?

1st Post ... 5th - 10th gen, your CPU is a 4th gen haswell when i remember correctly ...

oldAzzBstrd · September 18, 2022

On 9/13/2022 at 6:20 AM, alturismo said:

1st Post ... 5th - 10th gen, your CPU is a 4th gen haswell when i remember correctly ...

Yes, i`ve seen it. 1st Post is a year old.... Thats`s why i asked. Maybe there`s a workaround in the meantime to get it work on an 4th gen.

Edited September 18, 2022 by oldAzzBstrd

[Plugin] Intel-GVT-g

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

ich777

ich777

ich777

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation