[PLUGIN] GPU Statistics


Recommended Posts

17 hours ago, SimonF said:

Do you get any messages?

 

root@computenode:~# dmesg |grep i915
[   50.985890] i915 0000:00:02.0: [drm] VT-d active for gfx access
[   50.985939] i915 0000:00:02.0: [drm] Using Transparent Hugepages
[   50.986572] i915 0000:00:02.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=io+mem
[   50.992549] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adls_dmc_ver2_01.bin (v2.1)
[   51.021951] i915 0000:00:02.0: [drm] GT0: GuC firmware i915/tgl_guc_70.bin version 70.13.1
[   51.021955] i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
[   51.035422] i915 0000:00:02.0: [drm] GT0: HuC: authenticated for all workloads
[   51.035427] i915 0000:00:02.0: [drm] GT0: GUC: submission disabled
[   51.035429] i915 0000:00:02.0: [drm] GT0: GUC: SLPC disabled
[   51.036259] i915 0000:00:02.0: [drm] Protected Xe Path (PXP) protected content support initialized
[   52.537318] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 1
[   52.539040] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[   52.539530] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[   52.539600] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[   52.674124] mei_pxp 0000:00:16.0-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:00:02.0 (ops i915_pxp_tee_component_ops [i915])
[   52.677122] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_ops [i915])

~# dmesg |grep i915
[    8.365381] i915 0000:00:02.0: [drm] VT-d active for gfx access
[    8.376517]  i915_driver_probe+0x83f/0xc11 [i915]
[    8.383725]  i915_init+0x1f/0x7f [i915]
[    8.391432] Modules linked in: x86_pkg_temp_thermal intel_powerclamp coretemp i915(+) kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel sha512_ssse3 sha256_ssse3 ast sha1_ssse3 iosf_mbi drm_vram_helper drm_buddy aesni_intel i2c_algo_bit drm_ttm_helper drm_display_helper crypto_simd ttm cryptd drm_kms_helper rapl intel_cstate mei_hdcp mei_pxp drm i2c_i801 intel_gtt tpm_crb ipmi_ssif nvme i2c_smbus wmi_bmof tpm_tis ahci mei_me intel_uncore agpgart cdc_ether syscopyarea sr_mod input_leds sysfillrect usbnet video sysimgblt acpi_ipmi i2c_core igc joydev led_class cdrom mei nvme_core mii libahci corsair_psu vmd fb_sys_fops thermal fan tpm_tis_core wmi ipmi_si tpm backlight intel_pmc_core acpi_pad acpi_tad button unix

 

 

~# intel_gpu_top
No device filter specified and no discrete/integrated i915 devices found

 

 

SYSLOG---

 

image.thumb.png.02bfcbd60a7c2b597535b6e846868950.png

Edited by mikeyosm
Link to comment
6 hours ago, mikeyosm said:

~# dmesg |grep i915
[    8.365381] i915 0000:00:02.0: [drm] VT-d active for gfx access
[    8.376517]  i915_driver_probe+0x83f/0xc11 [i915]
[    8.383725]  i915_init+0x1f/0x7f [i915]
[    8.391432] Modules linked in: x86_pkg_temp_thermal intel_powerclamp coretemp i915(+) kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel sha512_ssse3 sha256_ssse3 ast sha1_ssse3 iosf_mbi drm_vram_helper drm_buddy aesni_intel i2c_algo_bit drm_ttm_helper drm_display_helper crypto_simd ttm cryptd drm_kms_helper rapl intel_cstate mei_hdcp mei_pxp drm i2c_i801 intel_gtt tpm_crb ipmi_ssif nvme i2c_smbus wmi_bmof tpm_tis ahci mei_me intel_uncore agpgart cdc_ether syscopyarea sr_mod input_leds sysfillrect usbnet video sysimgblt acpi_ipmi i2c_core igc joydev led_class cdrom mei nvme_core mii libahci corsair_psu vmd fb_sys_fops thermal fan tpm_tis_core wmi ipmi_si tpm backlight intel_pmc_core acpi_pad acpi_tad button unix

 

 

~# intel_gpu_top
No device filter specified and no discrete/integrated i915 devices found

 

 

SYSLOG---

 

image.thumb.png.02bfcbd60a7c2b597535b6e846868950.png

Adding echo "blacklist ast" > /boot/config/modprobe.d/ast.conf has helped.

Not sure why I am having to blacklist the ASPEED driver though. Thanks to @jakea333 and @JorgeB for pointing me in the right direction.

Link to comment
35 minutes ago, J05u said:

I am trying to get some stats from A380, but GPU statistics shows me this -  

Vendor command returned no data.

Unraid 6.12.8 doesn‘t support ARC GPUs, you have to wait for the next beta/RC release with a Kernel newer than 6.1.x

  • Thanks 1
Link to comment
On 2/17/2024 at 9:48 PM, kjake said:

For the "Vendor data valid, but not enough received." folks that are also Intel iGPUs, try this in the GPU Stats configuration UI...

 

Set (or confirm) that "UI Automatic Refresh / Interval (Milliseconds):" is not lower than 1000.

Set "Intel_top_command_timeout:" to "1 .9"

 

My system is getting older (Coffee Lake / Gen8) and I've noticed that the stats panel is having trouble when my system is really bogged down.

 

Hope this helps someone!

I used these on my 12th gen 12600k.  It stopped the message from appearing.  That said, I still think something weird is going on, as now I don't get any actual activity, everything just sits at 0%.  If I try to use the intel top command, everything is at 0% as well.  When I use Plex, I can see on the dashboard it is using hardware to transcode.

Link to comment

I just switched to using a Intel iGPU and I noticed that the plugin dosen't show any icon for the docker that is utilizing the transcoder anymore? (Before I had a small Plex icon below on the dashboard) Is this just a NVIDIA feature? or am I missing a setting or plugin addon?

Link to comment
On 1/26/2024 at 6:07 PM, Conan the Barbarian said:

Good morning,

 

installing now the pugin in a new machine, with Intel i5-13600T alder-lake-s.

All working fine, except IMC bus uutilization, showing N/A.

Is that normal? (In previous generations that value was always showing something. I include also the aspect with a i5-9400T).

Thanks

imagen.png.fc36f6aa345ab5dc0b5049abb983fa28.png

Captura de pantalla 2024-01-26 110754.png

 

Maybe your GPU status is normal.

It's great that your GPU statistics plugin have already display some information.

And the CPU of my unraid is same to you,  also 13th Gen Intel® Core™ i5-13600T.

But the GPU statistics plugin cannot display anything though I had tried my best.

Would you like to help me find where is wrong? Thanks.

 

image.png.a7614b8becf7c19461ab617bb133be29.png

image.png.0f23699cabc11f838c5590b72d252453.png

image.thumb.png.c108d0f08160361cfb0ea7611e344a04.png

image.png.b9b464e67d46c4b2f46809eae00116d3.png

Link to comment
10 minutes ago, nickniu001 said:

But the GPU statistics plugin cannot display anything though I had tried my best.

Everything seems alright, what is the exact issue?

Keep in mind that your GPU will only show utilization when the iGPU is actually in use.

It is a known issue that no IMC Bus utilization is shown on 13th Gen+

 

Mine looks almost the same when nothing is using the iGPU:

grafik.png.176f8d830a6e33711f1e24cda93eb016.png

 

When I put some load on it it looks like that:
grafik.png.b577ab4b23826a0297e8b94a3335bb7f.png

Link to comment
16 minutes ago, ich777 said:

Everything seems alright, what is the exact issue?

Keep in mind that your GPU will only show utilization when the iGPU is actually in use.

It is a known issue that no IMC Bus utilization is shown on 13th Gen+

 

Mine looks almost the same when nothing is using the iGPU:

grafik.png.176f8d830a6e33711f1e24cda93eb016.png

 

When I put some load on it it looks like that:
grafik.png.b577ab4b23826a0297e8b94a3335bb7f.png

 

There is running something on my Unraid now, and the sound of Fans is loudly.

 

But the GPU still display nothing . I don't know its reason.

 

eg. the jobs of Immich are running now.

 

image.png.ab5dae6eabea6370be83cd8dd8219b85.png

 

 

Link to comment
13 minutes ago, nickniu001 said:

But the GPU still display nothing . I don't know its reason.

Are you really sure that it is not running on the CPU instead of the iGPU?

 

Open up a Unraid terminal and issue:

intel_gpu_top

 

What is the output from that command, please post a screenshot.

Link to comment
25 minutes ago, ich777 said:

As you can see nothing is using your iGPU.

I assume that your container is using the CPU for face recognition and not your iGPU.

You're right.

When I play some movies, I can see the GPU workiing. But other movies not.

Thanks very much.

  • Like 1
Link to comment
9 minutes ago, alturismo said:

therefore you woul dneed a special type of installation ... and i assume you didnt go this way ;)

 

https://immich.app/docs/features/hardware-transcoding/

Haha...

Thanks a lot.

This feature allows you to use a GPU to accelerate transcoding and reduce CPU load. Note that hardware transcoding is much less efficient for file sizes. As this is a new feature, it is still experimental and may not work on all systems. 

 

Maybe try it if I have time in the future.

Link to comment
  • 3 weeks later...

So, relatively new unraid user here. 
I have gone through almost 45 pages of this discussion, Google, reddit etc. not finding anything that helps. So i'll start with my setup. I5 13500, Gigabyte Aorus b760I Pro DDR4. 64 gig ram, 2 nvme drives as cache ZFS. 4 platter drives XFS. that's it.

 

Bios is configured correctly all settings are as mentioned. When I run 

root@Unraid-NAS:~# lspci -nn | grep VGA 00:02.0

 

VGA compatible controller [0300]: Intel Corporation AlderLake-S GT1 [8086:4680] (rev 0c)


When I run, intel_gpu_top I get the same thing I see everyone else getting. 

 

intel-gpu-top: Intel Alderlake_s (Gen12) @ /dev/dri/card0 -    0/   0 MHz; 100% RC6;                   0.00/13.94 W;        0 irqs/s        

ENGINES                                                      BUSY   MI_SEMA MI_WAIT
Render/3D  0.00% |                                   |      0%      0%       

Blitter  0.00% |                                          |      0%      0%           

Video    0.00% |                                         |      0%      0%   

VideoEnhance    0.00% |                            |      0%      0

 

When I check my video settings in the dashboard I get this nothing in terms of IMC bus utilization. I had to set power draw to package to get anything out of it. 

Screenshot 2024-03-14 210203.jpg

 

So, I just feel lost and don't know what else I should be looking. 

 

root@Unraid-NAS:~# timeout -k .500 .400 intel_gpu_top -J -s 250
[

{
        "period": {
                "duration": 4.202526,
                "unit": "ms"
        },
        "frequency": {
                "requested": 0.000000,
                "actual": 0.000000,
                "unit": "MHz"
        },
        "interrupts": {
                "count": 0.000000,
                "unit": "irq/s"
        },
        "rc6": {
                "value": 100.000000,
                "unit": "%"
        },
        "power": {
                "GPU": 0.000000,
                "Package": 26.519811,
                "unit": "W"
        },
        "engines": {
                "Render/3D": {
                        "busy": 0.000000,
                        "sema": 0.000000,
                        "wait": 0.000000,
                        "unit": "%"
                },
                "Blitter": {
                        "busy": 0.000000,
                        "sema": 0.000000,
                        "wait": 0.000000,
                        "unit": "%"
                },
                "Video": {
                        "busy": 0.000000,
                        "sema": 0.000000,
                        "wait": 0.000000,
                        "unit": "%"
                },
                "VideoEnhance": {
                        "busy": 0.000000,
                        "sema": 0.000000,
                        "wait": 0.000000,
                        "unit": "%"
                }
        },
        "clients": {

        }
}
{
        "period": {
                "duration": 253.728411,
                "unit": "ms"
        },
        "frequency": {
                "requested": 0.000000,
                "actual": 0.000000,
                "unit": "MHz"
        },
        "interrupts": {
                "count": 11.823666,
                "unit": "irq/s"
        },
        "rc6": {
                "value": 100.000000,
                "unit": "%"
        },
        "power": {
                "GPU": 0.003127,
                "Package": 15.094948,
                "unit": "W"
        },
        "engines": {
                "Render/3D": {
                        "busy": 0.000000,
                        "sema": 0.000000,
                        "wait": 0.000000,
                        "unit": "%"
                },
                "Blitter": {
                        "busy": 0.000000,
                        "sema": 0.000000,
                        "wait": 0.000000,
                        "unit": "%"
                },
                "Video": {
                        "busy": 0.000000,
                        "sema": 0.000000,
                        "wait": 0.000000,
                        "unit": "%"
                },
                "VideoEnhance": {
                        "busy": 0.000000,
                        "sema": 0.000000,
                        "wait": 0.000000,
                        "unit": "%"
                }
        },
        "clients": {

        }

 

I don't know what else to do or look at. So any help would be great. I know I'm missing something but I have zero idea what I'm missing. 

Thanks in advanced

 

Edited by pugano
i smacked my keyboard before i finsihed typing
Link to comment
2 hours ago, pugano said:

So, I just feel lost and don't know what else I should be looking. 

But everything seems alright…

 

You don‘t get every reading on every CPU/iGPU.

 

Put a load on the iGPU and you’ll see that the values are populated.

Intel iGPUs will show every time zeros when they have no laod on them.

  • Like 1
Link to comment
2 hours ago, pugano said:

I know I'm missing something but I have zero idea what I'm missing. 

actually probably not ;) its like @ich777 mentioned, there is a "idle" and a "usage" on the iGPU, by default nothing is using it for real and you wont see any activity.

 

sample idle here

image.png.c28c6a9023235ec910222438ecbd8e15.png

 

sample Plex web usage here, while encoding

image.thumb.png.46bbc4b2f8d9dc9e6ba41aa8b84372ce.png

 

but before you wonder why its not steady on load ... media servers are "buffering" and pre rendering

 

sample, when you look at the bar you see the "buffer"

image.thumb.png.9304701e7702d54232559411815541a2.png

 

and the gpu load will drop as its working in "waves" ... so dont wonder about it, normal behaviour and also depending on clients buffer behaviour, how your Media Server Software is setup and so on ...

 

sample here, after 60 seconds it will throttle here as i dont want to have the full media pre transcoded (if possible, local media could be pre transcoded, live tv like the sample couldnt be pre transcoded anyhow ;))

 

image.thumb.png.2ef6c7047b140e29402bbd61a5af2671.png

 

i hope it makes it a little clearer

Link to comment

I'm trying to add compreface-gpu to the SUPPORTED_APPS. But it looks like the plugin don't use the files /usr/local/emhttp/plugins/gpustat/ for the running dashboard version. I wanted to test this, before creating an PR.
Is there a trick?

 

grafik.thumb.png.cebbca623870c61e2dadf25aad023176.png

 

gpustatus.page:
 

    $apps = [

        'plex', 'jellyfin', 'handbrake', 'emby', 'tdarr', 'unmanic', 'dizquetv', 'ersatztv',

        'fileflows', 'frigate', 'deepstack', 'nsfminer', 'shinobipro', 'foldinghome', 'compreface',

    ];

 

/lib/Nvidia.php


 

    const SUPPORTED_APPS = [ // Order here is important because some apps use the same binaries -- order should be more specific to less
.

.
        'compreface'  => ['uwsgi'],
    ];

 

and added compreface.png to /usr/local/emhttp/plugins/gpustat/images/

 

Did I miss something?

Edited by corgan
Link to comment
49 minutes ago, corgan said:

I'm trying to add compreface-gpu to the SUPPORTED_APPS. But it looks like the plugin don't use the files /usr/local/emhttp/plugins/gpustat/ for the running dashboard version. I wanted to test this, before creating an PR.
Is there a trick?

 

grafik.thumb.png.cebbca623870c61e2dadf25aad023176.png

 

gpustatus.page:
 

    $apps = [

        'plex', 'jellyfin', 'handbrake', 'emby', 'tdarr', 'unmanic', 'dizquetv', 'ersatztv',

        'fileflows', 'frigate', 'deepstack', 'nsfminer', 'shinobipro', 'foldinghome', 'compreface',

    ];

 

/lib/Nvidia.php


 

    const SUPPORTED_APPS = [ // Order here is important because some apps use the same binaries -- order should be more specific to less
.

.
        'compreface'  => ['uwsgi'],
    ];

 

and added compreface.png to /usr/local/emhttp/plugins/gpustat/images/

 

Did I miss something?

If you are on 6.12 you need to look at the multi versions.

 

https://github.com/SimonFair/gpustat-unraid/blob/master/src/gpustat/usr/local/emhttp/plugins/gpustat/gpustatusmovablemulti.page

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.