Using GPU in Docker?

[Panics]

Hello together,

i got a small problem trying to get my GPU working in a Docker container.. 
I´ve installed Agent-DVR from the appstore with the extra parameters: "--runtime=nvidia"
and gave it two fields with "NVIDIA_VISIBLE_DEVICES" and the GPU ID and also "NVIDIA_DRIVER_CAPABILITIES" with the value "all" trying to pass my GPU to it.
In Agent DVR i set every of my 7 cameras to use the GPU for playback so it should use the GPU but if i look in terminal with the command nvidia-smi it just idles 

Thu Jun 29 12:23:15 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.54.03              Driver Version: 535.54.03    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce GTX 970         Off | 00000000:11:00.0 Off |                  N/A |
|  0%   51C    P0              36W / 180W |      0MiB /  4096MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

am i doing something wrong?

The graphics card is an GTX 970 cuz i got it laying around and the installed version of unraid is 6.11.5 because if i update unraid, the docker containers and VMs get realy slow and unstable so i didnt update to 6.12.1.
The Server is an older kind with 160GiB DDR3 ECC Ram and 4 x Intel Xeon CPU E7- 4870 if this matters anything. 

 

If i am in the wrong area with this post i am sorry but i dont know where to post this

I also tryed the binhex-plexpass app just to be sure and it also never uses the GPU if i give it the GPU-ID..

Did someone have an idea what i am doing wrong? 
The GPU is just installed to support my cameras and got no other process so theres nothing to interfere with this. 
If someone could lead me in the right direction that would be awesome. 

Greetings 

Panics

Edited June 30, 2023 by Panics

[Thirs]

@Panics It seems you may be missing a bit of configuration in your Docker containers. Try adding:
Variable named 'NVIDIA_DRIVER_CAPABILITIES' and set the value to all

See: https://github.com/binhex/documentation/blob/master/docker/faq/plex.md
Q3/A3: How do I configure Plex to use my GPU for encoding/decoding (sometimes referred to as hardware transcoding)?

These instructions should be the same depending on the Docker container configured. Also might want to update the Nvidia Driver if that hasn't been done already using the plugin:

 

[Panics]

@Thirs Thank you at first for your reply.
As i wrote before i have set this variable but it does not work either way if it set or not.
 

I´ve also tryed to upgrade Unraid to its latest build and set up the container from scratch but with no luck at all

[ich777]

On 6/29/2023 at 12:40 PM, Panics said:

and gave it two fields with "NVIDIA_VISIBLE_DEVICES" and the GPU ID and also "NVIDIA_VISIBLE_DEVICES" with the value "all"

I think he was referring to this because you have wrote twice NVIDIA_VISIBLE_DEVICES

 

On 6/29/2023 at 12:40 PM, Panics said:

I´ve installed Agent-DVR

Are you sure that the container even support Nvidia HW acceleration?

I don't know this container but I would recommend that you post in the support thread from that container.

If you open up a container console (not a Unraid Terminal) and issue:

nvidia-smi

Do you get the same output? If yes the driver is working correctly and it is most likely that something in the container is missing so that Nvidia is not supported or something inside the application is not configured properly.

[Panics]

You are right i fixed that mistake so it is visible i set the values.. 

Agent-DVR gives an option to choose if you wanna use a GPU and also a specific renderer so it should work.
I´ve read other got this to work as intendet i just seem to miss something  

Yes the output is the same in Agent-DVR and also in binhex-plexpass. 
So it means the Driver does what it should.. are there any logs i can look into why both docker wont use my GPU? 

[ich777]

3 hours ago, Panics said:

Agent-DVR gives an option to choose if you wanna use a GPU and also a specific renderer so it should work.

But that doesn't mean it must work, it can work but must not if something is missing in the container or configured not properly.

 

3 hours ago, Panics said:

Yes the output is the same in Agent-DVR and also in binhex-plexpass. 

This means that the driver is just working fine in the container.

 

3 hours ago, Panics said:

I´ve read other got this to work as intendet i just seem to miss something  

3 hours ago, Panics said:

So it means the Driver does what it should.. are there any logs i can look into why both docker wont use my GPU? 

I don't know because I'm not familiar with Agent-DVR, please troubleshoot this in the support thread from Agent-DVR.

[Thirs]

AFAIK Nvidia GPU support is an open ticket with iSpy-Agent. If it does work might have to try different settings in the iSpy-Agent program itself to see if that helps. I've never gotten the GPU support working but haven't tried on newer versions of this program. See: https://github.com/doitandbedone/ispyagentdvr-docker/issues/79

What does work is using an Nvidia GPU with DeepStack for object detection. See: https://hub.docker.com/r/deepquestai/deepstack Tag: deepquestai/deepstack:gpu

This specific version seems to work with older and recent iSpy-Agent: deepquestai/deepstack:gpu-2021.09.1
I have an Nvidia 1050Ti GPU passed to the iSpy-Agent, DeepStack and Plex containers. I recall having to flash the GPU .rom file a long time ago so it would work with Unraid. However I think that was before the Nvidia driver support feature was available so that might not be needed anymore. Just mentioning this to see if that's something missing here.
I tested this by shutting down the DeepStack container in Unraid with iSpy-Agent running and ran the "nvidia-smi" console command. This returned no running processes.
I then started up the DeepStack container and left the iSpy-Agent container running as well. This time "nvidia-smi" returned two processes running which is typical when the DeepStack container is running.
Regarding the Plex container not showing a running process in "nvidia-smi" this will only show when transcoding is occurring. I have verified this works when a transcode stream is running. Otherwise you won't see Plex listed as a running process in "nvidia-smi".
An easy way to see processes and other stats of the GPU on the Unraid dashboard if you don't have it already is the GPU Statistics plugin. A bit more real-time then the "nvidia-smi" console command having to be re-entered.
Anyways, just some extra info @Panics and testing to show what I've seen with behavior of these containers and an Nvidia GPU.

Edited July 1, 2023 by Thirs

[Panics]

Thank you for the batch of information @Thirs i will take a look into this. 
My third child was born saturday so i was a bit absent in the last days :'D 

Plex never shows up even while transcoding anything so i think i got a problem right there.
DeepStack is something i will experiment with in the near future so thank you for the information here.