Jump to content

D34DC3N73R

Members
 View Profile  See their activity

  • Posts

    26

  • Joined

  • Last visited

 Content Type 

Posts posted by D34DC3N73R

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    On 10/7/2023 at 3:12 PM, Cyklone said:

    Thank you for this!

    It also worked for me.  Could not get the standard netdata to report nvidia stats.

    However, I seem to remember that more stats were available: pci_bandwidth, encoder/decoder utilisation, processes, etc.  Are these no longer available?

    Cheers!

     

    It doesn't look like everything from the python plugin has been added to the go plugin yet. I would hope they're coming, but the change was probably necessary for the new interface. These are what show up for me
    image.png.fc57c518dca196c88e1bfbfb84c63467.png

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted · Edited by D34DC3N73R
    add editor tip

    2 hours ago, mihcox said:

    Is there any direct guide on how to convert a docker container on your image on unraid to the new netdata/netdata one while preserving the ability to view my nvidia card?

     

    I haven't written it all out yet, but I do have a template that works. It need some documentation updates since the python nvidia-smi has been deprecated. If you'd like to test it out, you can follow these steps.

     

    Stop and delete the existing container. (configuration, not data)

     

    download the XML template 

    wget -O /boot/config/plugins/dockerMan/templates-user/my-netdata-gpu.xml https://raw.githubusercontent.com/D34DC3N73R/unraid-templates/netdata-image/netdata-glibc.xml

     
    In the unraid docker page, scroll to the bottom and click 'add container'. Select netdata-gpu from the template dropdown. The standard paths should be good for most new installs. The data and paths are compatible with the previous version of my image. However, my image shipped with python nvidia-smi enabled by default. Since that is now deprecated, you should disable it and enable the go.d implementation. 

     

    To do that, you should start the new container and then use the built in .edit-config 

     

    If you're using an existing configuration, you should edit python.d.conf 

    docker exec -it netdata bash
cd /etc/netdata
./edit-config python.d.conf

    You'll need to add a # in front of nvidia-smi to disable the python nvidia-smi module. Save and quit vi

     

    If you're using a new configuration or you've disabled the python nvidia-smi, then you can edit go.d.conf 

    docker exec -it netdata bash
cd /etc/netdata
./edit-config go.d.conf

     

    That will open the default go.d.conf file using vi, and save changes in your mounted directory /mnt/user/appdata/netdata/config. You'll need to delete the # and change 'no' to 'yes'.  

    #  nvidia_smi: no

    so it becomes  

      nvidia_smi: yes

    The two spaces before nvidia-smi are important.

     

    When you're done editing, save, exit the container cli, and restart the container. 

     

    If you have questions or run into problems please let me know. I'll be pushing the new template soon after I have time to run some more tests.

     

    Edit: Tip: If you'd like to use an alternate editor in place of vi, you can add the nano package for instance to the NETDATA_EXTRA_APK_PACKAGES environment variable (default `gcompat`). add a space between package names `gcompat nano`. Then you'll be able to call the edit config script using the new editor 

    ./edit-config --editor nano python.d.conf

     

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    11 minutes ago, veri745 said:

     

    The only files created are `.container-hostname` and `.opt-out-from-anonymous-statistics`

    netdata_config.thumb.PNG.2450c5e65dc76c887a105b44807d89c5.PNG

     

    I did get a bunch of folders and a `netdata.conf` when I installed the `stable` version, but not with `latest`

     

    I moved my entire netdata appdata folder to netdata.bak and started from scratch, but I'm not able to reproduce that with the latest or stable images and the current template from CA. Do you have auto-updates enabled? Is it possible you're using old images? Can you confirm it's running netdata v1.39.0?
    image.thumb.png.ebfb921bc2520306f6d50d63c8702daf.png

     

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    11 hours ago, veri745 said:

     

    I reinstalled and used those default mappings.  There does not appear to be any write activity to any of those mapped directories.

     

    The config directory remained complete empty.  I copied over the data from my "override" folder, but that doesn't seem to make any difference.

     

    I would just go ahead and use the official netdata docker, but the template for that image is completely empty, too

     

    /mnt/user/appdata/netdata/config will be populated on first run with the following folders & files:
    image.png.50658a00dfae512579e96f1d3508b045.png

    From here, you should use the netdata edit-config script to make changes to any conf. 

    For instance, to make changes to python.d.conf
      

    root@Tower:~# docker exec -it netdata-glibc bash
unraid:/# cd /etc/netdata
unraid:/etc/netdata# ./edit-config python.d.conf

     

    That will open up the default /usr/lib/netdata/conf.d/python.d.conf in a vi editor where you can make any changes. That fill will be saved to /etc/netdata as an override. This is the same functionality as the default netdata image.

     

    At this point, the only difference between this image and the default image is the glibc library required for nvidia GPUs and nvidia-smi being enabled by default in python.d.conf

    I do have a template I'm testing for the default netdata image now that it's compatible. If you want to test it out, see this post.

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    On 5/10/2023 at 11:58 AM, veri745 said:

    I tried mapping both 

    /var/lib/netdata

    and
    /var/cache/netdata

    as specified in the Docker Compose instructions to my appdata folder, but neither would let the container start

     

    You happen to install right before some significant changes to the template and image. I would suggest removing the container and reinstalling from the community app store. Afterwards, you can copy configuration files from /mnt/user/appdata/netdata/override to /mnt/user/appdata/netdata/config. 
     

    These are the new folders for persistent data
    image.thumb.png.fec07907e06d39fcfe157b386194169c.png

     

    There are also now environment variables to enter a netdata cloud token and room ID. Let me know if you run into any trouble.

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    Some exciting news, the official Netdata image can now be used to enable nvidia GPUs! I've submitted a PR (recently merged) to netdata/netdata documentation as to how this works. d34dc3n73r/netdata-glibc will be deprecated in favor of using the official image, but I will keep the template up with the required configuration to enable nvidia GPUs.

     

    For anyone who wants to test the template, you can run this command to get the new version using netdata/netdata
      

    wget -O /boot/config/plugins/dockerMan/templates-user/my-netdata-test.xml https://raw.githubusercontent.com/D34DC3N73R/unraid-templates/netdata-image/netdata-glibc.xml

    It's fully compatible with the previous dev image so the same directories can be used.

     

    A couple of important notes: 

     - This is currently works with :edge and :latest tags. When the next stable build of Netdata is released it will work will all tags.

     - You will likely have to edit python.d.conf to uncomment nvidia_smi: yes. More about that here: https://learn.netdata.cloud/docs/data-collection/monitor-anything/Hardware/nvidia_smi-python.d.plugin#docker-troubleshooting

     

    Let me know if you have any questions or problems.

     

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    7 minutes ago, Gex2501 said:

    Okay, so different question/issue. Is this docker missing the modules for windows and vnodes? I've tried following the instructions per this link, I have the exporter working on windows but I don't think Netdata is collecting any info.

     

    https://learn.netdata.cloud/docs/data-collection/monitor-anything/System Metrics/Windows-machines#configuration

     

    I'd recommend trying out the dev image & template. It's much more similar to stock netdata now that their image has override support. If it's a recent install, you're essentially using the new `edit-config` with the old method of custom override support. In the dev image, the only difference between this and stock netdata is the glibc support layer, enabling nvidia-smi by default, and adding nano as a config editor.

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted · Edited by D34DC3N73R

    2 hours ago, iel said:

     

    I tried to connect it to netdata cloud and it worked like a charm!

    Thank you for this thread; it tangentially helped shed some light/alleviate my frustration with unraid and docker

    (pebkac related issues)

     

    Thanks for trying out the dev image. The CA template and image will be updated with the next netdata release. The only thing you'll need to do is change the image tag from :dev to :stable or :latest depending on your preference.

    • Like 1

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    On 1/30/2023 at 8:02 AM, Wiseone001 said:

    currently yes.. i have disabled it but would like to get this active.

    Thanks

     

    I have a test template and image available to try out. 
    To get the template, in unraid terminal or ssh run 
      

    wget -O /boot/config/plugins/dockerMan/templates-user/my-netdata-glibc-dev.xml https://raw.githubusercontent.com/D34DC3N73R/unraid-templates/dev/netdata-glibc.xml


    In the unraid docker page, scroll to the bottom and click 'add container'. Select 'netdata-glibc-dev' from the template dropdown and change the image (Repository:) to 
    d34dc3n73r/netdata-glibc:dev

    If you've never installed before, this should be all it takes. If you have files and folders in /mnt/user/appdata/netdata, I recommend moving them to a temp directory like netdata-temp, and then starting the container once, stopping it and copying any configs you want to keep to the new netdata folders. 

     

    The new environment variables NETDATA_CLAIM_TOKEN, NETDATA_CLAIM_URL, and NETDATA_CLAIM_ROOMS are not required, but if you want to connect to netdata cloud, see https://learn.netdata.cloud/docs/agent/claim#connect-an-agent-running-in-docker for instructions on how to get the claim token and claim room (claim url is prefilled).

     

    I'd appreciate any feedback or experiences. 

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    2 hours ago, FCA Administrator said:

    Thanks so much, that fixed it! It never occurred to me to set the privileged option.

     

    Where is the log file exactly? /var/log/health.log seems to be the only one that gathers data, the others symlink to /dev/null, and I don't see an error like you mentioned...


    The container logs, or from unraid terminal / ssh 

    docker logs netdata-glibc 2>&1 | grep "go.d ERROR: ping"

     

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    On 2/11/2023 at 8:32 PM, FCA Administrator said:

    Hi there! I'm trying to enable the ping plugin to monitor the reachability of systems on my network. I've followed the online documentation but I'm somehow missing something. It doesn't appear in the GUI. I check it via the debug troubleshooting and it is pinging and collecting data, but I can't figure out how to see it in the web gui. Can someone help me with what I believe is a very basic feature of netdata?

     

    If you search the logs for "go.d ERROR: ping" you'll likely see "listen ip4:icmp : socket: operation not permitted" 
    when using ping, you can get around this by using unprivileged udp pings
     
    For example
      

    jobs:                                                                                                                   
  - name: pihole                                                                                                        
    hosts:                                                                                                              
      - 192.168.0.231                                                                                                   
      - 192.168.0.232                                                                                                   
    privileged: no


    After a reboot you'll see ping show up under sensors
    image.png.72afd50c52bd30ed5537782c1f4788ac.png

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    On 1/28/2022 at 8:10 AM, StephenCND said:

    I'm very interested in this docker, but unfortunately I do not have an Nvidia GPU. (Old AMD Radeon HD 5800). Is there any way I can install this docker?  I tried installing but it failed regardles of the "branch" I selected. (I did see the note about "turn on Advanced View and remove '--runtime=nvidia' from Extra Parameters, and remove NVIDIA_VISIBLE_DEVICES or set it to 'void'", but couldn't figure out how to do so.
    Any assistance would be greatly appreciated.

    You can use the standard netdata image, unless you'd like to use it for the override support. In that case, make sure you're in the advanced view (upper right corner after clicking "install")
    image.png.41f89e55c3dc9ddf66fe9b80670b26ad.png
    In "Extra Parameters" delete `--runtime=nvidia` 
    Then click "Show more settings ..."
    image.png.b478570053dcd1130f6257de101e9777.png
    and click "Remove" on the `NVIDIA_VISIBLE_DEVICES` environment variable.

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    On 3/4/2021 at 4:26 PM, derbo said:

    Anyone using this with 6.9? 

     

    I haven't used this in a long time and noticed it's now not showing nvidia-smi. Docker variable for Nvidia visible devices is set to all. 


    I haven't tried this out with 6.9 yet. Do you have other containers that use the GPU passthrough and are they working? I'll have to look into 6.9 to see if they changed the way GPU settings work.

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted · Edited by D34DC3N73R
    add fix info

    On 5/11/2020 at 5:14 PM, glompy said:

    I fixed this by opening up the docker console and executing the following command

      

    
chown netdata:netdata -R /usr/share/netdata

     

    I'm unable to reproduce this on a clean install. If you didn't edit the default template settings (specifically mounting a volume for custom dashboards), I'm guessing this was a problem with the netdata image and was fixed in a recent build. Please let me know if you run into any other problems.

    Edit: it does appear to be a problem with v1.22.0 and was fixed in v1.22.1

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted · Edited by D34DC3N73R
    add override update

    Update: Custom configuration support is now enabled. See the netdata configuration guide for details.

    /mnt/user/appdata/netdata/override is mapped to /etc/netdata.  The nvidia-smi plugin is now enabled in the stock configuration so GPU support will still work with a blank override directory.

    Note: /mnt/user/appdata/netdata/override will be empty by design. Stock configuration files are loaded from /usr/lib/netdata/conf.d/ 
    If you'd like to alter one of these config files, you can copy it to the /etc/netdata directory and make edits. 
     

    Edit: I've done some work on the image so files from /etc/netdata will appear in the volume mount /mnt/user/appdata/netdata/override. Please reach out if you have any problems.

     

    Also, side note in regards to custom dashboards: It appears netdata is working on a react dashboard, so I'm probably not going to put much time into custom dashboards for the current implementation. 

    • Like 1

    [Support] D34DC3N73R - Netdata GLIBC (GPU Enabled)

    in Docker Containers

    Posted

    @rjlan The current suggestion is mount a volume as a subdirectory of /usr/share/netdata/web. 
    To accomplish that, add a new path
    Name: Custom Dashboards
    Container Path: /usr/share/netdata/web/custom

    Host Path: /mnt/user/appdata/netdata/web

    After adding your custom dashboard.html to /mnt/user/appdata/netdata/web/ visit http://YOUR.SERVER.IP:19999/custom/dashboard.html

    I tested this with their tv.html and their simple demo.html. One caveat is make sure to use full urls in place of relative links to dashboard.js

    example:
    <script type="text/javascript" src="dashboard.js?v20190902-0"></script>
    becomes
    <script type="text/javascript" src="http://localhost:19999/dashboard.js?v20190902-0"></script>

     

    I'll think about adding this and an override path (for netdata.conf, etc) to the template, but I'd like to see if netdata implements any changes first.

×
×
  • Create New...