[Support] Data-Monkey - netdata


Recommended Posts

  • 1 month later...

Good evening (at least where I am):

 

I love this software and the beautiful graphs it creates.  The only thing I haven't figured out is how to enable IPMI monitoring.  I know it is available, because I have been successful in getting the ultimate unraid dashboard working and I can see data in my unRAID dashboard.  However, I can't seen to find any instructions on how to enable the freeipmi.plugin.

Link to comment
1 hour ago, tomfowler said:

Good evening (at least where I am):

 

I love this software and the beautiful graphs it creates.  The only thing I haven't figured out is how to enable IPMI monitoring.  I know it is available, because I have been successful in getting the ultimate unraid dashboard working and I can see data in my unRAID dashboard.  However, I can't seen to find any instructions on how to enable the freeipmi.plugin.

I am not using this plugin, but have you seen this explanation?

https://github.com/netdata/netdata/issues/6681

  • Like 1
Link to comment

Wow.  Thank you.  I was able to use the remote monitoring option of the plugin, which worked perfectly.  For those who are wondering, I passed command line options in netdata.conf to my IPMI "host" IP address, along with my normal username and password to access the IPMI interface on the LAN.  I have a Supermicro X10 board.

 

Appreciate the help!  I had missed that option before.

Link to comment

OK, so the plot thickens.  Thanks to @roland, I made this work on my X10 board.  Needing more power, I switched unRAID over to a dual E5 Supermicro X9 board this weekend.  Everything works, expect IPMI no longer feeds netdata.  I was about ready to post this question over at the netdata forums, but thought it may be more appropriate here.

 

## Environment
unRAID 6.9.2, using the Netdata app via the Community Plugins app which uses the official Netdata docker image

Netdata reports version v1.31.0-476-nightly

Supermicro X9DRi-LN4F+ (BIOS 3.61 dated January 2020) with twin E5-2667v2, 64GB of RAM

2117146228_Screenshot2021-11-085_28_12PM.thumb.png.7a3178f1cc15f39eadc2eac1571dd0e4.png

Configured (successfully in the past with different hardware) to access IPMI via the BMC using the following configuration settings in netdata.conf:

As a side note that may be helpful to troubleshooting, I am successfully pulling data from the system IPMI using the unRAID IPMI plugin and that data is moving from Telegraf to Influxdb and finally to Grafana.  However, I prefer to use Netdata as my sole monitoring agent.  This is the only thing holding me back.

 

Uncommented the freeipmi plugin line in the plugins section.  I don't think this is since the intro to the configuration file says these are all the default values:
 

         # charts.d = yes                                                                      
         # fping = yes                                                                         
          freeipmi = yes                                                                       
         # go.d = yes  

                                                                       

Provided specific commands to access remote server:

 [plugin:freeipmi]
   update every = 10
   command options = driver-type LAN_2_0 hostname 10.1.1.8 username myusername password mypassword

 

## Problem/Question
I switched from an X10SL7-F motherboard with an E3-1231v3 to the subject motherboard.  IPMI was successfully feeding Netdata.  With no configuration changes, I moved the unRAID USB stick and five drives to the new machine.  All came back up as expected, with the exception of IPMI no longer works in Netdata.

 

## What I expected to happen
Given that the IPMI data is successfully being harvested by unRAID and sent via Telegraf to an InfluxDB and Grafana instance, I have to think it is a specific issue with the remote access configuration to the specific board.

I have verified on the Supermicro website, that this board is IPMI 2.0 compatible

 

I have tried to use the command line to gain additional information on debugging reading through the netdata learn webpage, and it is pasted below:

/usr/libexec/netdata/plugins.d # ./freeipmi.plugin driver-type LAN_2_0 hostname 10.1.1.8 username myusername password mypassword
 
2021-11-08 22:10:53: freeipmi.plugin ERROR : MAIN : ipmi_monitoring_sensor_readings_by_record_id(): connection timeout
 
2021-11-08 22:10:53: freeipmi.plugin FATAL : MAIN : freeipmi.plugin: data collection failed. # : Invalid argument

 

The odd thing is that the response is exactly the same if I provide an IP address of a different computer that doesn't have IPMI/BMC.  I have also tried the "debug" option and I haven't been able to gain any helpful information there.

 

I have the default unRAID container settings, which has the network set to "host".  I am unable to ping the IPMI address, which may be the root of the problem.  However, I used the default container settings on the other machine, so this may be a red herring.

 

Any ideas?

 

Thanks in advance!

Edited by tomfowler
Link to comment

So, for anyone else here trying to solve this problem, I figured it out.  Essentially, the X9 boards are unable to talk to themselves through the shared IPMI/BMC and LAN port.  The ping problem was key.  I could ping every network address except the unRAID server and the IPMI address.  This website here describes the issue in detail.  All I had to do was connect another cable to the dedicated IPMI port and into the switch.  Then change the setting from shared to dedicated on the network page in IPMI configuration.  Boom, netdata immediately started picking up IPMI data.

 

Apparently, the X10 boards are able to do this on one cable.

  • Thanks 1
Link to comment
On 10/19/2021 at 7:10 PM, worldspawn said:

I installed the docker just now, but when I open the webUI, it's just a blank page that says 'Netdata, realtime monitoring done right' in the background.  Is there something I'm missing here?

Same, and this worked for me:

 

On 9/9/2021 at 9:01 PM, stFfn said:

i had the same issue. after the latest update netdata didnt work.

 

everyone who has the same issue just go back go the version before.

 

netdata/netdata:v1.31.0

 

image.thumb.png.9c52cd3176ef22e3c48c5b358d3ce2b7.png

 

Link to comment
  • 2 weeks later...
  • 2 weeks later...

Hi and thanks for this docker template!

I am trying to add settings to the `/override` folder, correctly mounted to `/etc/netdata/override` as per template default variables. I'd like to add simple settings that apply to `netdata.conf`, such as `hostname = hello-world`, without having to modify the `netdata.conf` file manually. Is that possible, or are `override` config only available for alarms and such? I can't find any documentation about that.

Link to comment
  • 2 weeks later...
On 12/2/2021 at 7:20 PM, OdinEidolon said:

Hi and thanks for this docker template!

I am trying to add settings to the `/override` folder, correctly mounted to `/etc/netdata/override` as per template default variables. I'd like to add simple settings that apply to `netdata.conf`, such as `hostname = hello-world`, without having to modify the `netdata.conf` file manually. Is that possible, or are `override` config only available for alarms and such? I can't find any documentation about that.

Bump!

Link to comment
On 12/2/2021 at 7:20 PM, OdinEidolon said:

Hi and thanks for this docker template!

I am trying to add settings to the `/override` folder, correctly mounted to `/etc/netdata/override` as per template default variables. I'd like to add simple settings that apply to `netdata.conf`, such as `hostname = hello-world`, without having to modify the `netdata.conf` file manually. Is that possible, or are `override` config only available for alarms and such? I can't find any documentation about that.

Still doubtful on how I'd do that. There must be a simple way!

Creating a custom `/etc/netdata/override/netdata.conf` does not work, and modifying the default file is overwritten on every container update. I guess one could add a path to the docker container template, but it does not seem like a proper solution to me.

Link to comment

Initially Netdata was working well for me but I have now noticed that every time after the container image is updated I get an Executiion error = Error code 403.

image.png.eb1aae613efda7ac89a0cc95c0bd780f.png

 

After reading around I a have managed to fix this by performing:
 

docker network disconnect host Netdata

 

...and I can then start the container normally. All is well until the container image next gets updated and I have to do this again.

 

This only happens with this particular container. Does anyone know why this might be happening? Thanks!

 

Link to comment
On 12/19/2021 at 12:50 PM, writablevulture said:

Initially Netdata was working well for me but I have now noticed that every time after the container image is updated I get an Executiion error = Error code 403.

image.png.eb1aae613efda7ac89a0cc95c0bd780f.png

 

After reading around I a have managed to fix this by performing:
 

docker network disconnect host Netdata

 

...and I can then start the container normally. All is well until the container image next gets updated and I have to do this again.

 

This only happens with this particular container. Does anyone know why this might be happening? Thanks!

 

 

This has happedned again. This time I include full diagnostics immedately after I see the 403 error in case anyone can help me shed some light on this please. Thank you!

tower-diagnostics-20211222-1410.zip

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.