FirstDose Posted November 25, 2021 Share Posted November 25, 2021 Hi there, I am trying to install the node exporter plugin but it's failing to do so. I have docker disabled and don't intend to enable it since I have other servers running prometheus, etc. I just want the node exporter running so that I can access it externally. The error it gives me: Something really wrong went on during onStartupScreen Post the ENTIRE contents of this message in the Community Applications Support Thread OS: 6.9.2 Browser: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36 Language: <br /> <b>Warning</b>: parse_ini_file(/boot/config/docker.cfg): failed to open stream: No such file or directory in <b>/usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php</b> on line <b>50</b><br /> <br /> <b>Warning</b>: array_replace_recursive(): Expected parameter 2 to be an array, bool given in <b>/usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php</b> on line <b>50</b><br /> {"status":false} This also forcefully unmounts my flashdrive, requiring a reboot to fix that. Is it possible to install the plugin without docker enabled or is that not an intended use case? Quote Link to comment
ich777 Posted November 25, 2021 Author Share Posted November 25, 2021 18 minutes ago, FirstDose said: I have docker disabled I will try this when I got home from work on my test syeten and report back if the same happens there, actually never tried to use unRAID without Docker but that should have nothing to do with the node exporter. Quote Link to comment
ich777 Posted November 25, 2021 Author Share Posted November 25, 2021 4 hours ago, FirstDose said: Is it possible to install the plugin without docker enabled or is that not an intended use case? I've now tested it and it works as expected: As you can see from the screenshot I have Docker also disabled and have the Prometheus Node Exporter running, here is also a screenshot from the output from the exporter: I have also rebooted the machine to check if it is working as it should and it rebooted without an issue (test was done on 6.9.2 but I can also test it on another version from unRAID but this will make no difference). Do you have enough space on the USB Boot device left? @Squid do you have a idea why he is getting this message from above? Quote Link to comment
FirstDose Posted November 25, 2021 Share Posted November 25, 2021 I have over a dozen of GB left on the flashdrive and I also successfully installed the nvidia smi exporter. But for some reason the node exporter is not working. Here is what it says when I try to install it: plugin: installing: https://github.com/ich777/unraid-prometheus_node_exporter/raw/master/prometheus_node_exporter.plg plugin: downloading https://github.com/ich777/unraid-prometheus_node_exporter/raw/master/prometheus_node_exporter.plg plugin: downloading: https://github.com/ich777/unraid-prometheus_node_exporter/raw/master/prometheus_node_exporter.plg ... done plugin: downloading: https://github.com/ich777/unraid-prometheus_node_exporter/raw/master/packages/prometheus_node_exporter-2021.11.18.tgz ... done +============================================================================== | Installing new package /boot/config/plugins/prometheus_node_exporter/prometheus_node_exporter-2021.11.18.tgz +============================================================================== Verifying package prometheus_node_exporter-2021.11.18.tgz. Installing package prometheus_node_exporter-2021.11.18.tgz: PACKAGE DESCRIPTION: Package prometheus_node_exporter-2021.11.18.tgz installed. -------------Starting Prometheus Node Exporter!-------------- Warning: copy(/boot/config/plugins/prometheus_node_exporter.plg): failed to open stream: Input/output error in /usr/local/emhttp/plugins/dynamix.plugin.manager/scripts/plugin on line 535 plugin: prometheus_node_exporter.plg installed Updating Support Links Finished Installing. If the DONE button did not appear, then you will need to click the red X in the top right corner DONE Once I click DONE it gives me the error I posted earlier. When I go to the CA tab it gives me the following error as my flashdrive has been forcefully unmounted: Fatal Flash Drive Error This usually means your flash drive is corrupted or has dropped offline. The rest of the Unraid GUI will also be affected. Post a new thread in the Unraid forums if this error continues OK Rebooting fixes the flashdrive. Quote Link to comment
ich777 Posted November 27, 2021 Author Share Posted November 27, 2021 On 11/25/2021 at 3:41 PM, FirstDose said: The error it gives me: @Squid do you have a clue what's going on here (first post on this page)? I tried it on my test server with Docker disabled and it installed just fine. Quote Link to comment
T0a Posted December 5, 2021 Share Posted December 5, 2021 (edited) I would like to add authentication for the node-exporter and the Prometheus metrics endpoint. What I did so far: scrape_configs: - job_name: 'prometheus' scrape_interval: 15s static_configs: - targets: ["localhost:9090"] - targets: ["<UNRAID_IP>:9100"] basic_auth: username: myuser password: mypassword I then updated the data source with the basic auth option in Grafana given the credentials listed above. Though, it seems like Grafana does not receive any metrics after that. Do I have to add the basic auth credentials to the node-exporter too? If so, how can I pass the web.yml with the field basic_auth_users and the bcrypted password to the node-exporter plugin then? Follow-up question: How can I pass a web.yml to the Prometheus container configuring basic authenticiation for connections to the Prometheus expression browser and HTTP API (reference) ? Would love to see an "Authentication" section in one of the entry posts. Edited December 5, 2021 by T0a Quote Link to comment
ich777 Posted December 5, 2021 Author Share Posted December 5, 2021 3 hours ago, T0a said: If so, how can I pass the web.yml with the field basic_auth_users and the bcrypted password to the node-exporter plugin then? You are talking about securing Prometheus am I right? I think you should head over to the Prometheus support thread to get help with that. ...no you shouldn't need to modify anything to the node-exporter plugin because Prometheus is polling the plugin. Quote Link to comment
deadsix Posted December 26, 2021 Share Posted December 26, 2021 (edited) Hello ich777 et al, I am having a little difficulty with generating the dashboard following the guide on the unraid blog. I've followed the instructions on the blog but my Grafana dashboard doesn't display any stats: The Prometheus target appears to be up and functioning according the target stats page and I am receiving an output from the /metrics endpoint: I've restarted my server and let waited for Grafana to pick up the output from Prometheus for over an hour, any thoughts on what might be wrong with my setup? Thanks in advance! Edited December 26, 2021 by deadsix Quote Link to comment
ich777 Posted December 26, 2021 Author Share Posted December 26, 2021 1 hour ago, deadsix said: I've restarted my server and let waited for Grafana to pick up the output from Prometheus for over an hour, any thoughts on what might be wrong with my setup? Are you sure that you've picked the right data source? Have you also other sources in Grafana? Quote Link to comment
deadsix Posted December 26, 2021 Share Posted December 26, 2021 13 minutes ago, ich777 said: Are you sure that you've picked the right data source? Have you also other sources in Grafana? I believe so I've included screenshots of my configuration below. I have no other sources and I received a 'Data source is working' response when I added the Prometheus data source seen below: I am importing the dashboard to my local Grafana with the following options: Quote Link to comment
ich777 Posted December 27, 2021 Author Share Posted December 27, 2021 8 hours ago, deadsix said: I am importing the dashboard to my local Grafana with the following options: I think that's an issue with the Dashboard itself, can you go into a graph itself and manually changing the data source to Prometheus, I think a few pages back a user experienced a similar issue with another Dasboard who has had also another Data Source in Grafana. Quote Link to comment
deadsix Posted December 27, 2021 Share Posted December 27, 2021 (edited) 11 hours ago, ich777 said: I think that's an issue with the Dashboard itself, can you go into a graph itself and manually changing the data source to Prometheus, I think a few pages back a user experienced a similar issue with another Dasboard who has had also another Data Source in Grafana. I went into the dashboard and manually changed the data source to Prometheus, but the dashboard still isn't displaying any data. While I was poking around I did notice that the query inspector is stating that the response is empty: Edit: I figured it out, it was a simple misconfiguration issue Prometheus didn't have the right target port in the YAML file. For others who may encounter issues the Prometheus Node Exporter is only on port 9100. Thanks ich777 for your help and great plugin! Edited December 27, 2021 by deadsix 1 Quote Link to comment
Fotis Yanchev Posted January 5, 2022 Share Posted January 5, 2022 Dear Ich777, read the whole treat, having following problem: I am using the Node exporter, its works fine with Grafana and prometheus.yml, feeding data etc. Thank you for your efforts. Secondly I wanted to set up the Adguard Home exporter too - installed it, added the additional line to promethus.yml with IP of the Adguard docker server (same as Unraid but with port 9617. Adguard Home Exporter is set with user pass and IP:3000 and its running. Restarting the Prometheus, when I open WebUI the Adguard home and node exporter are UP, loading 13300 Adguard to Grafana and targer Prometheus but no data is feeded. Going back to Adguard Exporter refreshing the page shows it as stopped - starting it and getting back to WebUI of Prometheus -but it shows the Adguard Home instance down with following error: Get "http://192.168.0.11:9617/metrics": dial tcp 192.168.0.11:9617: connect: connection refused If I restart the Adguard exporter and refresh Prometheus, some times it shows same error and sometimes its up,but Grafana does not show any data. What could be the reason for the Adguard home exporter to keep stopping when I refresh the page - in Backup pluging have set it to Don't stop, as suggested. Adguard Home is running in docker as host. Have Enabled host access in docker settings, although I am not running in a custom network. Would be great if you could suggest a solution. Your efforts and time taken is highly appreciated in advanced. Regards, Fotis Quote Link to comment
ich777 Posted January 5, 2022 Author Share Posted January 5, 2022 6 hours ago, Fotis Yanchev said: "http://192.168.0.11:9617/metrics": dial tcp 192.168.0.11:9617: connect: connection refused Can you connect from a local browser to this URL? http://192.168.0.11:9617/metrics 6 hours ago, Fotis Yanchev said: What could be the reason for the Adguard home exporter to keep stopping when I refresh the page The reason might be that it tries to connect and it shows that it's actually connected but fails after a few seconds and if you refresh the page it shows the "real" status of the exporter. I think something might be wrong with the connection itself, please try if you can connect to the URL above. Adguard is on the host network and Prometheus and Grafana are on the "bridge" network, correct? 1 Quote Link to comment
Fotis Yanchev Posted January 5, 2022 Share Posted January 5, 2022 When I start the exporter and before it stops - I open the http://192.168.0.11:9617/metrics and it shows that data is feeding - if I refresh the page several times - lets say after 10-20 sec. it shows site as unreachable, at that time i check the exporter and it shows as stopped. When the exporter is running, at the link there is data visible, but Grafana does load any data - maybe the time the exporter is running is not enough for the data to be collected and shown in Grafana, but as said when the exporter is running the link shows the data, before the exporter stops. Quote Link to comment
ich777 Posted January 5, 2022 Author Share Posted January 5, 2022 1 hour ago, Fotis Yanchev said: When the exporter is running, at the link there is data visible, but Grafana does load any data This may take a while to load the data in Grafana itself. Have you yet tried to restart unRAID itself? Do you have any other data sources in Grafana than Prometheus? 1 Quote Link to comment
Fotis Yanchev Posted January 5, 2022 Share Posted January 5, 2022 No only Prometheus, installed both with Grafana just yesterday, will restart unRaid now and start the exporter. I know it takes time to load data in Grafana, but if the exporter stops, it will not load any data. Quote Link to comment
ich777 Posted January 5, 2022 Author Share Posted January 5, 2022 1 minute ago, Fotis Yanchev said: No only Prometheus, installed both with Grafana just yesterday, will restart unRaid now and start the exporter. I know it takes time to load data in Grafana, but if the exporter stops, it will not load any data. This is really strange, I will try it on my test server with a fresh installed Prometheus Exporter, please give me some time since I'm currently on vacation. 1 Quote Link to comment
Fotis Yanchev Posted January 5, 2022 Share Posted January 5, 2022 I will test and keep you updated. Its nothing urgent, enjoy of course your vacation. Will restart the server and try again, is there any log that I can run on the AdGuard home Exporter to investigate the reason for stopping after 10-20 sec. Maybe it could be of help for you. Sorry for dummy questions- its my first server with Unraid and just adding some apps/plugins and trying to get to know the whole thing. Quote Link to comment
ich777 Posted January 5, 2022 Author Share Posted January 5, 2022 3 minutes ago, Fotis Yanchev said: I will test and keep you updated. Its nothing urgent, enjoy of course your vacation. Will restart the server and try again, is there any log that I can run on the AdGuard home Exporter to investigate the reason for stopping after 10-20 sec. Maybe it could be of help for you. In this case no. But we can try something else next week if that's not too late for you. I will test it and get back to you. 1 Quote Link to comment
gyto6 Posted January 13, 2022 Share Posted January 13, 2022 (edited) Hi everyone, I've been reading your tutorial but booting up Prometheus's docker is already a mess. I'm getting the following message in the logs before it goes back to "stopped" status : ts=2022-01-13T17:23:18.190Z caller=main.go:520 level=info build_context="(go=go1.17.5, user=root@54b6dbd48b97, date=20211217-22:08:06)" ts=2022-01-13T17:23:18.191Z caller=main.go:521 level=info host_details="(Linux 5.14.15-Unraid #1 SMP Thu Oct 28 09:56:33 PDT 2021 x86_64 947eed537a1c (none))" ts=2022-01-13T17:23:18.191Z caller=main.go:522 level=info fd_limits="(soft=40960, hard=40960)" ts=2022-01-13T17:23:18.191Z caller=main.go:523 level=info vm_limits="(soft=unlimited, hard=unlimited)" ts=2022-01-13T17:23:18.205Z caller=query_logger.go:86 level=error component=activeQueryTracker msg="Error opening query log file" file=/prometheus/queries.active err="open /prometheus/queries.active: permission denied" panic: Unable to create mmap-ed active query log Sounds like an permission problem. http://hostip/metrics is accessible whatever but not relevant Just in case, the docker folder has been set on a zfs dataset, but i doubt that it could lead to the source of the trouble. I'm not fully instructed to docker so if you may have an idea on how to resolve this little problem? Thanks. Edited January 13, 2022 by gyto6 Quote Link to comment
ich777 Posted January 13, 2022 Author Share Posted January 13, 2022 1 hour ago, gyto6 said: Sounds like an permission problem. Exactly, I would recommend to start over and delete the container and also the directory for this container. Also this seems more like an issue with the Prometheus container so I would recommend that you ask in the support forums from the container iteslf. 1 hour ago, gyto6 said: Just in case, the docker folder has been set on a zfs dataset, but i doubt that it could lead to the source of the trouble. Are you on 6.9.2 then it could be possible that if you have certain application files from Docker on ZFS that it simply doesn't work. 1 Quote Link to comment
gyto6 Posted January 13, 2022 Share Posted January 13, 2022 31 minutes ago, ich777 said: Also this seems more like an issue with the Prometheus container so I would recommend that you ask in the support forums from the container iteslf. Are you on 6.9.2 then it could be possible that if you have certain application files from Docker on ZFS that it simply doesn't work. My mistake! I forgot that you didn't design the docker by itself.. I'm doing my way on the docker support forum. Thanks for your help, I'll try your way around, even if for a first install, it sounds quite weird. Quote Link to comment
gyto6 Posted January 13, 2022 Share Posted January 13, 2022 (edited) 43 minutes ago, ich777 said: Exactly, I would recommend to start over and delete the container and also the directory for this container. Man/Woman, your solution's worked.. I should have done this before 😅 Thank you! Edited January 13, 2022 by gyto6 1 Quote Link to comment
RiDDiX Posted January 14, 2022 Share Posted January 14, 2022 On 9/23/2021 at 11:03 AM, ich777 said: Prometheus Fritzbox Exporter Download and install the Prometheus Fritzbox Exporter plugin from the CA App: Log in to your Fritzbox and go to "System -> FRITZ!Box-Benutzer": Click on "Benutzer hinzufügen": Create a new user with a password for the exporter (in this example "grafana") and tick the following check boxes ("FRITZ!Box Einstellungen", "Sprachnachrichten, Faxnachrichten, FRITZ!App Fon und Anrufliste" & "Smart Home"): Go to "Heimnetz -> Netzwerk -> Netzwerkeinstellungen" and select "Statusinformationen über UPnP übertragen": Go to the plugin settings by clicking on "Settings -> Fritzbox Exporter" (at the bottom of the Settings page) : Enter the username and password that you've created in Step 3 for the exporter in your Fritzbox and click on "Confirm & Start": After that you should see in the right top corner that the Exporter is running and details about it: Open up the prometheus.yml (Step 4 + 5 from the first post), add a line with '- targets: ["YOURSERVERIP:9042"]' (please change "YOURSERVERIP" to your Server IP), save and close the file: Go to the Docker page and restart Prometheus: Open up the Grafana WebUI: In Grafana click on "+ -> Import": Now we are going to import a preconfigured Dashboard for the Fritzbox Exporter from Grafana.com (Source), to do this simply enter the ID (12579) from the Dasboard and click "Load": In the next screen rename the Dashboard to your liking, select "Prometheus" as datasource and click on "Import": Now you should be greeted with something like this (please keep in mind that the Dashboard can display N/A at some values, especially at the gauges, since there is not enough data available, wait a few minutes and you will see that the values are filled in): This seems to be a nice plugin. But anyway I cant get it to work. Sadly. But anyway thank you for this! Quote Link to comment
Recommended Posts
Posted by SpencerJ,
1 reaction
Go to this post
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.