ich777 Posted June 20, 2022 Author Share Posted June 20, 2022 4 minutes ago, Bitbass said: I've rebooted as part of the 6.10.3 upgrades on both sides, but I just restarted the Prometheus container on the primary and same result. What are the two subnets? Are you really sure that those are not the Docker subnets that you are trying to connect to? I'm really not sure what is going on but barely nothing is hard coded to my plugins, the exporters should listen to all IPs since I'm mostly using 0.0.0.0 Quote Link to comment
Bitbass Posted June 20, 2022 Share Posted June 20, 2022 On the primary I have several docker containers that are internally using 172.17.0.0. My primary's subnet is 192.168.169.0/24. The target subnet is 172.16.172.0/24, which you can see in the logs. The target was running docker for another container but I've turned that off. It's really only the Prom node exporter plugin on the target Unraid system. Nothing else running on there. Quote Link to comment
ich777 Posted June 20, 2022 Author Share Posted June 20, 2022 11 minutes ago, Bitbass said: On the primary I have several docker containers that are internally using 172.17.0.0. My primary's subnet is 192.168.169.0/24. The target subnet is 172.16.172.0/24, which you can see in the logs. The target was running docker for another container but I've turned that off. It's really only the Prom node exporter plugin on the target Unraid system. Nothing else running on there. That is really weird but I really can't help there, I can only thing of a routing issue in your network somewhere since everything is working on many other networks just fine. Did you also set up proper routing tables on Unraid too? Quote Link to comment
Bitbass Posted June 20, 2022 Share Posted June 20, 2022 I don't think it's routing related. I get an immediate response from the Ubuntu server target, which is also in the same target network. And my Mac is in the same source network (192.168.169.0/24) and it can connect to the Unraid target without issue. The default route is set correctly in both Unraid systems for their respective subnets and there are no firewall rules that would allow the Mac (as source) or the Ubuntu (as target) to work while preventing other things. I'm pretty sure this has something to do with Unraid (which one?) getting confused about talking to another Unraid system that's on a different subnet. I get it that this is fringe stuff and that it works for everyone else. I'm not expecting you to have the magic answer, just hoping if we keep talking about it, it might generate an idea for you, me or someone else reading this. Thanks! Quote Link to comment
ich777 Posted June 20, 2022 Author Share Posted June 20, 2022 53 minutes ago, Bitbass said: I get it that this is fringe stuff and that it works for everyone else. I'm not expecting you to have the magic answer, just hoping if we keep talking about it, it might generate an idea for you, me or someone else reading this. My issue with this is that I don't know your exact network setup and I'm really not a network expert when it comes to something like your setup, sure basic VLANs and such are no issue for me but when it comes to a more complicated setup I really can't help as much as I want too... I will think about this a bit more but I really can't imagine what's preventing the two system to talk to each other... Quote Link to comment
Bitbass Posted June 20, 2022 Share Posted June 20, 2022 You're in luck, because I am a network guy by trade! What I lack is the understanding of how the plug-ins talk to the networking or how Unraid itself does networking. It doesn't necessarily behave like a router or firewall and that gets me turned around some times. I might just have to figure out how to pull a pcap from my network and look at the actual TCP session. The verbose curl output isn't quite enough detail and it's only curl. Quote Link to comment
Bitbass Posted June 20, 2022 Share Posted June 20, 2022 Well, I just figured it out because of Wireshark. I had set the MTU to 9000 on the target system to support jumbo frames. This isn't a bad thing, unless its crossing to another network through a firewall and the firewall isn't happy about having to fragment it. I changed the target system MTU to 1500 and it started working immediately. I'm not certain it was the firewall, but it's pretty likely. My Mac, which has always been set to 1500, was always able to connect. No fragmentation. Thanks for helping me work through this! Sometimes you just need to voice ideas to have the answer present itself. 1 Quote Link to comment
jakami99 Posted June 21, 2022 Share Posted June 21, 2022 (edited) Hey Just a little hint for people having issues, since nothing was working for me the last 2 hours. Make sure that UNRAID can reach your fritzbox over fritz.box over the web terminal ping fritz.box I didn't realize it is accessing this way, maybe a possibility to insert an IP in the setup would help a lot. Thanks for the plugin love it, now where it's working Edited June 21, 2022 by jakami99 grammitic Quote Link to comment
ich777 Posted June 21, 2022 Author Share Posted June 21, 2022 54 minutes ago, jakami99 said: I didn't realize it is accessing this way, maybe a possibility to insert an IP in the setup would help a lot. Please see here: Quote Link to comment
Masterwishx Posted June 22, 2022 Share Posted June 22, 2022 @ich777 can i add --collector.systemd --collector.processes to node exporter .cfg for full view data in grafana like author of 1860 wrote : (Recommended for prometheus-node-exporter the arguments '--collector.systemd --collector.processes' because the graph uses some of their metrics.) 1 Quote Link to comment
ich777 Posted June 22, 2022 Author Share Posted June 22, 2022 24 minutes ago, Masterwishx said: can i add --collector.systemd --collector.processes to node exporter .cfg for full view data in grafana like author of 1860 wrote Yes, why not? But I don‘t understand why you would add —collector.systemd since Unraid doesn‘t use systemd 1 Quote Link to comment
Masterwishx Posted June 22, 2022 Share Posted June 22, 2022 12 minutes ago, ich777 said: But I don‘t understand why you would add —collector.systemd since Unraid doesn‘t use systemd Thanks for tip ... i only asked becose dont have a knowledge about it , trying to learn more about promethus right now ... so only processes i can add ... Quote Link to comment
ich777 Posted June 23, 2022 Author Share Posted June 23, 2022 12 hours ago, Masterwishx said: i only asked becose dont have a knowledge about it , trying to learn more about promethus right now ... This has strictly speaking nothing to do with the exporter itself this is more of a Unraid thing because Unraid doesn't use systemd. 12 hours ago, Masterwishx said: so only processes i can add ... You can try of course to add both but I don't think that you have any benefit when adding the systemd parameter because as said above Unraid doesn't use systemd. 1 Quote Link to comment
Bitbass Posted July 6, 2022 Share Posted July 6, 2022 Is there a way to get the Node Exporter to report on HDD temps? It looks like it has all of the other smartd metrics but I don't see temp. Quote Link to comment
ich777 Posted July 6, 2022 Author Share Posted July 6, 2022 1 hour ago, Bitbass said: Is there a way to get the Node Exporter to report on HDD temps? It looks like it has all of the other smartd metrics but I don't see temp. Have you yet seen this: Click I have not released this to the public because I have no testers out there and this it is in "beta"... You can install it by simply putting this line into the plugin installation: https://raw.githubusercontent.com/ich777/unraid-prometheus_smartctl_exporter/master/prometheus_smartctl_exporter.plg 1 Quote Link to comment
Bitbass Posted July 6, 2022 Share Posted July 6, 2022 1 hour ago, ich777 said: Have you yet seen this: Click I have not released this to the public because I have no testers out there and this it is in "beta"... You can install it by simply putting this line into the plugin installation: https://raw.githubusercontent.com/ich777/unraid-prometheus_smartctl_exporter/master/prometheus_smartctl_exporter.plg Check that out! Works like a charm, although Grafana is reporting one of the drives at 119c. It shows up as 33c in the Unraid dashboard and in the actually Need Exporter Metrics page. Not sure what's going on there. Quote Link to comment
Bitbass Posted July 6, 2022 Share Posted July 6, 2022 24 minutes ago, Bitbass said: Check that out! Works like a charm, although Grafana is reporting one of the drives at 119c. It shows up as 33c in the Unraid dashboard and in the actually Need Exporter Metrics page. Not sure what's going on there. 24 minutes ago, Bitbass said: Check that out! Works like a charm, although Grafana is reporting one of the drives at 119c. It shows up as 33c in the Unraid dashboard and in the actually Need Exporter Metrics page. Not sure what's going on there. For some reason, "value" is reporting incorrectly in the exporter but "raw_value" is correct. I'll let this run for a while and let you know if I have any problems. Thanks for making this! 1 Quote Link to comment
Bitbass Posted July 15, 2022 Share Posted July 15, 2022 Is there an array space metric that I'm missing? I currently capture array used and total space through telegraph (I think) using "disk" and "/mnt/user" and pulling the space from there. Trying to move to Prometheus entirely. Quote Link to comment
ich777 Posted July 15, 2022 Author Share Posted July 15, 2022 2 minutes ago, Bitbass said: Is there an array space metric that I'm missing? I'm not entirely sure what you want to know here, you have to add all "diskX" to get the Array size. Quote Link to comment
Bitbass Posted July 15, 2022 Share Posted July 15, 2022 Currently have this in my dashboard: The query is pulling out of Influx. Quote Link to comment
ich777 Posted July 15, 2022 Author Share Posted July 15, 2022 Just now, Bitbass said: Currently have this in my dashboard: The query is pulling out of Influx. I'm really not that familiar with Influx that I can help here... Maybe someone else has a clue... Quote Link to comment
Bitbass Posted July 15, 2022 Share Posted July 15, 2022 Not so concerned about Influx. I want to replicate this with the Prometheus plugin, if I can. I figured this out but it's only available bytes. I guess I can do a calculation with the total bytes. Quote Link to comment
Bitbass Posted July 15, 2022 Share Posted July 15, 2022 Took a little noodling but I got to where I wanted to be: Quote Link to comment
ich777 Posted July 15, 2022 Author Share Posted July 15, 2022 4 minutes ago, Bitbass said: Took a little noodling but I got to where I wanted to be: Wouldn't it be better to use /mnt/user0? That should be actually the Array, or at least all disks without any cache drive. Quote Link to comment
Bitbass Posted July 15, 2022 Share Posted July 15, 2022 You're probably right, but I'm going off of what I had before. I'll look at changing it. Quote Link to comment
Recommended Posts
Posted by SpencerJ,
1 reaction
Go to this post
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.