hermy65 Posted September 9, 2020 Share Posted September 9, 2020 Not necessarily a support question but not sure where else to put this. Is there a plugin/app of some sort that we can install in unRAID that will allow us to visualize our storage usage over time? Perhaps it would show how much additional storage we are using per month, etc? Per month storage burn might be helpful in determining when you need to get additional drives ,etc. Also, it could show you if there was a deletion of data or something of that sort as your usage would drop, etc? 1 Quote Link to comment
KoNeko Posted September 9, 2020 Share Posted September 9, 2020 5 minutes ago, falconexe said: +1 Actually, this would be pretty sweet if it logged data growth and shrinkage (deletes) over time and threw these statistics in a pretty graph. I would love to know how many TB I ate up in a given timeframe, or the last 30 days. I currently track this manually. Trending would be a nice touch...Based on current usage, here's the date you need to buy new drives, and here's when to order them ha ha. If anyone knows of a docker or plugin that does this, let us know. @hermy65 Thanks for posing this question. Great stuff... kinda a prtg ish something or 1 of the other tools. Quote Link to comment
hermy65 Posted September 9, 2020 Author Share Posted September 9, 2020 @falconexe / @KoNeko Perhaps we can push this idea towards one of the plugin devs here and they can make something like it or integrate it into an existing plugin they offer? Quote Link to comment
KoNeko Posted September 9, 2020 Share Posted September 9, 2020 i do like graphs and lines Quote Link to comment
ChatNoir Posted September 9, 2020 Share Posted September 9, 2020 I have seen very nice things done with grafana. Seems very powerful and customizable. Didn't get into it yet, so I can't tell you for sure it is doable. Quote Link to comment
hermy65 Posted September 10, 2020 Author Share Posted September 10, 2020 @falconexe Nice find - let us know if it works for you! Quote Link to comment
GilbN Posted September 10, 2020 Share Posted September 10, 2020 Nice Btw I'm redoing most of the panels to be more efficient on the queries. I'll share a link here when I'm done. 2 Quote Link to comment
GilbN Posted September 11, 2020 Share Posted September 11, 2020 (edited) 3 hours ago, falconexe said: Hey man thanks for responding. I've been working on this all day and night. Just woke up from a 5 hour "nap". I am getting pretty good with it (I come from a SQL/Data Science background). However, I do have some questions, and have a few oddities in some of the queries. Perhaps what you are already working on... Hopefully you are down for and open to some questions. I'm really looking forward to picking your brain. Can't thank you enough! I haven't been this geeked out on my server for a while ha ha. The major item is I can't get CPU temps to show up. I'm thinking I'm missing a core plugin or something. Until I installed the HDDTemp docker, my HD temps looked empty as well (now working). When I dive into these queries, none of the values auto populate. Looks like yours are hard coded. I'm inferring that it can't query the database for these values because they don't exist in some array. Any thoughts? EDIT: @GilbN Here is the Query. The values for "sensors", "chip", and "feature", all do not have a drop down with auto populated values like every single other FROM statement. I am guessing the "sensors" table/plugin is missing, and therefore the WHERE clauses are not intelli-sensed. Let me know your thoughts. I appreciate it! EDIT 2: Not sure if it has anything to do with this. Bottom Right of GUI: The white values show up with it on. If I turn it off, only the green values show. In fact, this looks redundant, since with it off the native GUI already polls these correctly (shows 2 CPUs), where the plugin only has the ability to show 1 CPU at a time. Np. Heh, I don't have any database background so don't be surprised if you find some wierd queries in there. What happens if you run the `sensors` command in unraid terminal? Many of the panels have hard-coded stuff in them that I'm removing so they can be more dynamic. Edited September 11, 2020 by GilbN Quote Link to comment
GilbN Posted September 11, 2020 Share Posted September 11, 2020 2 hours ago, falconexe said: Not much ha ha. So I am getting more and more deep into this tool and I have found a number of issues and I've made some improvements to existing panels. I am most excited about the more custom stuff I have developed. Hope to share it with you all soon. Hmm, which cpu do you have? AMD? Heres, what I have so far btw: https://cdn.discordapp.com/attachments/397193259476058113/753738500217765969/Unraid_System_Dashboard_NEW-1599775585538.json Quote Link to comment
hermy65 Posted September 11, 2020 Author Share Posted September 11, 2020 (edited) @falconexe Looking good! are you planning on releasing your dashboards once you are satisfied with them? Edited September 11, 2020 by hermy65 Quote Link to comment
ChatNoir Posted September 12, 2020 Share Posted September 12, 2020 Looking forward testing this. A SpaceInvaderOne tutorial would be great to help lots of us to understand what need to be adapted to our specific configuration and how exactly. Quote Link to comment
GilbN Posted September 12, 2020 Share Posted September 12, 2020 7 hours ago, falconexe said: UPDATED IN VERSION 1.1 I FIXED THE BROKEN disk trending queries. They had some wacky hard coded values for DAY/WEEK/MONTH/YEAR. So I dove into the documentation and figured it out. Documentation: Link: https://grafana.com/docs/grafana/latest/panels/queries/ The other fun thing was that where you edit this in the query window is a little hidden. It is actually in the "Query Options" area in light blue. The relative time interval field shows "now" (current date/time) divided by the time frame (d/w/M/y) variable. This is then calculated when the "Calculation" field is set to "Difference" (Right Side Panel of Query Editor). This basically says show me the difference between today (now) and all data points up until the last data point in that time frame interval. In order get all 4 date ranges synced up, I had to calculate the total seconds in each date range and then divide by my interval (10 seconds). The original dashboard was using a mean() function to aggregate the average. This was very bad because it breaks the dashboard pulling so much data. I was using 4.5GB of RAM just on the 1 tab. Once I changed the selection to distinct(), it effectively said take the very last record from the aggregate table. This is the very last "difference" between the last 20 seconds. Once I made this change, these calculations were instantaneous! No more lag! Day: Week: Month: Year: Fun Note: The disk path of "user0" is equivalent to your entire array storage minus parity/cache/appdata (shares only). And For the Results... Great job! I was having issues with those panels and couldn't figure out how to do them correctly. Looking forward to take a look at your dash 😁 Quote Link to comment
GilbN Posted September 12, 2020 Share Posted September 12, 2020 A little teaser 2 Quote Link to comment
FreeMan Posted September 12, 2020 Share Posted September 12, 2020 @falconexe & @GilbN If the two of you figure out how to package this as a docker with some "easy" to follow instructions on setup & config (understanding that everyone's setup is different), you'll be the coolest kids at the playground for at least a month!! 1 Quote Link to comment
GilbN Posted September 12, 2020 Share Posted September 12, 2020 10 minutes ago, FreeMan said: @falconexe & @GilbN If the two of you figure out how to package this as a docker with some "easy" to follow instructions on setup & config (understanding that everyone's setup is different), you'll be the coolest kids at the playground for at least a month!! Well all this is setup by using 3 containers. Telegraf, InfluxDB and Grafana. You can take a look here https://technicalramblings.com/blog/how-to-setup-grafana-influxdb-and-telegraf-to-monitor-your-unraid-system/ Though I'm gonna rewrite parts of that guide when I'm finished with the dashboard. But the installation and setup of the containers will be the same. Quote Link to comment
FreeMan Posted September 12, 2020 Share Posted September 12, 2020 1 minute ago, GilbN said: You can take a look here Have the tab open already but haven't started reading quite yet. I'd better get working on the house update project first - if I don't it may be several more days and the wife will be a bit upset with me. 🤣 Quote Link to comment
FreeMan Posted September 12, 2020 Share Posted September 12, 2020 (edited) OK, I got the dockers set up. @GilbN thanks for your guide! Easy as pie, thouh you may want to make a note in your blog about setting it up that when you install telegraf, since it doesn't find the telegraf.conf file, it creates a directory named that. The directory has to be removed & replaced with the telegraph.conf file which then needs to be edited. This kept me quite confused for a bit. @falconexe - There's probably something just as simple for fixing this... I downloaded the v1.1 JSON file you posted. I was able to upload it just fine and everything comes up, except it throws an error that says it can't find MassEffect. That makes perfect sense, since that's your server name, not mine. Where the heck do I go to tweak that? There's nothing listed in the "host" drop-down menu: and I haven't (yet) been able to figure that one out. It does, though, list my server (NAS) for all the other options: Edited September 12, 2020 by FreeMan Quote Link to comment
GilbN Posted September 13, 2020 Share Posted September 13, 2020 11 hours ago, FreeMan said: OK, I got the dockers set up. @GilbN thanks for your guide! Easy as pie, thouh you may want to make a note in your blog about setting it up that when you install telegraf, since it doesn't find the telegraf.conf file, it creates a directory named that. The directory has to be removed & replaced with the telegraph.conf file which then needs to be edited. This kept me quite confused for a bit. You didn't see the big red warning box? 😛 I'll add a note on how to fix if you skip that step, thanks 👍🏻 1 Quote Link to comment
GilbN Posted September 13, 2020 Share Posted September 13, 2020 8 hours ago, falconexe said: @FreeMan Simply open the drop down, and in the blank line, type "NAS" and then it will automatically make everything work instantly. That changes the following lines in the JSON with that variable... Yeah I noticed that too. I think you missed to select the datasource in the variable. Set it to the other variable where you select the database Quote Link to comment
GilbN Posted September 13, 2020 Share Posted September 13, 2020 @falconexe And option 3 sound good to me! Great work! 1 Quote Link to comment
FreeMan Posted September 13, 2020 Share Posted September 13, 2020 11 hours ago, falconexe said: Simply open the drop down, and in the blank line, type "NAS" and then it will automatically make everything work instantly. Well, not quite "everything", but a fair chunk! I'd hit that drop down and saw that the box was empty - never occurred to me to just type the server name in. I'd been doing construction - working on fixing the porch ceiling - earlier yesterday, so my brain wasn't fully in IT mode. That's my excuse and I'm sticking with it! 🤣 Oddly, I'm not getting any disk space usage, which was the whole point of this exercise. I'm getting disk temps and I/O, but not for the cache drives. I've got a bit of tweaking to do, but I love it so far. Quote Link to comment
hermy65 Posted September 13, 2020 Author Share Posted September 13, 2020 @falconexe yeah man im still here, just trying to take in what you have done here so far. Probably going to start playing with what you have here tomorrow and see what we can accomplish here. Great work again! Quote Link to comment
JaseNZ Posted September 15, 2020 Share Posted September 15, 2020 This is where I ended up with mine. Quote Link to comment
Holaf Posted September 19, 2020 Share Posted September 19, 2020 (edited) this topic reminded me that I did set up a simple grafana dashboard a long time ago 😅 I have more than 2 years of logs now ... Edited September 19, 2020 by Holaf Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.