grumpy Posted April 2 Share Posted April 2 I have had no luck installing Varken; I think it may be a dead project. Github is 4-6 years old, discord link both at Grafanna and GitHub dead, Wiki 304... Any suggestions? Quote Link to comment
falconexe Posted April 2 Author Share Posted April 2 (edited) 1 hour ago, grumpy said: I have had no luck installing Varken; I think it may be a dead project. Github is 4-6 years old, discord link both at Grafanna and GitHub dead, Wiki 304... Any suggestions? Varken is no longer being developed, but I still see it in Community Applications, and it doesn't appear to be officially deprecated. Do you see this? This is the way it is installed. Edited April 2 by falconexe Quote Link to comment
falconexe Posted April 2 Author Share Posted April 2 (edited) While I was researching how to currently install Varken, I noticed that it only supports InfluxDB 1.8. From the Varken GitHub: https://github.com/Boerderij/Varken So, if we were to move the UUD to Influx 3.X using SQL, it would require someone to pick up the mantel on Varken, and branch it. If anyone is serious about it, and willing to continue that project, I'm willing to try and get the UUD to Influx 3.X in tandem, once they release it and we can get our hands on it. Once Varken's repository is gone gone, the main part of the UUD that I use (Plex) would need something else to get the Tautulli data into InfluxDB. I'm open to suggestions. We don't necessarily need Varken, we just need the code equivalent that moves the data. Perhaps a direct API call from Tautulli to InfluxDB? Let me know your thoughts (anyone). @Jufy111 @viktortras Edited April 2 by falconexe 2 Quote Link to comment
Scheev Posted April 2 Share Posted April 2 Need some advice on here, Ive been bashing my head for awhile trying to get this all working. I am finding that all the guides are fairly outdated. When I use telegraf 1.20.2 it doesn't gather metrics cause the only way ive found to get the config provides the latest config. Using an older config wouldn't exactly solve my problem either as explained below. Switching to the latest telegraf branch gives me permissions errors and won't start. Ive tried to do some chmod antics but can't seem to get the hang of where or what this is supposed to be. In my post arguments I have it running apt install arguments that have been outlined in guides and such. Would that be the culprit? how do I elevate permissions for this? E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied) I imagine if telegraf started I could likely configure it to give metrics to influx. Next problem, my influxDB docker is running influxdb v2 because I have a OPNsense instance that is posting metrics to Influx and requires it. However, without telegraf starting I can't even use the influxdb_v2 inputs in the config. As well, how would I configure influx if I have two different instances of telegraf providing metrics to the database? Thanks for any and all help! Quote Link to comment
Jufy111 Posted April 2 Share Posted April 2 (edited) 5 hours ago, falconexe said: While I was researching how to currently install Varken, I noticed that it only supports InfluxDB 1.8. From the Varken GitHub: https://github.com/Boerderij/Varken So, if we were to move the UUD to Influx 3.X using SQL, it would require someone to pick up the mantel on Varken, and branch it. If anyone is serious about it, and willing to continue that project, I'm willing to try and get the UUD to Influx 3.X in tandem, once they release it and we can get our hands on it. Once Varken's repository is gone gone, the main part of the UUD that I use (Plex) would need something else to get the Tautulli data into InfluxDB. I'm open to suggestions. We don't necessarily need Varken, we just need the code equivalent that moves the data. Perhaps a direct API call from Tautulli to InfluxDB? Let me know your thoughts (anyone). @Jufy111 @viktortras It's possible to enable the influxdb v1 API on v2, but it requires some terminal work set up the backwards compatible bucket and endpoint. https://docs.influxdata.com/influxdb/v2/reference/api/influxdb-1x/ Hopefully the backwards compatibility remains with the release of V3. I'll have a look at it and see how hard it would be to modify the post functions for v2/sql support. But I'm no dev, so I'm probably out of my depth. **edit It actually doesn't look too hard to modify the payload in varken to make it compatible with influxdb2 or SQL. If I find some spare time someday I might fork it and make a version that is natively compatible with both influxdb v1 and v2. Edited April 2 by Jufy111 1 Quote Link to comment
Jufy111 Posted April 4 Share Posted April 4 On 4/3/2024 at 5:12 AM, falconexe said: While I was researching how to currently install Varken, I noticed that it only supports InfluxDB 1.8. From the Varken GitHub: https://github.com/Boerderij/Varken So, if we were to move the UUD to Influx 3.X using SQL, it would require someone to pick up the mantel on Varken, and branch it. If anyone is serious about it, and willing to continue that project, I'm willing to try and get the UUD to Influx 3.X in tandem, once they release it and we can get our hands on it. Once Varken's repository is gone gone, the main part of the UUD that I use (Plex) would need something else to get the Tautulli data into InfluxDB. I'm open to suggestions. We don't necessarily need Varken, we just need the code equivalent that moves the data. Perhaps a direct API call from Tautulli to InfluxDB? Let me know your thoughts (anyone). @Jufy111 @viktortras Also I don't think it is dead. The dockerhub shows a nightly build updated 18 hours prior to this post. https://hub.docker.com/r/boerderij/varken/tags Quote Link to comment
Kilrah Posted April 4 Share Posted April 4 (edited) 2 hours ago, Jufy111 said: Also I don't think it is dead. The dockerhub shows a nightly build updated 18 hours prior to this post. https://hub.docker.com/r/boerderij/varken/tags That's just because there's still an automated action building the code every night even though it hasn't changed in 4 years. Edited April 4 by Kilrah Quote Link to comment
1471 Posted April 4 Share Posted April 4 (edited) On 4/2/2024 at 11:32 PM, Jufy111 said: It's possible to enable the influxdb v1 API on v2, but it requires some terminal work set up the backwards compatible bucket and endpoint. https://docs.influxdata.com/influxdb/v2/reference/api/influxdb-1x/ Hopefully the backwards compatibility remains with the release of V3. I'll have a look at it and see how hard it would be to modify the post functions for v2/sql support. But I'm no dev, so I'm probably out of my depth. **edit It actually doesn't look too hard to modify the payload in varken to make it compatible with influxdb2 or SQL. If I find some spare time someday I might fork it and make a version that is natively compatible with both influxdb v1 and v2. Its not something I have the skills to contribute to, but thanks for looking, Varken has been really useful over the years. Sounds like updating Varken at some point would probably be the easiest option for the Ultimate UNRAID Dashboard? If not prometheus-plex-exporter might be another route, would also need some adapting though. Blog post about it HERE, github repository HERE for info. Edited April 4 by 1471 Quote Link to comment
PJsUnraid Posted April 4 Share Posted April 4 (edited) For some reason my Plex statistics shows N/A until i restart Varken, then it reoccurs every 5ish minutes.. any ideas? Edit: more specifically the total values (total movies, total seasons, total shows etc). The rest seems to be fine Editx2: Looks like it only updates when tautulli starts streams, then goes back to N/A after a few mins Edited April 4 by PJsUnraid Quote Link to comment
grumpy Posted April 4 Share Posted April 4 My original post was poorly worded, I had Varken installed; but it would not start. I got past that by changing owner of the config.example from root to nobody, why it interfered with the required config.ini beats me. Every time I edit the config and remove some stuff when I restart it replaces the removed stuff. There needs to be an updated How to! Which is true of many contributions. Quote Link to comment
grumpy Posted April 4 Share Posted April 4 So now that I have all components running as expected (I think) I have more questions. Did you setup a Varken database in influx as your Dashboard variable tab picture suggests you did. As the guides I followed does not suggest that but to use the Telegraph database. I have no Plex (no media) information showing on my Dashboard but as far as I can tell Telegraph, Taututlli and Varken are all processing the data as seen through the log of each. I do not have IPMI so how do I get that information? (TEMPS) Array Drive, System Temps/Power, CPU, Ramm DIMM, Fan Speeds I use NUTs(slave) for the UPS; the information is shown in Uraid but not in the dashboard. How to get that? SSD Temp, Life, Power On, Lifetime reads/writes, are missing. Quote Link to comment
Jufy111 Posted April 4 Share Posted April 4 5 hours ago, PJsUnraid said: For some reason my Plex statistics shows N/A until i restart Varken, then it reoccurs every 5ish minutes.. any ideas? Edit: more specifically the total values (total movies, total seasons, total shows etc). The rest seems to be fine Editx2: Looks like it only updates when tautulli starts streams, then goes back to N/A after a few mins I have come across issues with varken on consistent writing data to the database. It is suppose to write points every 30s, but it does not. In the in influx V2 version of UUD that I've been working I've modified the queries so that it gets the the most recent result from the last 10 minutes. You could try and do the same to the v1 queries. Quote Link to comment
Jufy111 Posted April 4 Share Posted April 4 (edited) 54 minutes ago, grumpy said: So now that I have all components running as expected (I think) I have more questions. Did you setup a Varken database in influx as your Dashboard variable tab picture suggests you did. As the guides I followed does not suggest that but to use the Telegraph database. I have no Plex (no media) information showing on my Dashboard but as far as I can tell Telegraph, Taututlli and Varken are all processing the data as seen through the log of each. I do not have IPMI so how do I get that information? (TEMPS) Array Drive, System Temps/Power, CPU, Ramm DIMM, Fan Speeds I use NUTs(slave) for the UPS; the information is shown in Uraid but not in the dashboard. How to get that? SSD Temp, Life, Power On, Lifetime reads/writes, are missing. The dashboard variable just points to a database, not a bucket. The system stats are collected by telegraf and stired in the 'system' bucket. You will need to configure the telegraf config file to collect all the system stats. The ups info is also configured in the telegraf config. Do you get hdd stats (smart data, not dusk usage and IO? My guess is telegraf config is not configured properly. Or you haven't installed smartmontools in the telegraf container. The telegraf config is quite long, but most of it is just comments. Once I'm back at my computer, I'll post a shortened version for you. Edited April 4 by Jufy111 Quote Link to comment
Jufy111 Posted April 4 Share Posted April 4 12 hours ago, Kilrah said: That's just because there's still an automated action building the code every night even though it hasn't changed in 4 years. Ah bummer, I might have a go at forking varken and updating it to be compatible with influx 2.0. There is an outstanding pull request on GitHub that apparently enables it and works, so I might just see if I can fork it and manually merge the PR. Quote Link to comment
grumpy Posted April 4 Share Posted April 4 @Jufy111 Thank You, I'm looking forward to your example, as far as the smartmontools. I think it is installed with the docker image post argument; which is how I did it. /bin/sh -c 'apt update && apt install -y smartmontools && apt install -y lm-sensors && telegraf' --user 0 I do get the panels: Drive SMART Health Overview, Drive Life Quote Link to comment
grumpy Posted April 4 Share Posted April 4 I'm an idiot; ok things I do not know. I needed to add a varken data source connection, not just telegraph now I see Plex stuff; not everything yet, but at least moving forward. Quote Link to comment
Jufy111 Posted April 4 Share Posted April 4 (edited) 16 minutes ago, grumpy said: @Jufy111 Thank You, I'm looking forward to your example, as far as the smartmontools. I think it is installed with the docker image post argument; which is how I did it. /bin/sh -c 'apt update && apt install -y smartmontools && apt install -y lm-sensors && telegraf' --user 0 I do get the panels: Drive SMART Health Overview, Drive Life Hmm, not sure about the SSD data then. Here is my telegraf config file (all though I'm using influxdb2 so that section will look different for you). It works for me, but still may not work for you. Full disclosure too, I've not actually tested UUD1.7, as it's all in InfluxDB v1 queries. I'm slowly changing them over to v2. Spoiler [agent] interval = "10s" round_interval = true metric_batch_size = 1000 metric_buffer_limit = 10000 collection_jitter = "0s" flush_interval = "10s" flush_jitter = "0s" precision = "" debug = false quiet = false logfile = "" hostname = "" omit_hostname = false [[outputs.influxdb_v2]] urls = ["http://192.168.100.100:8086"] token = "your-token-here" organization = "your-org-here" bucket = "system" [[inputs.cpu]] percpu = true totalcpu = true collect_cpu_time = false report_active = false [[inputs.disk]] ignore_fs = ["tmpfs", "devtmpfs", "devfs", "overlay", "aufs", "squashfs"] [[inputs.diskio]] [[inputs.docker]] endpoint = "unix:///var/run/docker.sock" gather_services = false container_names = [] container_name_include = [] container_name_exclude = [] timeout = "5s" perdevice = true total = true docker_label_include = [] docker_label_exclude = [] [[inputs.mem]] [[inputs.net]] # interfaces = ["eth0","br-0d215e2e2923","br-8be3c4b3369e"] [[inputs.processes]] [[inputs.swap]] [[inputs.system]] [[inputs.smart]] use_sudo = false # path_smartctl = "/usr/bin/smartctl" attributes = true [[inputs.sensors]] [[inputs.apcupsd]] Edited April 4 by Jufy111 Quote Link to comment
Jufy111 Posted April 4 Share Posted April 4 On 4/3/2024 at 8:58 AM, Scheev said: Need some advice on here, Ive been bashing my head for awhile trying to get this all working. I am finding that all the guides are fairly outdated. When I use telegraf 1.20.2 it doesn't gather metrics cause the only way ive found to get the config provides the latest config. Using an older config wouldn't exactly solve my problem either as explained below. Switching to the latest telegraf branch gives me permissions errors and won't start. Ive tried to do some chmod antics but can't seem to get the hang of where or what this is supposed to be. In my post arguments I have it running apt install arguments that have been outlined in guides and such. Would that be the culprit? how do I elevate permissions for this? E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied) I imagine if telegraf started I could likely configure it to give metrics to influx. Next problem, my influxDB docker is running influxdb v2 because I have a OPNsense instance that is posting metrics to Influx and requires it. However, without telegraf starting I can't even use the influxdb_v2 inputs in the config. As well, how would I configure influx if I have two different instances of telegraf providing metrics to the database? Thanks for any and all help! I'm running influx v2 as well. Telegraf 1.20.2 was the last version before the devs locked down the permission an made it impossible to install smartmontools and other packets in the container. Any release newer than this, and you will have to remove the install argument for smartmontools and remove [inputs.smart] out of the telegraf config. This will sacrifice any smart data. I don't use OPNsense, but I just have my pfsense configured to write to a bucket called 'pfsense'. Pfsense has it's of telegraf instance. I assume that OPNsense is the same I'm still not quite sure why you need the newer version of telegraf. OPNsense should have it's own telegraf instance pointed to separate bucket in your influxdb 2 instance. This should leave the docker telegraf free to be installed as v 1.20.2 See my post above for my telegraf :1.20.2 config that is configured for infludb v2 1 Quote Link to comment
Scheev Posted April 4 Share Posted April 4 24 minutes ago, Jufy111 said: I'm running influx v2 as well. Telegraf 1.20.2 was the last version before the devs locked down the permission an made it impossible to install smartmontools and other packets in the container. Any release newer than this, and you will have to remove the install argument for smartmontools and remove [inputs.smart] out of the telegraf config. This will sacrifice any smart data. I don't use OPNsense, but I just have my pfsense configured to write to a bucket called 'pfsense'. Pfsense has it's of telegraf instance. I assume that OPNsense is the same I'm still not quite sure why you need the newer version of telegraf. OPNsense should have it's own telegraf instance pointed to separate bucket in your influxdb 2 instance. This should leave the docker telegraf free to be installed as v 1.20.2 See my post above for my telegraf :1.20.2 config that is configured for infludb v2 I was able to figure it out, but my only hurdle is related to the use of InfluxQL for the dashboard instead of Flux. Is there a setting that will alleviate this? I started rewriting the dashboard in Flux...but realized theres probably another way around it. Quote Link to comment
Jufy111 Posted April 4 Share Posted April 4 (edited) 12 minutes ago, Scheev said: I was able to figure it out, but my only hurdle is related to the use of InfluxQL for the dashboard instead of Flux. Is there a setting that will alleviate this? I started rewriting the dashboard in Flux...but realized theres probably another way around it. Give it a few days. I'm currently working on updating all the queries to Flux for UUD1.7. I havent been able to find a way around it. Edited April 4 by Jufy111 Quote Link to comment
Scheev Posted April 4 Share Posted April 4 1 hour ago, Jufy111 said: Give it a few days. I'm currently working on updating all the queries to Flux for UUD1.7. I havent been able to find a way around it. Oh awesome, I spent like 4 hours today converting some of it already, happy to help if you want. Quote Link to comment
Jufy111 Posted April 5 Share Posted April 5 (edited) 2 hours ago, Scheev said: Oh awesome, I spent like 4 hours today converting some of it already, happy to help if you want. Thanks, I'm almost done I think. Could definitely use your help for testing though. I also have a few questions for @falconexe too. Edited April 5 by Jufy111 Quote Link to comment
PJsUnraid Posted April 5 Share Posted April 5 16 hours ago, Jufy111 said: I have come across issues with varken on consistent writing data to the database. It is suppose to write points every 30s, but it does not. In the in influx V2 version of UUD that I've been working I've modified the queries so that it gets the the most recent result from the last 10 minutes. You could try and do the same to the v1 queries. Yeah my knowledge with all this is very limited lol. I'll wait and monitor the progress of the v2 influx for the time being Quote Link to comment
grumpy Posted April 5 Share Posted April 5 (edited) On 4/4/2024 at 9:23 AM, PJsUnraid said: For some reason my Plex statistics shows N/A until i restart Varken, then it reoccurs every 5ish minutes.. any ideas? Edit: more specifically the total values (total movies, total seasons, total shows etc). The rest seems to be fine Editx2: Looks like it only updates when tautulli starts streams, then goes back to N/A after a few mins @PJsUnraid I do not believe it is Varken causing the issue, I think it is a timing issue. If you look at the influx database you will see all of plex data being updated constantly; not sure if it is 30sec or not but those metrics you are missing are done hourly. So you no longer see N/A go to the panels that show Total 3 dots on top right drop down - edit now in edit mode for the panel top left side click on query options Relative time change to 62m or 1h top right of page Apply to close panel Now it will show the number in a persistent state till it changes on the next query. Edited April 5 by grumpy SP Quote Link to comment
grumpy Posted April 5 Share Posted April 5 Now after watching the panel for 16 hours, and correcting some issues for myself I have noticed some other issues that go beyond my understanding. Stream Log (Overview) misses some entries. In music playing from Plex it can keep up sometimes, skip a couple of songs, or even skip 10's of songs. Probably not noticed during movies or tv shows; I do not know as I haven't made it that far during testing. Stream Log (Detailed) Keeps up even though there are delays, to be expected Plex -> Tautulli -> Varken -> Influx, same with Current Streams. During Music play percentage is of no use, while the song can have 72% remaining when in fact the next song has started playing. Even though most of my posts seem like complaints I really like to thank @falconexe for all of his hard work, it is functional and visual pleasing. I know I wouldn't have been able to do this on my own accord. 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.