axipher Posted November 2, 2021 Share Posted November 2, 2021 I'm running on the 'telegraf:alpine' default tag and had to make the following changes for it to work, this resulted in running 'Telegraf 1.20.3' per the docker logs. 1) Remove the following 'Post Arguments' under 'Advanced View'. This gets rid of the 'apk error' but also gets rid of 'smartmontools' which means you will lose out on some disk stats for certain dashboards in Grafana. /bin/sh -c 'apk update && apk upgrade && apk add smartmontools && telegraf' 2) Open up '/mnt/user/appdata/telegraf/telegraf.conf' and comment out the following two lines. This gets rid of the 'missing smartctl' errors and also gets rid of the constant error for not being able to access 'docker.sock'. There might be a fix for the docker problem if someone can share it as I find it really useful to monitor the memory usage of each docker from Grafana but currently had to give that up on current version of Telegraf #[[inputs.smart]] #[[inputs.docker]] As others stated, reverting to an older version works as well if the new Telegraf provides nothing new for your use case and would be the recommended route, but I just wanted to document the couple things I had to change to get the latest Telegraf docker running again alongside InfluxDB 1.8 (tag: influxdb:1.8) and Grafana v8 (tag: grafana/grafana:8.0.2). At this point I will probably spend next weekend locking most of my dockers in to current versions and setup a trial Unraid box on an old Dell Optiplex for testing latest dockers before upgrading on my main system. Quote Link to comment
Enver Posted November 3, 2021 Share Posted November 3, 2021 Hey Guys, Telegraf has been logging the below over and over again whilst its running. This has been happening for the last week or so. I have tried downgrading Telegraf but that doesn't seem to fix the problem. I have deleted Telegraf and reinstalled it however the problem persists. Any advice you can provide would be greatly appreciated. Running: unRAID 6.9.2 InfluxDB:1.8.4-alpine Telegraf:latest 1.20.3 2021-11-03T10:06:58Z I! [agent] Hang on, flushing any cached metrics before shutdown 2021-11-03T10:06:58Z I! [agent] Stopping running outputs 2021-11-03T10:06:59Z I! Starting Telegraf 1.20.3 2021-11-03T10:06:59Z I! Using config file: /etc/telegraf/telegraf.conf 2021-11-03T10:06:59Z I! Loaded inputs: apcupsd cpu disk diskio docker kernel mem modbus net net_response netstat nginx processes sensors smart swap system temp 2021-11-03T10:06:59Z I! Loaded aggregators: 2021-11-03T10:06:59Z I! Loaded processors: 2021-11-03T10:06:59Z I! Loaded outputs: influxdb 2021-11-03T10:06:59Z I! Tags enabled: host=Tower 2021-11-03T10:06:59Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"Tower", Flush Interval:10s 2021-11-03T10:06:59Z W! [inputs.smart] nvme not found: verify that nvme is installed and it is in your PATH (or specified in config) to gather vendor specific attributes: provided path does not exist: [] 2021-11-03T10:06:59Z W! [inputs.docker] 'perdevice' setting is set to 'true' so 'blkio' and 'network' metrics will be collected. Please set it to 'false' and use 'perdevice_include' instead to control this behaviour as 'perdevice' will be deprecated 2021-11-03T10:06:59Z W! [inputs.docker] 'total' setting is set to 'false' so 'blkio' and 'network' metrics will not be collected. Please set it to 'true' and use 'total_include' instead to control this behaviour as 'total' will be deprecated 2021-11-03T10:07:00Z E! [inputs.docker] Error in plugin: Got permission denied while trying to connect to the Docker daemon socket at unix:///rootfs/var/run/docker.sock: Get "http://%2Frootfs%2Fvar%2Frun%2Fdocker.sock/v1.21/info": dial unix /rootfs/var/run/docker.sock: connect: permission denied 2021-11-03T10:07:00Z E! [inputs.docker] Error in plugin: Got permission denied while trying to connect to the Docker daemon socket at unix:///rootfs/var/run/docker.sock: Get "http://%2Frootfs%2Fvar%2Frun%2Fdocker.sock/v1.21/containers/json?filters=%7B%22status%22%3A%5B%22running%22%5D%7D&limit=0": dial unix /rootfs/var/run/docker.sock: connect: permission denied 2021-11-03T10:07:10Z E! [inputs.docker] Error in plugin: Got permission denied while trying to connect to the Docker daemon socket at unix:///rootfs/var/run/docker.sock: Get "http://%2Frootfs%2Fvar%2Frun%2Fdocker.sock/v1.21/info": dial unix /rootfs/var/run/docker.sock: connect: permission denied Quote Link to comment
cjlee89 Posted November 3, 2021 Share Posted November 3, 2021 (edited) 12 hours ago, Enver said: Hey Guys, Telegraf has been logging the below over and over again whilst its running. This has been happening for the last week or so. I have tried downgrading Telegraf but that doesn't seem to fix the problem. I have deleted Telegraf and reinstalled it however the problem persists. Any advice you can provide would be greatly appreciated. Running: unRAID 6.9.2 InfluxDB:1.8.4-alpine Telegraf:latest 1.20.3 2021-11-03T10:06:58Z I! [agent] Hang on, flushing any cached metrics before shutdown 2021-11-03T10:06:58Z I! [agent] Stopping running outputs 2021-11-03T10:06:59Z I! Starting Telegraf 1.20.3 2021-11-03T10:06:59Z I! Using config file: /etc/telegraf/telegraf.conf 2021-11-03T10:06:59Z I! Loaded inputs: apcupsd cpu disk diskio docker kernel mem modbus net net_response netstat nginx processes sensors smart swap system temp 2021-11-03T10:06:59Z I! Loaded aggregators: 2021-11-03T10:06:59Z I! Loaded processors: 2021-11-03T10:06:59Z I! Loaded outputs: influxdb 2021-11-03T10:06:59Z I! Tags enabled: host=Tower 2021-11-03T10:06:59Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"Tower", Flush Interval:10s 2021-11-03T10:06:59Z W! [inputs.smart] nvme not found: verify that nvme is installed and it is in your PATH (or specified in config) to gather vendor specific attributes: provided path does not exist: [] 2021-11-03T10:06:59Z W! [inputs.docker] 'perdevice' setting is set to 'true' so 'blkio' and 'network' metrics will be collected. Please set it to 'false' and use 'perdevice_include' instead to control this behaviour as 'perdevice' will be deprecated 2021-11-03T10:06:59Z W! [inputs.docker] 'total' setting is set to 'false' so 'blkio' and 'network' metrics will not be collected. Please set it to 'true' and use 'total_include' instead to control this behaviour as 'total' will be deprecated 2021-11-03T10:07:00Z E! [inputs.docker] Error in plugin: Got permission denied while trying to connect to the Docker daemon socket at unix:///rootfs/var/run/docker.sock: Get "http://%2Frootfs%2Fvar%2Frun%2Fdocker.sock/v1.21/info": dial unix /rootfs/var/run/docker.sock: connect: permission denied 2021-11-03T10:07:00Z E! [inputs.docker] Error in plugin: Got permission denied while trying to connect to the Docker daemon socket at unix:///rootfs/var/run/docker.sock: Get "http://%2Frootfs%2Fvar%2Frun%2Fdocker.sock/v1.21/containers/json?filters=%7B%22status%22%3A%5B%22running%22%5D%7D&limit=0": dial unix /rootfs/var/run/docker.sock: connect: permission denied 2021-11-03T10:07:10Z E! [inputs.docker] Error in plugin: Got permission denied while trying to connect to the Docker daemon socket at unix:///rootfs/var/run/docker.sock: Get "http://%2Frootfs%2Fvar%2Frun%2Fdocker.sock/v1.21/info": dial unix /rootfs/var/run/docker.sock: connect: permission denied I had the same error and no data in Grafana. I had to downgrade to "telegraf:1.20.2-alpine". Previously using "telegraf:alpine". Seems like 1.20.3 breaks the Docker plugin with some type of permission problem. Edited November 3, 2021 by cjlee89 1 Quote Link to comment
joshallen2k Posted November 4, 2021 Share Posted November 4, 2021 Hi - I just updated to Unraid 6.10.rc-2. I noticed that I have new Docker container named "focused_diffie". The container is stopped, and when I hit its associated docker page it goes to the Telegraf page. I do indeed have Telegraf installed. The support link brings me here. Googling the docker name itself comes up empty. Can someone tell me what this is and if its safe to delete? Thanks! Quote Link to comment
ziggie216 Posted November 4, 2021 Share Posted November 4, 2021 Looks like there was a change with telegraf 1.20.3 where it's no longer running as root, which is now causing an issue for anyone who is running post argument. https://www.influxdata.com/blog/docker-run-telegraf-as-non-root/ Any idea what we need to do in order for us to use telegraf 1.20.3 and going forth? I'm not sure what to do about this user permission part Quote Link to comment
VladoPortos Posted November 4, 2021 Share Posted November 4, 2021 7 hours ago, joshallen2k said: Hi - I just updated to Unraid 6.10.rc-2. I noticed that I have new Docker container named "focused_diffie". The container is stopped, and when I hit its associated docker page it goes to the Telegraf page. I do indeed have Telegraf installed. The support link brings me here. Googling the docker name itself comes up empty. Can someone tell me what this is and if its safe to delete? Thanks! Hi, Quote focused_diffie is a random-generated name for container by docker. It gives it funny names if the name is not specified. If it is stopped (and you are not missing any previous running service) I would delete it. It looks like there was a template error or something. Quote Link to comment
EDalcin Posted November 6, 2021 Share Posted November 6, 2021 Hi folks, I installed InfluxDB a few days ago and created a few boards and charts. Last night the docker updates and I missed everything! It starts all over, asking me to create a new account. What I did wrong? How can I avoid it? Thanks in advance. Eduardo Quote Link to comment
axipher Posted November 6, 2021 Share Posted November 6, 2021 3 hours ago, EDalcin said: Hi folks, I installed InfluxDB a few days ago and created a few boards and charts. Last night the docker updates and I missed everything! It starts all over, asking me to create a new account. What I did wrong? How can I avoid it? Thanks in advance. Eduardo Which docker updated? InfluxDB has some changes from 1.8 to 2.0 that can break other dockers that assume basic Authorization (from my understanding) I've locked mine to: 'influxdb:1.8' Grafana has some changes form v7 to v8 that can break some dashboards and plugins using lots of the options or thresholds and value mappings I've locked mine to: 'grafana/grafana:8.0.2' Telegraf's most recent update also breaks collection of SMART and Docker statistics. I've locked mine to: 'telegraf:1.20.2-alpine' By locking the versions, then I know updates won't break those connections. It does mean you might miss out on important updates later on, so keeping tabs on those docker's development pages, or on here is still a good idea. 1 1 Quote Link to comment
Luddi Posted November 8, 2021 Share Posted November 8, 2021 On 11/6/2021 at 11:06 AM, EDalcin said: Hi folks, I installed InfluxDB a few days ago and created a few boards and charts. Last night the docker updates and I missed everything! It starts all over, asking me to create a new account. What I did wrong? How can I avoid it? Thanks in advance. Eduardo Hi, I noticed the same thing after updating and thereby restarting the container. The problem was that /var/lib/influxdb2 in the container was not mapped to a path on the host. This path is new in addition to /var/lib/influxdb. See Upgrade Example - Minimal here for example. To fix it add a path like this Container Path: /var/lib/influxdb2 Host Path: /mnt/user/appdata/influxdb2 1 Quote Link to comment
Shantarius Posted November 8, 2021 Share Posted November 8, 2021 (edited) On 11/2/2021 at 12:29 PM, axipher said: I'm running on the 'telegraf:alpine' default tag and had to make the following changes for it to work, this resulted in running 'Telegraf 1.20.3' per the docker logs. 1) Remove the following 'Post Arguments' under 'Advanced View'. This gets rid of the 'apk error' but also gets rid of 'smartmontools' which means you will lose out on some disk stats for certain dashboards in Grafana. /bin/sh -c 'apk update && apk upgrade && apk add smartmontools && telegraf' 2) Open up '/mnt/user/appdata/telegraf/telegraf.conf' and comment out the following two lines. This gets rid of the 'missing smartctl' errors and also gets rid of the constant error for not being able to access 'docker.sock'. There might be a fix for the docker problem if someone can share it as I find it really useful to monitor the memory usage of each docker from Grafana but currently had to give that up on current version of Telegraf #[[inputs.smart]] #[[inputs.docker]] As others stated, reverting to an older version works as well if the new Telegraf provides nothing new for your use case and would be the recommended route, but I just wanted to document the couple things I had to change to get the latest Telegraf docker running again alongside InfluxDB 1.8 (tag: influxdb:1.8) and Grafana v8 (tag: grafana/grafana:8.0.2). At this point I will probably spend next weekend locking most of my dockers in to current versions and setup a trial Unraid box on an old Dell Optiplex for testing latest dockers before upgrading on my main system. Hello there, i have a solution for the problem with the access to /var/run/docker.sock. For me it works with telegraf:1.18.3 telegraf:alpine. You just add the following in the Extra Parameters Value: --user telegraf:$(stat -c '%g' /var/run/docker.sock) Then Telegraf has access to the docker.sock and i have some data from all dockers in Grafana (CPU Usage, RAM Usage, Network Usage). Its pretty nice 🙂 For the smartctl Problem i have no solution. If i add to Post Arguments /bin/sh -c 'apk update && apk upgrade && apk add smartmontools && telegraf' the telegraf docker doesn't starts up and in the docker log i found the error ERROR: Unable to lock database: Permission denied ERROR: Failed to open apk database: Permission denied Has anyone a solution for this? 🙂 Edited November 8, 2021 by Shantarius 1 Quote Link to comment
axipher Posted November 8, 2021 Share Posted November 8, 2021 14 minutes ago, Shantarius said: Hello there, i have a solution for the problem with the access to /var/run/docker.sock. For me it works with telegraf:alpine. You just add the following in the Extra Parameters Value: --user telegraf:$(stat -c '%g' /var/run/docker.sock) Then Telegraf has access to the docker.sock and i have some date from all dockers in Grafana (CPU Usage, RAM Usage, Network Usage). Its pretty nice 🙂 For the smartctl Problem i have no solution. If i add to Post Arguments /bin/sh -c 'apk update && apk upgrade && apk add smartmontools && telegraf' the telegraf docker doesn't starts up and in the docker log i found the error ERROR: Unable to lock database: Permission denied ERROR: Failed to open apk database: Permission denied Has anyone a solution for this? 🙂 Thanks for the Docker solution, I'll keep that in mind for if I try to go back to the current Telegraf docker once we find a solution for the smartmontools. Quote Link to comment
Fredrick Posted November 26, 2021 Share Posted November 26, 2021 (edited) On 11/8/2021 at 9:17 AM, Luddi said: I noticed the same thing after updating and thereby restarting the container. The problem was that /var/lib/influxdb2 in the container was not mapped to a path on the host. This path is new in addition to /var/lib/influxdb. See Upgrade Example - Minimal here for example. This didn't work for me. The entire /mnt/user/appdata/influxdb folder has been wiped for me, I don't know how that could have happend during the update. Just to be clear this was already a InfluxDB v2 docker, it just updated to a newer build Any ideas? Edited November 26, 2021 by Fredrick Quote Link to comment
stephenmcd1 Posted December 4, 2021 Share Posted December 4, 2021 On 6/9/2021 at 1:53 PM, dandiodati said: SpeedTestForInfluxDB seems to no longer work. I downgraded from influx 2 back to 1.8.6 since it broke too many things. But the speed test is no longer working (was working before) and get the following in the logs: It seems the SpeedTestForInfluxDB project has been retired by its owner and replaced with a new project called SpeedMon. See here: https://github.com/barrycarey/Speedtest-for-InfluxDB-and-Grafana/issues/46#issuecomment-933066819 Maybe it's time for me to learn how to make an Unraid template from scratch for SpeedMon but kind of hoping somebody else might have the time and experience to do it 🙂 Quote Link to comment
Luddi Posted December 12, 2021 Share Posted December 12, 2021 On 11/26/2021 at 9:37 PM, Fredrick said: This didn't work for me. The entire /mnt/user/appdata/influxdb folder has been wiped for me, I don't know how that could have happend during the update. Just to be clear this was already a InfluxDB v2 docker, it just updated to a newer build Any ideas? In my limited experience with unraid I can only think of the Appdata Cleanup plugin which deleted your folder. You could test for that by disabling it during the next update of influxdb. Quote Link to comment
Squid Posted December 12, 2021 Share Posted December 12, 2021 1 hour ago, Luddi said: In my limited experience with unraid I can only think of the Appdata Cleanup plugin which deleted your folder. You could test for that by disabling it during the next update of influxdb. It's 100% a manual run. Nothing automatic 1 Quote Link to comment
Squid Posted December 12, 2021 Share Posted December 12, 2021 On 12/4/2021 at 1:01 AM, stephenmcd1 said: It seems the SpeedTestForInfluxDB project has been retired by its owner and replaced with a new project called SpeedMon. See here: https://github.com/barrycarey/Speedtest-for-InfluxDB-and-Grafana/issues/46#issuecomment-933066819 Maybe it's time for me to learn how to make an Unraid template from scratch for SpeedMon but kind of hoping somebody else might have the time and experience to do it 🙂 @atribe ^^^^ Quote Link to comment
nraygun Posted December 14, 2021 Share Posted December 14, 2021 (edited) Is the coast clear on the components involved in this monitoring dashboard setup in unRaid as it relates to the recent log4j/log4shell vulnerability? Are Influxdb/Grafana/Telegraf all good, ie, no java or use of log4j? I found at least this for Grafana: https://www.reddit.com/r/homelab/comments/recvfp/list_of_known_common_homelab_apps_with_log4j_bad/?utm_source=share&utm_medium=web2x&context=3 Granted, my setup is not exposed/accessible to the external internet, but still. PS - I searched for log4j* in each docker at the console for each of these containers and didn't find anything. But I don't think that's a complete enough test. Edited December 14, 2021 by nraygun Quote Link to comment
greenflash24 Posted December 17, 2021 Share Posted December 17, 2021 On 11/8/2021 at 3:18 PM, axipher said: Thanks for the Docker solution, I'll keep that in mind for if I try to go back to the current Telegraf docker once we find a solution for the smartmontools. Are there any news in how to install smartmontools in the latest telegraf container (without creating a Dockerfile and building a seperate image)? Quote Link to comment
JaseNZ Posted December 17, 2021 Share Posted December 17, 2021 (edited) 9 minutes ago, greenflash24 said: Are there any news in how to install smartmontools in the latest telegraf container (without creating a Dockerfile and building a seperate image)? I don't know if this helps you and I have not read back to see what the issue is, I have this in my post arguments for the telegraf container and have no issue's with it. /bin/sh -c 'apt-get update && apt-get -y upgrade && apt-get -y install smartmontools && apt-get -y install lm-sensors && apt-get -y install nvme-cli && telegraf' Edited December 17, 2021 by JaseNZ Quote Link to comment
greenflash24 Posted December 17, 2021 Share Posted December 17, 2021 10 minutes ago, JaseNZ said: I don't know if this helps you and I have not read back to see what the issue is, I have this in my post arguments for the telegraf container and have no issue's with it. /bin/sh -c 'apt-get update && apt-get -y upgrade && apt-get -y install smartmontools && apt-get -y install lm-sensors && apt-get -y install nvme-cli && telegraf' Yes, with telegraf 1.20.2 i am also using these post arguments, but starting from telegraf 1.20.3 the container does not run with the root-user anymore (for security reasons), so the installation of additional packages is no longer possible. At least i don't know a soloution for this problem. Thats why i currently have to pin the image to the 1.20.2 tag and i am not able to install smartmontools while using the latest tag. Quote Link to comment
JaseNZ Posted December 17, 2021 Share Posted December 17, 2021 3 minutes ago, greenflash24 said: Yes, with telegraf 1.20.2 i am also using these post arguments, but starting from telegraf 1.20.3 the container does not run with the root-user anymore (for security reasons), so the installation of additional packages is no longer possible. At least i don't know a soloution for this problem. Thats why i currently have to pin the image to the 1.20.2 tag and i am not able to install smartmontools while using the latest tag. Ahh right fair enough Quote Link to comment
Lohita Posted December 19, 2021 Share Posted December 19, 2021 On 11/27/2021 at 7:37 AM, Fredrick said: This didn't work for me. The entire /mnt/user/appdata/influxdb folder has been wiped for me, I don't know how that could have happend during the update. Just to be clear this was already a InfluxDB v2 docker, it just updated to a newer build Any ideas? Hey @Fredrick, did you manage the figure out what went wrong? @Convington and I have the same issue and noone replied, so we're not sure how to fix this. @Convington did you manage to fix the issue yourself? Quote Link to comment
Fredrick Posted December 20, 2021 Share Posted December 20, 2021 6 hours ago, Lohita said: Hey @Fredrick, did you manage the figure out what went wrong? @Convington and I have the same issue and noone replied, so we're not sure how to fix this. I'm not, sorry. I set it up again and forgot about this problem. It does mean it's kinda likely to happen again though.. Quote Link to comment
mattbridges Posted December 23, 2021 Share Posted December 23, 2021 I'm recently (last few days) running into an error with the Glances container. docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/images/040acfe07efced14a30c3992ff559b883c6920800b2f291f7f28650d3857860d/json: Not Found ("no such image: 040acfe07efced14a30c3992ff559b883c6920800b2f291f7f28650d3857860d: No such image: sha256:040acfe07efced14a30c3992ff559b883c6920800b2f291f7f28650d3857860d") Is anyone able to point me in the right direction please? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.