[Support] Data-Monkey - netdata

[Shehan]

3 hours ago, crafty35a said:

 

The instructions I had followed were to map the /netdata/dbengine subfolder, not the entire netdata folder -- I will try it the way you show here and let you know if I lose history again tonight. Is this the only change you made from the template defaults?

 

Also, did you make the "delete obsolete charts files" change or was that not necessary?

 

Edit - I'm not quite sure how to set this up, the container won't start with this setting ('netdata ERROR : MAIN : Failed to initialize database at /var/cache/netdata/netdata-meta.db, due to "unable to open database file'). Guessing I need to copy the db over manually before starting, but I'm not clear on how to do so

you can copy the file to the new location using the command

 

docker cp netdata:/var/cache/netdata/netdata-meta.db /mnt/user/appdata/netdata/netdatacache/netdata

 

after that you need to set the proper permissions on those folders

after that netdata starts up

but i tried an update and the data is gone again

 

maybe i set the permissions wrong or something

 

Edit: I changed the permissions on all cache folders(/mnt/user/appdata/netdata/netdatacache/netdata) and now it seems to be working fine. i removed the container and installed it again and the data is still here. i dont know if this will persist through an update or not.

 

I'll post an update after an update 😁

Edited August 9, 2022 by Shehan

[Shehan]

Good news. I just updated and my data is still there. there is a little bit of data (I think what was in the ram which was not flushed to the disk before update) was lost. but everything else is still there.
thanks everyone, specially roland for helping.
👍

[BreakfastPurrito]

I just installed Netdata, and I see there is a netfilter/firewall tab. Does that mean I can install a firewall on unraid somehow?

[ChronicDiablo]

On 4/12/2022 at 2:39 PM, GeneralPILK said:

If you log in to netdata cloud you can click on the connect nodes entry at the top of the sidebar, this opens a slide out on the right side of the page where you can select the docker tab, and then copy the three extra params that are shown as new variables in the netdata template

NETDATA_CLAIM_TOKEN
NETDATA_CLAIM_URL
NETDATA_CLAIM_ROOMS

Add these 3 variables as the keys, with the values shown in the cloud UI as the values.

It took a while for the data to show up for me, but it works perfectly for me.

 

Hey man, thanks a ton for this. I've been digging around trying to figure it out for at least a few hours, and that advice got it working perfectly for me. You're a gentleman and a scholar.

[Mason736]

I recently installed NetData, and noticed none of my dockers are showing in the app.  I mainly use a custom br0 for most of my dockers that I expose to NGINX reverse proxy, which I intended to do with Netdata.  If I have Netdata on br0, rather than bridge, is that the reason its not seeing the docker containers?

[TRusselo]

On 8/26/2022 at 7:25 AM, Mason736 said:

I recently installed NetData, and noticed none of my dockers are showing in the app.  I mainly use a custom br0 for most of my dockers that I expose to NGINX reverse proxy, which I intended to do with Netdata.  If I have Netdata on br0, rather than bridge, is that the reason its not seeing the docker containers?

check your netdata docker config

read the ORANGE words.  dont copy my number 281

Edited September 14, 2022 by TRusselo

[TRusselo]

Instructions for fixing data loss after reboot/update

edit docker config with these 3 new paths

 

save the config.
stop the container
open UNRAID terminal
enter the following lines:  (it starts a temp docker of netdata, copies the files to the paths created in the last step, then removes the temp docker)
 

cd /mnt/cache/appdata/netdata/
mkdir config
mkdir cache
mkdir lib
docker run -d --name netdata_tmp netdata/netdata
docker cp netdata_tmp:/etc/netdata /mnt/user/appdata/netdata/cacheconfig/
docker cp netdata:/var/cache/netdata/netdata-meta.db /mnt/user/appdata/netdata/cache/netdata
docker cp netdata:/var/lib/netdata /mnt/user/appdata/netdata/lib
docker rm -f netdata_tmp
chmod 777 -R /mnt/user/appdata/netdata

 

you can now start your docker.

Edited September 16, 2022 by TRusselo

[dopeytree]

Getting a sh*t-ton of NGINX errors with this docker...

 

 

But also suricata packet inspection on PFsence firewall is reporting this docker is trying to connect to a .TK domain name... (can be malware call home) .tk domains can be had for free so are often used for various things.

 

 

I have now removed netdata and system is more stable..

It was locking up at night.

Edited October 7, 2022 by dopeytree

[JNZ]

I have recently installed this docker and can see an update is available. How do I know what has been updated? Is it the the template config or the docker version itself it is pulling through? 

 

Also, there is a previous comment above that details dns queries to a potentially suspicious domain. Can we verify this? Isnt this using the official netdata container so more people would have this concern as well? 

[Squid]

9 hours ago, JNZ said:

I have recently installed this docker and can see an update is available. How do I know what has been updated? Is it the the template config or the docker version itself it is pulling through? 

Netdata gets updated almost every day.  

 

https://learn.netdata.cloud/docs/agent/changelog/

[Squid]

On 10/7/2022 at 3:39 PM, dopeytree said:

Getting a sh*t-ton of NGINX errors with this docker...

 

Normal, and probably only show up when starting up the container

[dopeytree]

On 10/13/2022 at 11:16 AM, Squid said:

Normal, and probably only show up when starting up the container

@JorgeB indicated the NGNIX Errors weren't normal. They show up through out the day not just at startup. Anyway I have removed Net Data.

 

Someone else with suricata on PFsence or another packet inspector should monitor the '.tk domains' flags.

[Bob Cadenza]

Does this docker app out of the box allow me to see network traffic on per container basis?

[T_Matz]

netdata is unable to connect to the netdata cloud. is there a setting i have to change in order for it to get access to the internet?

[JonasH]

On 10/15/2022 at 10:33 PM, dopeytree said:

@JorgeB indicated the NGNIX Errors weren't normal. They show up through out the day not just at startup. Anyway I have removed Net Data.

 

Someone else with suricata on PFsence or another packet inspector should monitor the '.tk domains' flags.

How do I remove it completely? Do I need to manually remove some files like with other docker containers in the appdata folder? Do I need to remove something in the /proc folder?

[issabankz]

On 11/2/2022 at 11:26 AM, T_Matz said:

netdata is unable to connect to the netdata cloud. is there a setting i have to change in order for it to get access to the internet?

Hey Mate, did you got it fixed? i tried with creating the new Variables for Claiming but did not work for me.

 

edit:

i SSH into the docker netapp and entred the script given by Netdata Cloud, and now it seems to work with netdata cloud.

Edited December 15, 2022 by issabankz

[CowboyRedBeard]

I keep getting "not available" update status on the docker tab, is this still the right URL?

 

https://hub.docker.com/r/netdata/netdata

 

[primeval_god]

9 hours ago, CowboyRedBeard said:

I keep getting "not available" update status on the docker tab, is this still the right URL?

 

https://hub.docker.com/r/netdata/netdata

 

 

[Xxharry]

I can not see any of my containers on the right side menu. I have the correct PGID input when creating the container. what am I missing?

 

edit: I found out that if I go to the local address of the container http://unraid:19999, I can see the containers but if I go to https://app.netdata.cloud/, I cant see them. how do I fix that?

Edited February 27, 2023 by Xxharry
additional info

[Leoskari]

On 11/2/2022 at 12:26 PM, T_Matz said:

netdata is unable to connect to the netdata cloud. is there a setting i have to change in order for it to get access to the internet?

Did someone fix this or know how to fix it? Same problem here, cant connect to cloud with stock netdata install

[nerbonne]

On 5/11/2019 at 10:23 AM, Caldorian said:

Just installed it on a 6.6.7 install, and it's looking good. However, I'm getting an alert in it about my cache drive, saying that the btrfs allocation is 97.3%. However, if I look at my unraid dashboard, it's only around 60% allocated (157GB out of 256GB).

Any clue why this alert is being raised, and how I may clear it?

 

 

Did you ever get an answer to this?  I'm getting btrfs 100% on 2/4 servers...?

 

[doctortt]

I've been scratching my head for a few days and need some help with Netdata to monitor disk temperature using the smartd / smartmontool. I'm not sure if this is just me or what. By default, my Netdata app doesn't show any disk temperature. I researched on Netdata's website and got hddtemp plugin to work, but the hddtemp project has not been maintained for so long and it just doesn't work that well in Netdata (i.e. hddtemp is picking up no temp for some drives and say they're sleeping when they're not).

 

This is when I found out that smartd / smartmontool is a better plugin because it has disk temperature, SMART data, etc., I researched on the web extensively on how to get this to work in Unraid but no luck. The Unraid OS already has smartd / smartmontool, so I'm not sure if the debug log below indicates that (i) there it cannot find smartmontool or that (ii) it cannot access the logs. 

 

I thought the issue was related to mounting the path to Netdata so that It can see the log path. However, when I go into /var/log/ and /var/lib/ in Unraid's console,  there are no smartd nor smartmontools folders, respectively. 

 

/ # cd /usr/libexec/netdata/plugins.d/
/usr/libexec/netdata/plugins.d # ./python.d.plugin smartd_log debug trace
2023-05-31 01:36:10: python.d INFO: plugin[main] : using python v3
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : looking for '/etc/netdata/python.d.conf'
2023-05-31 01:36:10: python.d INFO: plugin[main] : '/etc/netdata/python.d.conf' was not found
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : looking for '/usr/lib/netdata/conf.d/python.d.conf'
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : '/usr/lib/netdata/conf.d/python.d.conf' is loaded
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : looking for 'pythond-jobs-statuses.json' in /var/lib/netdata
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : loading '/var/lib/netdata/pythond-jobs-statuses.json'
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : '/var/lib/netdata/pythond-jobs-statuses.json' is loaded
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : [smartd_log] loaded module source : '/usr/libexec/netdata/python.d/smartd_log.chart.py'
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : [smartd_log] looking for 'smartd_log.conf' in ['/etc/netdata/python.d', '/usr/lib/netdata/conf.d/python.d']
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : [smartd_log] loading '/usr/lib/netdata/conf.d/python.d/smartd_log.conf'
2023-05-31 01:36:10: python.d DEBUG: plugin[main] : [smartd_log] '/usr/lib/netdata/conf.d/python.d/smartd_log.conf' is loaded
2023-05-31 01:36:10: python.d INFO: plugin[main] : [smartd_log] built 2 job(s) configs
2023-05-31 01:36:10: python.d DEBUG: smartd_log[custom] : scanning /var/log/smartd/
2023-05-31 01:36:10: python.d WARNING: plugin[main] : smartd_log[custom] : unhandled exception on check : FileNotFoundError(2, 'No such file or directory'), skipping the job
2023-05-31 01:36:10: python.d DEBUG: smartd_log[debian] : scanning /var/lib/smartmontools/
2023-05-31 01:36:10: python.d WARNING: plugin[main] : smartd_log[debian] : unhandled exception on check : FileNotFoundError(2, 'No such file or directory'), skipping the job
2023-05-31 01:36:10: python.d INFO: plugin[main] : no jobs to serve
2023-05-31 01:36:10: python.d INFO: plugin[main] : exiting from main...
/usr/libexec/netdata/plugins.d # 

 

drwxr-xr-x 11 root   root      680 May 30 04:40 ./
drwxr-xr-x 14 root   root      340 May 17 15:04 ../
-rw-------  1 root   root        0 May 17 15:02 btmp
-rw-r--r--  1 root   root        0 Apr 28  2021 cron
-rw-r--r--  1 root   root        0 Apr 28  2021 debug
-rw-r--r--  1 root   root    99524 May 29 19:55 dmesg
-rw-r--r--  1 root   root    22905 May 30 02:09 docker.log
-rw-r--r--  1 root   root        0 Feb 13  2021 faillog
-rw-r--r--  1 root   root        0 Apr  7  2000 lastlog
drwxr-xr-x  4 root   root      140 May 29 19:57 libvirt/
-rw-r--r--  1 root   root        0 Apr 28  2021 maillog
-rw-r--r--  1 root   root        0 May 29 19:55 mcelog
-rw-r--r--  1 root   root        0 Apr 28  2021 messages
drwxr-xr-x  2 root   root       40 Aug 10  2022 nfsd/
drwxr-x---  2 nobody root       60 May 29 19:55 nginx/
lrwxrwxrwx  1 root   root       24 May 17 15:01 packages -> ../lib/pkgtools/packages/
drwxr-xr-x  5 root   root      100 May 29 19:55 pkgtools/
drwxr-xr-x  2 root   root      300 May 29 19:55 plugins/
drwxr-xr-x  2 root   root       40 May 30 21:20 pwfail/
lrwxrwxrwx  1 root   root       25 May 17 15:03 removed_packages -> pkgtools/removed_packages/
lrwxrwxrwx  1 root   root       24 May 17 15:03 removed_scripts -> pkgtools/removed_scripts/
lrwxrwxrwx  1 root   root       34 May 29 19:55 removed_uninstall_scripts -> pkgtools/removed_uninstall_scripts/
drwxr-xr-x  2 root   root       40 Jan 30 13:03 sa/
drwxr-xr-x  3 root   root      320 May 29 19:55 samba/
lrwxrwxrwx  1 root   root       23 May 17 15:01 scripts -> ../lib/pkgtools/scripts/
-rw-r--r--  1 root   root        0 Apr 28  2021 secure
lrwxrwxrwx  1 root   root       21 May 17 15:01 setup -> ../lib/pkgtools/setup/
-rw-r--r--  1 root   root        0 Apr 28  2021 spooler
drwxr-xr-x  3 root   root       60 Sep 27  2022 swtpm/
-rw-r--r--  1 root   root  1063374 May 30 21:38 syslog
-rw-r--r--  1 root   root 14145267 May 30 04:38 syslog.1
-rw-r--r--  1 root   root        0 May 29 19:54 vfio-pci
-rw-r--r--  1 root   root        0 May 29 19:54 vfio-pci-errors
-rw-rw-r--  1 root   utmp     6912 May 29 19:55 wtmp
root@powernas:/var/log# 

 

 

drwxr-xr-x 15 root       root   320 May 30 04:40 ./
drwxr-xr-x 14 root       root   340 May 17 15:04 ../
drwxr-xr-x  2 root       root    40 Feb 20 17:27 arpd/
drwxr-xr-x  2 messagebus root    60 May 29 19:55 dbus/
drwxr-x---  2 root       root    40 Aug 28  2022 dhcpcd/
drwx--x---  1 root       root   240 May 29 19:57 docker/
drwxr-xr-x 10 root       root   220 May 17 15:03 libvirt/
-rw-r-----  1 root       root   699 May 30 04:40 logrotate.status
drwxr-xr-x  6 rpc        rpc    220 May 29 19:57 nfs/
drwx------  7 root       root   140 May 29 19:55 nginx/
drwxrwxr-x  2 ntp        ntp     60 May 30 19:55 ntp/
drwxrwx---  2 root       apache 160 May 30 21:50 php/
drwxr-xr-x  6 root       root   160 May 17 15:03 pkgtools/
drwxr-xr-x  5 root       root   220 May 29 19:57 samba/
drwxr-xr-x  2 root       root    40 May  1  2022 swtpm-localca/
drwxr-xr-x  2 root       root    60 Feb  7 15:09 xkb/
root@powernas:/var/lib# 

 

[doctortt]

1 hour ago, doctortt said:

I've been scratching my head for a few days and need some help with Netdata to monitor disk temperature using the smartd / smartmontool. I'm not sure if this is just me or what. By default, my Netdata app doesn't show any disk temperature. I researched on Netdata's website and got hddtemp plugin to work, but the hddtemp project has not been maintained for so long and it just doesn't work that well in Netdata (i.e. hddtemp is picking up no temp for some drives and say they're sleeping when they're not).

 

This is when I found out that smartd / smartmontool is a better plugin because it has disk temperature, SMART data, etc., I researched on the web extensively on how to get this to work in Unraid but no luck. The Unraid OS already has smartd / smartmontool, so I'm not sure if the debug log below indicates that (i) there it cannot find smartmontool or that (ii) it cannot access the logs. 

 

I thought the issue was related to mounting the path to Netdata so that It can see the log path. However, when I go into /var/log/ and /var/lib/ in Unraid's console,  there are no smartd nor smartmontools folders, respectively. 

 

 

 

 

 

 

After doing further research, i think my issue is that I need to add the line below in /etc/default/smartmontools. However, as shown below, there is no smartmontools file in that path. 

smartd_opts="-A /var/log/smartd/ -i 600

 

Is it 

/etc/smartd.conf 

I should be adding that line to?

 

 

 

Edited May 31, 2023 by doctortt

[TexasUnraid]

10 hours ago, doctortt said:

 

After doing further research, i think my issue is that I need to add the line below in /etc/default/smartmontools. However, as shown below, there is no smartmontools file in that path. 

smartd_opts="-A /var/log/smartd/ -i 600

 

Is it 

/etc/smartd.conf 

I should be adding that line to?

 

 

 

I have no idea but I am interested in what you find out. I used hddtemp for awhile and it works but I found it broke drive spinddown. I would like to get drive temps back into netdata.

The better place to ask about this type of stuff is the github, they have been real good when I have had questions over there.

[frodr]

I'm struggling with the basics here. How to get the node connected to the Could Account? Trying to "Sign in" I get a mail with a link to the Cloud Account, but how to link the node?

 

Anyone?

 

//