April 26, 20242 yr Living in a rural area I have been struggling with internet connections for years, and now that I have Fiber being installed (in a year maybe?) I have found the tool I always wished I had: NetProbe Sadly there is no listing in Community Applications... I managed to install it from the Git repository (https://github.com/plaintextpackets/netprobe_lite) using Compose.Manager and make it work, but boy does it make a mess of my Docker Tab. I'm just posting ot see if there is enough interest, someone might decide to make a more functional Docker out of it. Further, am I wrong in thinging it would be better to have one install of Grafana rather than multiple? Same with MariaDB? Thanks Arbadacarba
April 28, 20242 yr ? so your wanting a all in one encompassing docker with graffan / mysql for this service? or to introudce this to the community app store? see: https://forums.unraid.net/topic/151967-publishing-to-community-apps/ https://forums.unraid.net/topic/101424-how-to-publish-docker-templates-to-community-applications-on-unraid/ Closest I can find of a premade Docker pull is https://hub.docker.com/r/aojea/docker-netprobe/tags But no template though... Looks interesting. Though Edited April 28, 20242 yr by bmartino1 Forum crazy ness with post linkes and deleting words
May 17, 20242 yr With some updates on how this projects uses its data. if i were to remove the docker or update the docker I lose all the stats colected. To fix and use with unraids to use the docker compose plugin we have to make a simple change and not use user script. add a new stack, hit advance and set the git download pat to the docker: This will create a docekr-compose.yml file in that path we want to rename the compose.yml file as a backup Due to the project layout. we want to keep the netproe data in/from Grafana. That data should persist between docker stops, image deletes, upgrades and Grafana changes. and use the web UI to add the compose file: to do this, I made some adjustments to the compose file: version: "3" networks: custom_network: driver: bridge ipam: config: - subnet: 172.21.0.0/16 services: redis: restart: always container_name: netprobe-redis image: redis:latest labels: - net.unraid.docker.icon='https://raw.githubusercontent.com/A75G/docker-templates/master/templates/icons/redis.png' volumes: - ./config/redis/redis.conf:/etc/redis/redis.conf networks: - custom_network logging: options: max-size: 50m max-file: 1 netprobe: restart: always container_name: netprobe-probe build: . volumes: - .:/netprobe_lite environment: MODULE: NETPROBE labels: - net.unraid.docker.icon='https://raw.githubusercontent.com/simonjenny/fastcom-mysql/master/fastlogo.jpg' networks: - custom_network logging: options: max-size: 50m max-file: 1 presentation: restart: always container_name: netprobe-presentation build: . volumes: - .:/netprobe_lite environment: MODULE: PRESENTATION labels: - net.unraid.docker.icon='https://github.com/chvb/docker-templates/raw/master/chvb/img/OnlyOfficeDocumentServer.jpg' networks: - custom_network logging: options: max-size: 50m max-file: 1 prometheus: restart: always container_name: netprobe-prometheus image: prom/prometheus labels: - net.unraid.docker.icon='https://raw.githubusercontent.com/selfhosters/unRAID-CA-templates/master/templates/img/prometheus.png' volumes: - ./config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml networks: - custom_network logging: options: max-size: 50m max-file: 1 grafana: restart: always image: grafana/grafana-enterprise:latest labels: - net.unraid.docker.webui=http://[IP]:[PORT:3001] - net.unraid.docker.icon='https://github.com/atribe/unRAID-docker/raw/master/icons/grafana.png' container_name: netprobe-grafana volumes: - ./config/grafana/datasources/automatic.yml:/etc/grafana/provisioning/datasources/automatic.yml - ./config/grafana/dashboards/main.yml:/etc/grafana/provisioning/dashboards/main.yml - ./config/grafana/data/:/var/lib/grafana/ ports: - 3001:3000 networks: - custom_network logging: options: max-size: 50m max-file: 1 environment: - GF_SECURITY_ADMIN_PASSWORD=admin ^ with constant fights with how logging and data that is taken up within the docker image. I've added temp login to help reduce and lower the data. we care about the other database but don't need them as grafan is reported and keeps the log for display. Their is more that need to be done to grafan to maintain and keep accounts. ATM i've decided to keep it admin admin and added Grafana environment stuff to fix. For grafan review: https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/ I've also cleaned it up to use Unraids webui and icons across the community apps Please not the volume for grafana as this has changed. To keep its data, we need to make a change! ^this is more to grab the .db file to save on disk as this is where the dashboard is keeping that data. Chagned: Quote - ./config/grafana/dashboards/netprobe.json:/var/lib/grafana/dashboards/netprobe.json to - ./config/grafana/data/:/var/lib/grafana/ we will need to create and move/copy the data to fix this: There is a quite a few things we need to do to recreate and fix. #echo Path to your git download. cd /mnt/user/appdata/netprobe_lite cd ./config/grafana #echo make save location to keep grafana data mkdir data cd data #echo copy the missing data that we chagned from compsoe: cp /mnt/user/appdata/netprobe_lite/config/grafana/dashboards /mnt/user/appdata/netprobe_lite/config/grafana/data/ #echo Recreate the file paths: mkdir csv mkdir pdf mkdir plugins mkdir png #echo fixing grafa file paths: mkdir alerting cd alerting mkdir 1 cd 1 touch __default__.tmpl cd .. cd .. touch grafana.db #echo seting corect docker permissions: chmod 777 -R * chown nobody:users -R * ^added as zipped contents here We may need to delete previous image data in docekr click advance togle: Finally, with the folder data in we need to run update stack to rebuild and update any dockers... now we have a docker tab web ui controlled netprobe system with icons, web ui and data saved between docker updates and changes *Not sure the full need of the multiple database in these configurations. not sure if redis is being used and have not used promethus db and/or docekr redis db... docker before some research is need before we can re-script this to use MySQL... -But with this setup I can keep my records in Grafana DB and happy where its at. data.zip Edited May 17, 20242 yr by bmartino1
May 29, 20242 yr Author I don't know why but I havenet been getting alerts that you have been working in here... This is excellent, I'm in the process of working through what you have done here and installing it.
May 29, 20242 yr Solution On 5/29/2024 at 12:49 PM, Arbadacarba said: I don't know why but I havenet been getting alerts that you have been working in here... This is excellent, I'm in the process of working through what you have done here and installing it. Welcome back. I will try to rewrite the installation procedures to reproduce. There's a lot and i used this as notes/sound board. Assumptions: I assume you already have a data spot in mind and the compose plugin installed. I chose /mnt/user/appdata Step 1 open terminal and go to the installation path of your choosing. I chose to keep this together with my docker data. Unraid terminal script setup: run one line at a time (./config is the config folder where your dat path is) #It is best to use full paths. ./config/ in my case is /mnt/user/appdata/netprobe_lite/config #Download GitHub project and files/scripts cd /mnt/user/appdata git clone https://github.com/plaintextpackets/netprobe_lite.git #enter project to make changes cd netprobe_lite/ #step 1 prepare for unraid UI rename existing compose file cp compose.yml compose.yml.org rm compose.yml #step 2 prepare for better docker Data path to keep between docker updates. #First Grafana: cd ./config/grafana #echo make save location to keep Grafana data mkdir data cd data #echo copy the dashboard to correct spot -- missing data that we changed in compose(I use MC to copy the folder): cp -R ./config/grafana/dashboards ./config/grafana/data/ #echo Recreate the empty folder paths Grafana needed for latter use inside the “./config/grafana/data/” (issue in past with docker not making folder needed Docker should make this at first run): cd data mkdir csv mkdir pdf mkdir plugins mkdir png #echo fixing grafana files and paths(grafana entprise config) docker should make these at first run this should be in “./config/grafana/data/”: mkdir alerting cd alerting mkdir 1 cd 1 touch __default__.tmpl cd .. cd .. touch grafana.db #Next Promethus - The actual data that needs saved to keep between dockers updates and deletion! “./config/prometheus" makin its "/data/” location #Drop back to config folder “./config/”: cd .. cd .. cd prometheus/ mkdir data #Fix permission – set unraid docker safe perms….: #Drop back to config folder: cd .. #setting unraid docker safe permissions to config folder only!: #echo setting correct docker permissions “./config/”: chmod 777 -R * chown nobody:users -R * Step 2 setup docker compose: our data exist: at /mnt/user/appdata/netprobe_lite/ This will make the compose file we use in unraid and import the .env file used to configure and control netprobe. Next we want to edit the stack: Step 3 env file first: Github default env file: https://github.com/plaintextpackets/netprobe_lite/blob/master/.env we want to make a few changes: DOUBLE CHECK YOUR .ENV FILE FOR IP and EDITS YOU WANT. If you start this docker and then edit this Grafana may display weird by picking up the data in the Prometheus database. You will need to delete the contents of the “./config/prometheus/data this will erase all collected data and use the env file config. Quote # CUSTOM VARIABLES # Notes: # 1. Do not change any variable names # 2. Read instructions carefully # Site target list # - If modifying this list, make sure you limit to 5 websites and use the domain name as shown here #SITES="google.com,facebook.com,twitter.com,youtube.com,amazon.com" SITES="fast.com,google.com,youtube.com,amazon.com,uillinois.edu" # DNS test site # - This is the site which is resolved in DNS to test DNS servers, if modified only use one domain name DNS_TEST_SITE="uillinois.edu" # This is the site used in the DNS test # DNS name servers # - This is the list of DNS servers which are tested # - Netprobe only supports 4 DNS servers, you can change the value of "DNS_NAMESERVER_4_IP" to test your own home DNS server # - Note: do not change the value of "DNS_NAMESERVER_4" DNS_NAMESERVER_1="Google_DNS" DNS_NAMESERVER_1_IP="8.8.8.8" DNS_NAMESERVER_2="Quad9_DNS" DNS_NAMESERVER_2_IP="9.9.9.9" DNS_NAMESERVER_3="CloudFlare_DNS" DNS_NAMESERVER_3_IP="1.1.1.1" DNS_NAMESERVER_4="My_DNS_Server" # Do not change this line at all! DNS_NAMESERVER_4_IP="192.168.2.10" # Replace this IP with the DNS server you use at home # Health Score Weights # - These are the relative weights used to calculate your 'Internet Quality Score', they can be modified but must add up to 1.0 weight_loss = ".6" # Loss is 60% of score weight_latency = ".15" # Latency is 15% of score weight_jitter = ".2" # Jitter is 20% of score weight_dns_latency = "0.05" # DNS latency is 0.05 of score # Health Score Thresholds # - These threshold values are used in the calculation of your 'Internet Quality Score', they can be modified if required threshold_loss = "5" # 5% loss threshold as max threshold_latency = "100" # 100ms latency threshold as max threshold_jitter = "30" # 30ms jitter threshold as max threshold_dns_latency = "100" # 100ms dns latency threshold as max # Speetest configuration (be very careful when running on a metered connection!) # - This configuration is for setting up a "speed test" or rather a test of your internet bandwidth. # - In order to test your upload and download bandwidth we use speedtest.net as source. So your client will connect there and upload and download some data. # - That also means that a random server is selected for the test (usually the nearest one) # - Setting the SPEEDTEST_INTERVAL too agressively will cause speedtest.net to block your requests, recommend 15 minutes (900 seconds) and above SPEEDTEST_ENABLED="True" # set this to "True" to enable speed test function SPEEDTEST_INTERVAL="3600" # interval on which the speedtest will run, in seconds - note using a prime number helps reduce the number of collisions between netprobe and speed tests # SYSTEM VARIABLES - DO NOT TOUCH PRESENTATION_PORT = "5000" PRESENTATION_INTERFACE = "0.0.0.0" REDIS_URL = "netprobe-redis" REDIS_PORT = "6379" REDIS_PASSWORD = "password" PROBE_INTERVAL="30" PROBE_COUNT="50" ^Here we enabled speedtest it runs every hour. I then set a custom dns IP(I run Pihole) if you don't run a in house dns server use opendns 208.67.222.222 it will be labled as My_DNS_Server *^ Do NOT! Change the name the grafana main and other dashboard file call it weirdly to display correctly. I run piehole docker, a custom DNS, and it currently set to that IP. Step 4: Then edit the docker compose file with the following edits: Quote version: "3" networks: custom_network: driver: bridge ipam: config: - subnet: 172.21.0.0/16 services: redis: pull_policy: always restart: always container_name: netprobe-redis image: redis:latest labels: - net.unraid.docker.icon='https://raw.githubusercontent.com/A75G/docker-templates/master/templates/icons/redis.png' volumes: - ./config/redis/redis.conf:/etc/redis/redis.conf networks: - custom_network logging: options: max-size: 50m max-file: 1 netprobe: pull_policy: always restart: always container_name: netprobe-probe build: . volumes: - .:/netprobe_lite environment: MODULE: NETPROBE labels: - net.unraid.docker.icon='https://raw.githubusercontent.com/simonjenny/fastcom-mysql/master/fastlogo.jpg' networks: - custom_network logging: options: max-size: 50m max-file: 1 presentation: pull_policy: always restart: always container_name: netprobe-presentation build: . volumes: - .:/netprobe_lite environment: MODULE: PRESENTATION labels: - net.unraid.docker.icon='https://github.com/chvb/docker-templates/raw/master/chvb/img/OnlyOfficeDocumentServer.jpg' networks: - custom_network logging: options: max-size: 50m max-file: 1 prometheus: pull_policy: always restart: always container_name: netprobe-prometheus image: prom/prometheus labels: - net.unraid.docker.icon='https://raw.githubusercontent.com/selfhosters/unRAID-CA-templates/master/templates/img/prometheus.png' volumes: - ./config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml - ./config/prometheus/data/:/prometheus/ networks: - custom_network logging: options: max-size: 50m max-file: 1 grafana: pull_policy: always restart: always image: grafana/grafana-enterprise:latest labels: - net.unraid.docker.webui=http://[IP]:[PORT:3001] - net.unraid.docker.icon='https://github.com/atribe/unRAID-docker/raw/master/icons/grafana.png' container_name: netprobe-grafana volumes: - ./config/grafana/datasources/automatic.yml:/etc/grafana/provisioning/datasources/automatic.yml - ./config/grafana/dashboards/main.yml:/etc/grafana/provisioning/dashboards/main.yml - ./config/grafana/data/:/var/lib/grafana/ ports: - 3001:3000 networks: - custom_network logging: options: max-size: 50m max-file: 1 environment: - GF_SECURITY_ADMIN_USER=admin - GF_SECURITY_ADMIN_PASSWORD=admin speedtest: restart: always container_name: netprobe-speedtest image: "plaintextpackets/netprobe:latest" pull_policy: always labels: - net.unraid.docker.icon='https://raw.githubusercontent.com/selfhosters/unRAID-CA-templates/master/templates/img/speedtest-tracker.png' volumes: - .:/netprobe_lite environment: MODULE: "SPEEDTEST" networks: - custom_network logging: options: max-size: 50m max-file: 1 ^Here is my working unraid Compose file that keeps data. Note the Compose items for lables, theses http links and data go in the UI lables when prompted. This adds the docker image to the docker tab. You can set an admin account at this time by editing the grafana section option of ... environment: - GF_SECURITY_ADMIN_USER=admin - GF_SECURITY_ADMIN_PASSWORD=admin Default admin/admin Then set the UI labels (compose no longer fills in the data.) ^In the UI lables copy from the compose file Edited June 23, 20242 yr by bmartino1 finalizing data. script at top is guidelines
May 29, 20242 yr Author Data location: mnt/systems/system/appdata That's where all my dockers are, its a separate pool on an SSD Array in step 2 after: #step 2 prepare for better docker Data path to keep between docker updates. #First Grafana: cd ./config/grafana #echo make save location to keep grafana data mkdir data cd data #echo copy the dashboard to correct spot -- missing data that we chagned in compsoe: cp ./config/grafana/dashboards ./config/grafana/data/ i get: cp: cannot stat './config/grafana/dashboards': No such file or directory
May 29, 20242 yr 3 minutes ago, Arbadacarba said: Data location: mnt/systems/system/appdata That's where all my dockers are, its a separate pool on an SSD Array in step 2 after: #step 2 prepare for better docker Data path to keep between docker updates. #First Grafana: cd ./config/grafana #echo make save location to keep grafana data mkdir data cd data #echo copy the dashboard to correct spot -- missing data that we chagned in compsoe: cp ./config/grafana/dashboards ./config/grafana/data/ i get: cp: cannot stat './config/grafana/dashboards': No such file or directory you want to copy the dashbaord folder into the data folder: Edited May 29, 20242 yr by bmartino1
May 29, 20242 yr Author cp -R /mnt/systems/system/appdata/netprobe_lite/config/grafana/dashboards /mnt/systems/system/appdata/netprobe_lite/config/grafana/data/ Worked
May 29, 20242 yr Author #echo Recreate the empty folder paths nedded for latter(issue in past with docker not making folder needed): cd data mkdir csv mkdir pdf mkdir plugins mkdir png Where am I starting here? I think I am already in the data folder
May 29, 20242 yr 1 minute ago, Arbadacarba said: #echo Recreate the empty folder paths nedded for latter(issue in past with docker not making folder needed): cd data mkdir csv mkdir pdf mkdir plugins mkdir png Where am I starting here? I think I am already in the data folder correct this needs to be in the data folder these are the emapy files path that grafan should created.
May 29, 20242 yr Author You mention changes that needed to be made to the env and compose files... But I don't see them. Did you just mean the ones pasted in to previous messages? Thanks so much for your work here.
May 29, 20242 yr 38 minutes ago, Arbadacarba said: You mention changes that needed to be made to the env and compose files... But I don't see them. Did you just mean the ones pasted in to previous messages? Thanks so much for your work here. I quoted them again they should be the ones in the post marked as solution. There was a bit that I changed from the original env. Set different site I would normally go to for ping. Quote #SITES="google.com,facebook.com,twitter.com,youtube.com,amazon.com" SITES="fast.com,google.com,youtube.com,amazon.com,uillinois.edu" Set a university in my area for a dns name check Quote # - This is the site which is resolved in DNS to test DNS servers, if modified only use one domain name DNS_TEST_SITE="uillinois.edu" # This is the site used in the DNS test Set a custom dns in house server to compare to in house v public. (pihole uses fastest in return) Quote DNS_NAMESERVER_4_IP="192.168.2.10" # Replace this IP with the DNS server you use at home Set speed test to enable: Quote # - Setting the SPEEDTEST_INTERVAL too agressively will cause speedtest.net to block your requests, recommend 15 minutes (900 seconds) and above SPEEDTEST_ENABLED="True" # set this to "True" to enable speed test function SPEEDTEST_INTERVAL="3600" # interval on which the speedtest will run, in seconds - note using a prime number helps reduce the number of collisions between netprobe and speed tests Then Set speed test to run 1 every hour. Docker compose was quite a bit from the original. Added labeling for unraid icons and web ui added logging: more to keep logs small form other dockers *-When I first downloaded and used their compose file to make my own, there was no dns option -Fixed docker network for unraid creation with compose up from stack. Changed the dashboard to inc operate Grafana environment variables and save the DB set docker execution policy to download and update the docker each time they are started. -added admin account creation to docker compose. (when Grafana is taken down this reset accounts) another variable can be added to move the conf file to physical data disk and used similar to the data folder... Most of Grafana needed variables from official dock: https://grafana.com/docs/grafana/latest/setup-grafana/installation/docker/ For this to be as a CA it would need to be used as 1 docker. it may be a Linux image that runs docker in docker. Then trying to come up with multi path between the 2 and that wasn't worth the fight. Edited May 29, 20242 yr by bmartino1 Spelling
June 2, 20242 yr Hi bmartino1, Thank you so much for your detailed posts. I have been trying to work through this but couldn't get things to work properly. But have been through again and your post marked as the solution got me a lot further. Thank you for your initial posts and coming back with more info too I have now successfully been able to get all the apps to start in docker, and been able to log into Grafana. However I am missing the dashboard with all the info that I was expecting. Clearly I have missed something. Or is there some config I need to do once I log into grafana? watching the YouTube video - the dashboard is just there... I did also got stuck on the ./config thing but managed to work it out (thought later I realised I should have kept reading lol) I just used your post with the env file details and docker compose files. Only changing the last dns setting. Short of deleting it all and starting again, did you have any ideas? Thanks heaps for your work on this! Been a project I've been wanting to get on unraid for some time now! Kept checking the CA but i now see its too complex to be there.
June 2, 20242 yr 3 hours ago, hbilling said: Hi bmartino1, Thank you so much for your detailed posts. I have been trying to work through this but couldn't get things to work properly. But have been through again and your post marked as the solution got me a lot further. Thank you for your initial posts and coming back with more info too I have now successfully been able to get all the apps to start in docker, and been able to log into Grafana. However I am missing the dashboard with all the info that I was expecting. Clearly I have missed something. Or is there some config I need to do once I log into grafana? watching the YouTube video - the dashboard is just there... I did also got stuck on the ./config thing but managed to work it out (thought later I realised I should have kept reading lol) I just used your post with the env file details and docker compose files. Only changing the last dns setting. Short of deleting it all and starting again, did you have any ideas? Thanks heaps for your work on this! Been a project I've been wanting to get on unraid for some time now! Kept checking the CA but i now see its too complex to be there. I moved the dashboard to make grafana keep its data. you need to make the data folder and copy the dashbaord fodler into the data folder. run theses camoands 1 at atime folowing the above notes: #Download github project and files/scripts cd /mnt/user/appdata git clone https://github.com/plaintextpackets/netprobe_lite.git #enter project to make changes cd netprobe_lite/ #step 1 prepare for unraid UI rename existing compse file cp compose.yml compose.yml.org rm compose.yml #step 2 prepare for better docker Data path to keep between docker updates. #First Grafana: cd ./config/grafana #echo make save location to keep grafana data mkdir data cd data #echo copy the dashboard to correct spot -- missing data that we chagned in compsoe: cp -R ./config/grafana/dashboards ./config/grafana/data/ #^copy the netprgbe grafan dashboard into the data folder ^ #echo Recreate the empty folder paths nedded for latter(issue in past with docker not making folder needed): cd data mkdir csv mkdir pdf mkdir plugins mkdir png #echo fixing grafana file paths(grafana entprise config): mkdir alerting cd alerting mkdir 1 cd 1 touch __default__.tmpl cd .. cd .. touch grafana.db #Next Promethus - The actualy data that needs saved! #Drop back to config folder: cd .. cd .. cd prometheus/ mkdir data #Fix permission: #Drop back to config folder: cd .. #setting unraid docker safe permissions to config folder only!: #echo seting corect docker permissions: chmod 777 -R * chown nobody:users -R * seen more here: copy the netprbe grafan dash board folder (2 files) into graffan data dashboard folder. Edited June 2, 20242 yr by bmartino1
June 2, 20242 yr Thanks for the quick reply, tried again and all up this time. I clearly didn't get a step right copying the dashboard directory. Cheers! 😁
November 29, 2025Nov 29 I've decided to rebuild somthign for a project... Since gfrafan has moved to a different direction and with bitanmi company back ends chagnes(no redis)...esental this needs 3 components...1 grafan to display the data.2 a database (I wil use postgress) to hold the dat) I don't like promethus....3 a docker to run the tests and colet and store the data-the comands and script that run to colect the dns response times the latecny etc...https://github.com/bmartino1/NetProbe_PythonWIP... Bare Bones ATM lots to rebuild and think.. Similar to other muti docker stacks I may add to CA latter...with a all in one docker done I will now add this to the CA... Edited December 1, 2025Dec 1 by bmartino1
December 1, 2025Dec 1 docker image now exist as a all in one systems...Python build web server to probe... why run grafana, prmethus, redis... when we can have 1 docker do this???https://hub.docker.com/r/bmmbmm01/netprobe@Abracadadra sorry this was on the back burner for a while as a friend of mine needed it. with bitnami redis edits and ohter going on I decided to refactor there project and repurpse it all under 1 python based docker... unraid CA tempalte latter. I have testd this iwht docker copose on unrad and it works well
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.