[support] Spants - NodeRed, MQTT, Dashing, couchDB


Recommended Posts

52 minutes ago, spants said:

Which file is that size? Can you post your mosquitto.conf file?

I was logging the default which was everything, I believe..  I've recently changed it to just errors and warnings.  But I can't figure out how to purge it..  Or, potentially, how to use log rotate to keep it manageable.   How do other folks handle the mqtt logging?  Do they log everything but put it in appdata area?

 

What are best practices?

 

Jim

Link to comment

The nodered docker keeps crashing on me.  If I restart it, it runs for a day or so.  I have deleted the container image and reinstalled and the outcome is the same.  Here is the log....any suggestions?

 

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node',
1 verbose cli   '/usr/local/bin/npm',
1 verbose cli   'start',
1 verbose cli   '--cache',
1 verbose cli   '/data/.npm',
1 verbose cli   '--',
1 verbose cli   '--userDir',
1 verbose cli   '/data' ]
2 info using [email protected]
3 info using [email protected]
4 verbose config Skipping project config: /usr/src/node-red/.npmrc. (matches userconfig)
5 verbose run-script [ 'prestart', 'start', 'poststart' ]
6 info lifecycle [email protected]~prestart: [email protected]
7 info lifecycle [email protected]~start: [email protected]
8 verbose lifecycle [email protected]~start: unsafe-perm in lifecycle true
9 verbose lifecycle [email protected]~start: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/usr/src/node-red/node_modules/.bin:/usr/lo>
10 verbose lifecycle [email protected]~start: CWD: /usr/src/node-red
11 silly lifecycle [email protected]~start: Args: [ '-c',
11 silly lifecycle   'node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"' ]
12 silly lifecycle [email protected]~start: Returned: code: 1  signal: null
13 info lifecycle [email protected]~start: Failed to exec start script
14 verbose stack Error: [email protected] start: `node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"`
14 verbose stack Exit status 1
14 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
14 verbose stack     at EventEmitter.emit (events.js:198:13)
14 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
14 verbose stack     at ChildProcess.emit (events.js:198:13)
14 verbose stack     at maybeClose (internal/child_process.js:982:16)
14 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
15 verbose pkgid [email protected]
16 verbose cwd /usr/src/node-red
17 verbose Linux 4.19.107-Unraid
18 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "start" "--cache" "/data/.npm" "--" "--userDir" "/data"
19 verbose node v10.22.1
20 verbose npm  v6.14.6
21 error code ELIFECYCLE
22 error errno 1
23 error [email protected] start: `node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"`
23 error Exit status 1
24 error Failed at the [email protected] start script.
24 error This is probably not a problem with npm. There is likely additional logging output above.
25 verbose exit [ 1, true ]

 

Link to comment
On 11/5/2020 at 12:29 AM, goinsnoopin said:

The nodered docker keeps crashing on me.  If I restart it, it runs for a day or so.  I have deleted the container image and reinstalled and the outcome is the same.  Here is the log....any suggestions?

 


0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node',
1 verbose cli   '/usr/local/bin/npm',
1 verbose cli   'start',
1 verbose cli   '--cache',
1 verbose cli   '/data/.npm',
1 verbose cli   '--',
1 verbose cli   '--userDir',
1 verbose cli   '/data' ]
2 info using [email protected]
3 info using [email protected]
4 verbose config Skipping project config: /usr/src/node-red/.npmrc. (matches userconfig)
5 verbose run-script [ 'prestart', 'start', 'poststart' ]
6 info lifecycle [email protected]~prestart: [email protected]
7 info lifecycle [email protected]~start: [email protected]
8 verbose lifecycle [email protected]~start: unsafe-perm in lifecycle true
9 verbose lifecycle [email protected]~start: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/usr/src/node-red/node_modules/.bin:/usr/lo>
10 verbose lifecycle [email protected]~start: CWD: /usr/src/node-red
11 silly lifecycle [email protected]~start: Args: [ '-c',
11 silly lifecycle   'node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"' ]
12 silly lifecycle [email protected]~start: Returned: code: 1  signal: null
13 info lifecycle [email protected]~start: Failed to exec start script
14 verbose stack Error: [email protected] start: `node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"`
14 verbose stack Exit status 1
14 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
14 verbose stack     at EventEmitter.emit (events.js:198:13)
14 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
14 verbose stack     at ChildProcess.emit (events.js:198:13)
14 verbose stack     at maybeClose (internal/child_process.js:982:16)
14 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
15 verbose pkgid [email protected]
16 verbose cwd /usr/src/node-red
17 verbose Linux 4.19.107-Unraid
18 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "start" "--cache" "/data/.npm" "--" "--userDir" "/data"
19 verbose node v10.22.1
20 verbose npm  v6.14.6
21 error code ELIFECYCLE
22 error errno 1
23 error [email protected] start: `node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"`
23 error Exit status 1
24 error Failed at the [email protected] start script.
24 error This is probably not a problem with npm. There is likely additional logging output above.
25 verbose exit [ 1, true ]

 

ok, you can start NodeRed in SAFE mode by enabling an option.. Please edit the settings to add the option below and set the option to true. They may be a faulty node loaded?. There are also cli commands that will check to see if nodes are outdated and repair them.

nodered-unraid.png

Link to comment

As of today, I am having the exact same issue.

 

It started w/ trying to add a module, and getting error's that the ownership of /data/.npm needed to be updated to 1000.  It said, "your cache has files that are owned by root".

 

After going through all of that, it worked fine, but then i got the itch to just make sure it was fresh, deleted and installed from template.

 

Now it will not start, crashes immediately with the exact same log file as above... 

 

2020-11-10T17_12_01_430Z-debug.log

 

I tried the safe mode you mentioned above, and it still will not start, is just says starting.  No web UI, and no console.

Edited by wdenny
Link to comment
1 hour ago, wdenny said:

As of today, I am having the exact same issue.

 

Did you guys update to the new repository? The repo changed a while back and I noticed when I tried to upgrade the image off the old repo it failed with a similar error (it was actually launching and running node-red for me but all NPM commands failed so I was in a broken state so none of my flows were running).

 

I'm running with this currently:

nodered/node-red:latest-12

 

Also if you started over by re-installing did you nuke the previous folder or choose a new location in appdata to install to? If you are using the same folder as before that's most likely why things are broke. Whenever you start over it's a good idea to either nuke your appdata folder or choose a new one to install to.

 

On 10/30/2020 at 7:22 AM, jbuszkie said:

Another question..  My MQTT log file is > 4GB!  how do I get rid of it?

image.png.c8c62cb4eb7a96819d4a2435860a3d01.png

Run this command to see what log files are using space on your system (If you install the User Scripts plugin this is one of their scripts):

du -ah /var/lib/docker/containers/ | grep -v "/$" | sort -rh | head -60 | grep .log

 

Now run this command to truncate the log files that are too big:

echo "" > <log file path from previous command>

Example:

echo "" > /var/lib/docker/containers/d6f6b5093722d4f93589e3f4e9211b9720b28568f924b4fe4edd9634cfb728df/d6f6b5093722d4f93589e3f4e9211b9720b28568f924b4fe4edd9634cfb728df-json.log

 

Now run the du command again to list out log file sizes and it should be empty.

 

On 10/31/2020 at 8:18 AM, jbuszkie said:

I was logging the default which was everything, I believe..  I've recently changed it to just errors and warnings.  But I can't figure out how to purge it..  Or, potentially, how to use log rotate to keep it manageable.   How do other folks handle the mqtt logging?  Do they log everything but put it in appdata area?

 

What are best practices?

 

Jim

There aren't really "Best Practices" as it all depends on what you need logged and your use case. I'm using MQTT for my home automation so I really only care about logging what devices connect and error messages. This is what I have my log options in mosquitto.conf set to:

log_dest stderr
log_type error
connection_messages true
log_timestamp true

If you really want to add log rotating it is an option you can pass to the container. Here is a post that describes it very well:
https://medium.com/@Quigley_Ja/rotating-docker-logs-keeping-your-overlay-folder-small-40cfa2155412

 

 

  • Thanks 2
Link to comment

So I went in and deleted the modules, one at a time backwards from when they were added.

 

Removing these two modules allowed it to start and run as is did before:

node-red-contrib-amazon-echo

node-red-contrib-hue

 

As for the current repo, i am a bit confused.  I am using the official node red docker from CA...  but, it seems to be pulling from the depreciated repo?

image.thumb.png.22dfe9be8748bc4721e9927c01ed315d.png

 

When i check that out, it looks like it was depreciated:

image.png.ddfe44164280e49ed0088eb8f20b8453.png

 

Should it be pulling from: 

https://hub.docker.com/r/nodered/node-red

 

Link to comment
1 hour ago, Skylord123 said:

 

There aren't really "Best Practices" as it all depends on what you need logged and your use case. I'm using MQTT for my home automation so I really only care about logging what devices connect and error messages. This is what I have my log options in mosquitto.conf set to:


log_dest stderr
log_type error
connection_messages true
log_timestamp true

If you really want to add log rotating it is an option you can pass to the container. Here is a post that describes it very well:
https://medium.com/@Quigley_Ja/rotating-docker-logs-keeping-your-overlay-folder-small-40cfa2155412

 

 

Thanks for the info.  so it was really just as easy as purging the log!  I wanted to make sure.  Thanks!  I'll have to look into the log rotate if I want to do that.

Link to comment
59 minutes ago, wdenny said:

So I went in and deleted the modules, one at a time backwards from when they were added.

 

Removing these two modules allowed it to start and run as is did before:

node-red-contrib-amazon-echo

node-red-contrib-hue

 

As for the current repo, i am a bit confused.  I am using the official node red docker from CA...  but, it seems to be pulling from the depreciated repo?

image.thumb.png.22dfe9be8748bc4721e9927c01ed315d.png

 

Should it be pulling from: 

https://hub.docker.com/r/nodered/node-red

 

Looks like your repository actually is updated. The old one had -docker on the end of it (so it was nodered/node-red-docker). The docker hub URL doesn't get updated when you change the repo and has to be updated manually (and is really just there so you have a link to where the docker container came from.. It auto fills from the CA but if you ever change the repository you need to manually update the Docker Hub URL if you want it to be correct).

 

I recommend using a version tag from Node-RED as breaking changes can occur between major versions (i.e. 12.0.0 to 13.0.0) that can require changes to be made before updating (or some of your modules aren't updated to run on the new version which may be why you had to remove those modules.. you may now be able to reinstall them via node-red and it will pull the updated version for the latest NR release). It is generally bad practice to use no tag as it will always pull the latest tag which can sometimes be development (or whatever was built last on dockerhub for this repo). This is why I am using nodered/node-red:latest-12 instead of nodered/node-red (which is the same as nodered/node-red:latest) so I always have the latest version of Node-RED 12. I actually recommend going through all your containers and making this update as it can prevent issues with updating down the road (each container has their own tags though so you will need to look those up yourself on their respective hub pages).

 

19 minutes ago, jbuszkie said:

Thanks for the info.  so it was really just as easy as purging the log!  I wanted to make sure.  Thanks!  I'll have to look into the log rotate if I want to do that.

 

You're welcome! I ran into the same issue quite a while back so glad I was able to help you out.

Link to comment
1 hour ago, wdenny said:

Should it be pulling from: 

https://hub.docker.com/r/nodered/node-red

 

I was actually wrong in my previous post. The Docker Hub URL is actually used in Unraid to check for updates. So keeping this up-to-date is handy if you like to see new versions that are released and update via the Unraid GUI. So yes, do update the Docker Hub URL to point to the new non-deprecated URL.

 

Link to comment
18 hours ago, Skylord123 said:

I was actually wrong in my previous post. The Docker Hub URL is actually used in Unraid to check for updates. So keeping this up-to-date is handy if you like to see new versions that are released and update via the Unraid GUI. So yes, do update the Docker Hub URL to point to the new non-deprecated URL.

 

There seems to have been a problem with a template not updating to Github. I have changed the template with the correct url and added the Safe Mode parameter. It should roll out soon for new users.

Link to comment
  • 2 weeks later...
  • 4 weeks later...

Has anyone been able to successfully get the nbrowser node working in the Node-RED container? Whenever I try it it crashes Node-RED.

26 Dec 19:43:59 - Error: spawn /data/node_modules/electron/dist/electron ENOENT

at Process.ChildProcess._handle.onexit (internal/child_process.js:240:19)
at onErrorNT (internal/child_process.js:415:16)
at process._tickCallback (internal/process/next_tick.js:63:19)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"`
npm ERR! Exit status 1


npm ERR! Failed at the [email protected] start script.

npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /data/.npm/_logs/2020-12-27T03_43_59_278Z-debug.log

Per this thread it seems like there may be some additional package needed although that’s the extent of my technical skills.

https://github.com/segmentio/nightmare/issues/224

 

Edited by unRaide
Link to comment
On 12/27/2020 at 3:52 AM, unRaide said:

Has anyone been able to succssfully get the nbrowser node working in the Node-RED container? Whenever I try it it crashes Node-RED.

Per this thread it seems like there may be some additional package needed although that’s the extent of my technical skills.

https://github.com/segmentio/nightmare/issues/224

 

The NodeRed team did not allow SUDO or APT-GET in their docker images so unfortunately you will need to build a different docker file if you need to do this. (INCORRECT)

I just built the template for the official docker image (and abandoned my custom one some time ago).

 

What are you trying to do with nbrowser?

 

 

EDIT:

add "--user=0" without the quotes to Extra Parameters, this will allow you to use the APK command to install any dependancies

Edited by spants
Link to comment
5 hours ago, spants said:

What are you trying to do with nbrowser?

Hi @spants, thx for the response!

 

I'm trying to create a flow that periodically logs into a site (audible.com) and extracts one piece of data. nbrowser seems like a node that would allow me to handle this entire flow e.g.  login to a site and handle the cookies, redirect, etc. but unfortunately it has the issues mentioned above :(.

 

Definitely open to other ideas on how to accomplish the same task.

 

Link to comment
On 12/27/2020 at 7:59 PM, unRaide said:

Hi @spants, thx for the response!

 

I'm trying to create a flow that periodically logs into a site (audible.com) and extracts one piece of data. nbrowser seems like a node that would allow me to handle this entire flow e.g.  login to a site and handle the cookies, redirect, etc. but unfortunately it has the issues mentioned above :(.

 

Definitely open to other ideas on how to accomplish the same task.

 

nbrowser seems to have not been updated for a long while. There is another node (scrape-it) that may help?. I used to use the HTTP nodes when I wrote an integration to HiveHome - but I swapped that over to HomeAssistant.

Link to comment
On 11/21/2020 at 7:26 PM, UNOPARATOR said:

I would prefer if you would change the "Icon Url" to a square icon, because it doesn't look good at the dashboard or docker tab.

image.png.03ab48249482fa16110f67ab399eeff4.png

Such as https://nodered.org/about/resources/media/node-red-icon.png from the Node-RED Resources page.

Which would look much better in unRAID:

image.png.bd51c92acc5ae6221aab2884bd8262c1.png

@spants, have you missed my message? I know this is a cosmetic issue but it is a nice *nice to have* feature. ;)

Link to comment
21 hours ago, UNOPARATOR said:

@spants, have you missed my message? I know this is a cosmetic issue but it is a nice *nice to have* feature. ;)

I have pushed this - note that you will not see the change if you already downloaded NR as the template is cached on your boot drive.

Thanks for the link - I didnt see it when I first made the template in 2016!

 

You can change it on your own system btw

Edited by spants
  • Thanks 1
Link to comment
1 hour ago, spants said:

I have pushed this - note that you will not see the change if you already downloaded NR as the template is cached on your boot drive.

Thanks for the link - I didnt see it when I first made the template in 2016!

 

You can change it on your own system btw

I already did change the iconUrl on my side, but with each NodeRed update, it was overwritten hence my cry for help ;) 

Glad to hear that I won't be needing to do this manually from now on. 👍

Link to comment
I already did change the iconUrl on my side, but with each NodeRed update, it was overwritten hence my cry for help  
Glad to hear that I won't be needing to do this manually from now on.
You may have to keep changing it as the template is not refreshed on the update. Check your /boot/config directory structure for the cached templates and also edit it there


Sent from my SM-N976B using Tapatalk

  • Thanks 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.