Main GUI NGINX crash. Any smart way to re-start it with a script?

[dopeytree]

Occasionally the main GUI nginx instance crashes.

 

I'd like to know if anyone has a solution for this. 

 

Is there a script I can run to detect a crash and re-start the main GUI NGINX?

 

Other services are always working but no access to server.

 

I currently keep SSH turned off.

 

Currently new server build has no soft power button on my server to initiate a restart. Although have now bought one on amazon.

 

Edited April 19 by dopeytree

[dopeytree]

Ok here's a user script.

 

#!/bin/bash

NGINX_LOG="/var/log/nginx/error.log"

# Monitor NGINX error log for specific error messages
tail -n0 -F ${NGINX_LOG} | while read LINE; do
    if echo $LINE | grep -q "504 Gateway Time-out"; then
        echo "NGINX error detected: $LINE"
        echo "Restarting NGINX..."
        systemctl restart nginx
    fi
done

 

and schedules to run every minute with * * * * *

 

 

[bmartino1]

I sometimes see this on older hardware and only at boot.

Using your script above one could use the flash drive's go file for this.

Otherwise, the above script in a cron via the user script plug doe indeed accomplish this.

The go file is located at /boot/config in unraid or in the config folder when the unraid flash drive is plugged into another PC.

My understanding is that unraid uses nginx for ssl https. They now use emhttp for the main web gui server.

 

This is why the go file default starts the web server:
#!/bin/bash
# Start the Management Utility
/usr/local/sbin/emhttp &

to only have this on boot:

edit the go file to:

 

#!/bin/bash
# Start the Management Utility
/usr/local/sbin/emhttp &

#Nginx SSL - Https Check
NGINX_LOG="/var/log/nginx/error.log"

# Monitor NGINX error log for specific error messages
tail -n0 -F ${NGINX_LOG} | while read LINE; do
    if echo $LINE | grep -q "504 Gateway Time-out"; then
        echo "NGINX error detected: $LINE"
        echo "Restarting NGINX..."
        /etc/rc.d/rc.nginx restart
    fi
done

Unraid doesn't have services nor systemctl to check a server

/etc/rc.d/rc.%service_Name%

 

ex

/etc/rc.d/rc.nginx status

 

Edited April 21 by bmartino1

[dopeytree]

Thanks hmm ok weird.

 

For me a I have a 13500H and it only happens very rarely usually after a container update.

Specifically this happened after updating 'netdata' while the server was doing some other intensive tasks. It would also happen occasionally on the old server. Hence looking for a solution other than shutdown / reboot the server which can cause a parity error if its locked up something else.

 

When I do a search for '504 nginx' its quite common but no one seems to have built this kind of script into the OS yet.

 

Edited April 22 by dopeytree

[bmartino1]

are you  runnign lattes release?
6.12.10

if runing 6.12.8 or 6.12.9 you need to install the docker patch from squid.

[dopeytree]

On latest release.

 

Anyway why would docker lock up the GUI's nginx SSL layer?

Edited April 22 by dopeytree

[bmartino1]

I have only seen this once or twice. That was when I had docker run create a container and the system hit a ram out of memory. Once the docker template create and at thje docker run screen finished creating the docker, the web gui became normal.

 

If I were to guess, You may have a zombie process or a memory leak with an application taking up too many resources.

Terminal top to see command htop to search and kill.

https://itsfoss.com/kill-zombie-process-linux/

https://www.redswitches.com/blog/check-linux-memory-usage/

This could also be early signs of hardware failure.

Edited April 23 by bmartino1