Out of nowhere my GUI stop working and it happened 3 times already.  A reboot will fix it but I am annoyed as I don't know why on my last reboot from the CLI is doing a parity check now

 

How do I troubleshoot this?  Dockers are working fine. Nothing changed.  Can I do a command line diagnostic? what's the syntax? I am on 6.12.13

 

 

Edited December 11, 20241 yr by DivideBy0

2 minutes ago, DivideBy0 said:

Out of nowhere my GUI stop working and it happened 3 times already.  A reboot will fix it but I am annoyed as I don't know why on my last reboot from the CLI is doing a parity check now

 

How do I troubleshoot this?  Dockers are working fine. Nothing changed.  Can I do a command line diagnostic? what's the syntax? I am on 6.12.13

 

 

Run diagnostics in cli

Well, it did it again in less than 24 hours. Luckily the parity check is done.  I am flying blind now, web GUI is gone

 

Attached see the diagnostic and syslog files. Hopefully is something simple?

 

 

 

 

nas-unraid-2-diagnostics-20241121-1607.zip syslog-192.168.0.13.log

Reviewing logs

?you don't have a docker in host network that is also a web server?

as docker running host will replace unraids ports and can break webui access

Edited November 21, 20241 yr by bmartino1
typo - acident clipbaod form another post

enable ssh when its stable and try to ssh into the machine.
* we can also then use termainl camds to recoverstart stor reset dockers and services...

From the logs provided, it seems your Unraid system's web GUI stops responding intermittently. Based on the syslogs, here are potential troubleshooting steps and actions to resolve the issue:

 

Diagnosing the Issue

Check for High Memory or CPU Usage
Intermittent web UI outages can occur if the system is under heavy load. Check the logs for processes consuming excessive CPU or memory.

Run htop or top from the CLI to monitor usage in real time.

Faulty Plugins or Dockers
A misbehaving Docker container or plugin could be overwhelming the system.

Stop all Dockers and see if the web UI stabilizes.

Re-enable Dockers one by one to identify the culprit.

Log Overflow or Disk Space Issues
Check if the /var/log directory is full:

df -h /var/log

If it’s full, it might prevent services from functioning. Rotate or clear logs if necessary.

Browser Cache or Connectivity Issues
Ensure it’s not a client-side issue by accessing the UI from a different browser or device. Alternatively, try accessing via:

http://[SERVER_IP]:[PORT]
 

 

Immediate Workarounds

Restart the WebGUI Service Without Rebooting If the GUI becomes unresponsive, restart it via CLI:

/etc/rc.d/rc.nginx restart
 

/etc/rc.d/rc.nginx restart
/etc/rc.d/emhttp restart

 

Access Docker Containers or Shares Even without the GUI, you can manage Dockers using docker commands:

docker ps
docker stop <container_id>
docker start <container_id>

 

To check shares:

ls /mnt/user/

Did aray/fuse stop?

 

Possible Fixes

Update Unraid Ensure you are running the latest stable version of Unraid. Version 6.12.13 might have updates or fixes available.

Check for Hardware Errors Hardware instability (e.g., bad RAM or failing drives) can cause such issues. Run a memory test or check the drives for errors.

Disable Problematic Plugins If you suspect a plugin, try booting into "Safe Mode" (without plugins) and see if the GUI becomes stable. Then re-enable plugins one by one.

Increase nginx Timeout Settings
If the issue is related to heavy web GUI processes, increase the nginx timeout:

Edit /etc/nginx/nginx.conf and increase client_body_timeout, client_header_timeout, and keepalive_timeout.

Enable Syslog Mirroring To capture logs over time, configure remote logging to an external syslog server. This helps identify patterns if the GUI becomes unresponsive.

 

/etc/rc.d/rc.nginx restart  will not restart the GUI, I get the same blank screen

 

/etc/rc.d/emhttp restart it does not work, I don't have emhttp.

 

All dockers and shares are working find in the background. No full logs or hardware errors.

 

The only change I made to this was the iGPU plugin which stopped working. Other than than, nothing substantially changed

 

As I am looking this further even tho the server is running without a GUI I also can not see it in the Unraid Connect.  Something is going on here

 

 

On 11/21/2024 at 5:51 PM, DivideBy0 said:

/etc/rc.d/rc.nginx restart  will not restart the GUI, I get the same blank screen

 

/etc/rc.d/emhttp restart it does not work, I don't have emhttp.

 

All dockers and shares are working find in the background. No full logs or hardware errors.

 

The only change I made to this was the iGPU plugin which stopped working. Other than than, nothing substantially changed

 

emhttp is the unraid web ui. i don't rember the comand to restart it it may be a direct call
emhttp stop
emhttp start

check your go file note the call for emhttp that is calling the unriad web ui. it used to be nginx...

THIS is is very very very ANNOYING problem and trying to troubleshoot it. It happened again after 4 days or so. Can anyone help here? 

 

Diagnostic attached

 

When trying to restart I get this in the syslog

 

Nov 30 13:54:39 NAS-UNRAID-2 nginx: 2024/11/30 13:54:39 [crit] 8285#8285: *224 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 192.168.0.89, server: , request: "GET /Dashboard HTTP/2.0", subrequest: "/auth-request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192.168.0.13"
Nov 30 13:54:39 NAS-UNRAID-2 nginx: 2024/11/30 13:54:39 [error] 8285#8285: *224 auth request unexpected status: 502 while sending to client, client: 192.168.0.89, server: , request: "GET /Dashboard HTTP/2.0", host: "192.168.0.13"
Nov 30 13:54:39 NAS-UNRAID-2 nginx: 2024/11/30 13:54:39 [crit] 8285#8285: *224 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 192.168.0.89, server: , request: "GET /favicon.ico HTTP/2.0", subrequest: "/auth-request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192.168.0.13", referrer: "https://192.168.0.13/Dashboard"
Nov 30 13:54:39 NAS-UNRAID-2 nginx: 2024/11/30 13:54:39 [error] 8285#8285: *224 auth request unexpected status: 502 while sending to client, client: 192.168.0.89, server: , request: "GET /favicon.ico HTTP/2.0", host: "192.168.0.13", referrer: "https://192.168.0.13/Dashboard"
Nov 30 13:54:55 NAS-UNRAID-2 nginx: 2024/11/30 13:54:55 [crit] 8285#8285: *224 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 192.168.0.89, server: , request: "GET /Dashboard HTTP/2.0", subrequest: "/auth-request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192.168.0.13"
Nov 30 13:54:55 NAS-UNRAID-2 nginx: 2024/11/30 13:54:55 [error] 8285#8285: *224 auth request unexpected status: 502 while sending to client, client: 192.168.0.89, server: , request: "GET /Dashboard HTTP/2.0", host: "192.168.0.13"
Nov 30 13:54:55 NAS-UNRAID-2 nginx: 2024/11/30 13:54:55 [crit] 8285#8285: *224 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 192.168.0.89, server: , request: "GET /favicon.ico HTTP/2.0", subrequest: "/auth-request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "192.168.0.13", referrer: "https://192.168.0.13/Dashboard"
Nov 30 13:54:55 NAS-UNRAID-2 nginx: 2024/11/30 13:54:55 [error] 8285#8285: *224 auth request unexpected status: 502 while sending to client, client: 192.168.0.89, server: , request: "GET /favicon.ico HTTP/2.0", host: "192.168.0.13", referrer: "https://192.168.0.13/Dashboard"
Nov 30 13:56:46 NAS-UNRAID-2 usbhid-ups[4652]: nut_libusb_get_string: Pipe error

 

 

nas-unraid-2-diagnostics-20241130-1337.zip

Edited November 30, 20241 yr by DivideBy0

Reviewing diag attached. may not get any other useful info as you would need a diag when problems occur before a reboot...

Can't help with unraid connect and how that connects and manages contact support and review the unraid connect stuff. as for unraid stabilizes issues, diag are only apart for the picture. it sounds more like docker issues, theses are not captured in diag for secutiy reasons. docker inspect is your frined. I would recommend resource management and pin some dockers down as you may be too resource intensive for the hardware.
 

root@BMM-Unraid:~# emhttp
emhttp: Starting emhttpd...
emhttp: emhttpd already started.
root@BMM-Unraid:~# 

as emhttp is the unraid webserver ui... what you are doing on unraid to break that is hard to diagnose. form ramfss bad moutns misconfigurations... regadless not enoth info...

so let use what we have whcih is a good start and baseline... the unriad diag...


you may have instablilty with your power managment and nut plugin and settings...


The syslog reveals several recurring errors related to USB and power management, specifically tied to the usbhid-ups (UPS monitoring) service:

 

Key Observations

usbhid-ups Errors:

Repeated errors indicate communication issues with the UPS:

nut_libusb_get_report: Input/Output Error
nut_libusb_get_string: Input/Output Error

 

This suggests instability in the USB connection or configuration of the UPS monitoring service.

While These errors might not directly cause the GUI to stop responding but could indicate broader USB or system resource management issues that could indirectly affect system stability.

 

The filtered syslog entries reveal regular activity related to the emhttpd (Unraid's web server) process, but no clear errors or failures directly related to emhttpd that might explain the GUI becoming unresponsive. Here's what the log shows:

Observations

Normal Disk Activity:

emhttpd frequently reads SMART data and issues spin-down commands for drives (/dev/sdh, /dev/sdg, etc.), which is expected behavior for Unraid's management system

 

Flash Backup:

A scheduled task related to flash backup was triggered

flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update

BUT... No Critical Errors Detected with EMHTTP! so it not urnaaids default configuration its a addon plugin/docker... safemode and start removing till you find the culprit...
?autotweak?

 

now lets look at the previos sylog that unraid was able to capture:
 

Depednign on motheborad and gcard bios opptions resizeable bare error:

Quote

pci 0000:01:00.0: BAR 9: failed to assign [mem size 0x00400000 64bit]
pci 0000:01:00.0: BAR 7: failed to assign [mem size 0x00040000 64bit]
 

 

This could indicate hardware conflicts or BIOS/firmware issues.

 

Looks like ntp error. turn off ntp or fix and time syn it..

ntpd[1721]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized

*fix in bios if problems...

 

The emhttpd process starts successfully but detects an unclean shutdown

emhttpd: unclean shutdown detected
*Where the user power cycled the machine... i rather you push the button let singal term do a 15 min shutdown stoping stuff to colect a better previous boot diagnostic...

Again..
Plugin Activity:

Several plugins, such as dynamix and intel-gpu-top, were being installed or configured. Misbehaving plugins could cause instability.

 

So here are my recomendataion atm:

 

PCI Resource Conflicts:

Check the BIOS for resource allocation settings. Updating the BIOS or disabling unused hardware (e.g., a secondary GPU) might help.

*Make sure pcie uefi resizeale bar is enabled

 

NTP Configuration:

Ensure the server can reach NTP servers to synchronize the clock

Settings > Date and Time > NTP Servers
 

ntpdate -u pool.ntp.org

 

Review any plugins new that may be misconfigured or causing problems most likly Nut/power related...

 

 

I really appreciate it man but this is a bit of shotgun approach here.  This server was STABLE as hell for years, nothing changed.  I don't think USB UPS errors can cause this. Everything is running perfect in the background, all dockers and shares are fine.  Besides the latest apps or plugin updates nothing really changed. Unless one of the updates ef-ed it up.  There is gotta be a more surgical way to troubleshoot this. 

setup a syslog server have unraid reoport to that server for logs.

Does this mean anything?

 

Nov 30 14:26:18 NAS-UNRAID-2 unraid-api[1051]: 👋 Farewell. UNRAID API shutting down!
Nov 30 14:26:23 NAS-UNRAID-2 root: {"level":"info","time":"2024-11-30T20:26:23.708Z","pid":5378,"hostname":"NAS-UNRAID-2","logger":"cli","msg":"Starting [email protected]"}
Nov 30 14:26:23 NAS-UNRAID-2 root: {"level":"info","time":"2024-11-30T20:26:23.870Z","pid":5378,"hostname":"NAS-UNRAID-2","msg":"Watch Setup on Config Path: /boot/config/plugins/dynamix.my.servers/myservers.cfg"}
Nov 30 14:26:25 NAS-UNRAID-2 unraid-api[5727]: ✔️ UNRAID API started successfully!
Nov 30 14:26:58 NAS-UNRAID-2 webGUI: Successful login user root from 192.168.0.89
Nov 30 14:28:01 NAS-UNRAID-2 crond[1782]: exit status 255 from user root php /usr/local/emhttp/plugins/community.applications/scripts/notices.php > /dev/null 2>&1
Nov 30 14:35:55 NAS-UNRAID-2 nginx: 2024/11/30 14:35:55 [error] 5376#5376: *36 open() "/usr/local/emhttp/state/plugins/dynamix.docker.manager/images/tautulli-icon.png" failed (2: No such file or directory) while sending to client, client: 192.168.0.7, server: , request: "GET /state/plugins/dynamix.docker.manager/images/tautulli-icon.png?1732281398 HTTP/2.0", host: "192.168.0.13", referrer: "https://192.168.0.13/Dashboard"

Edited November 30, 20241 yr by DivideBy0

yes that's unraid connect api you would need to contact unraid suport or the unraid connect forum post on issues with that. unraid uses a API to send your uraid webui and configuration to their server for when you access unraid you can contral said server. Ther was a api update did you update the plugin?

The api is failing, but not enoth info as to why. also, not my area of expertise. As I couldn't get the unraid conect to work for me without sacrificing security and having my webui Public accesible...

Review

https://unraid.net/connect

https://connect.myunraid.net/

https://docs.unraid.net/connect/

 

Ask on entire different sub forum:
https://forums.unraid.net/forum/94-connect-plugin-support/

I found my solution and it was GPU Statistics. I read several posts on reddit pointing to this pluggin and I removed. No more GUI issues, so far. One week so far.