6.12.3 Restarts multiple times daily

[Teknoclown]

Since upgrading to 6.12.2 and now with 6.12.3 my server randomly crashes and resets itself.  The only thing I can see in the logs is an NGINX error, but not sure what thats related to.  Attached is the log files and the persistent syslog.  Any help would be appreciated.

mediacenter-diagnostics-20230720-1138.zip syslog-10.0.1.140.log

[Teknoclown]

Consistently I see this error:

Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *151 open() "/usr/local/emhttp/server-status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /server-status?auto HTTP/1.1", host: "localhost"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *152 open() "/usr/local/emhttp/server-status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /server-status?auto HTTP/1.1", host: "127.0.0.1"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *154 open() "/usr/local/emhttp/us" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /us HTTP/1.1", host: "localhost"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *155 open() "/usr/local/emhttp/us" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /us HTTP/1.1", host: "127.0.0.1"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *156 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 127.0.0.1, server: , request: "GET /admin/api.php?auth=&version=true HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "127.0.0.1"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *158 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 127.0.0.1, server: , request: "GET /admin/api.php?auth=&version=true HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "localhost"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *160 open() "/usr/local/emhttp/server-status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /server-status?auto HTTP/1.1", host: "localhost"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *161 open() "/usr/local/emhttp/server-status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /server-status?auto HTTP/1.1", host: "127.0.0.1"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *162 open() "/usr/local/emhttp/status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /status?full&json HTTP/1.1", host: "localhost"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *163 open() "/usr/local/emhttp/status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /status?full&json HTTP/1.1", host: "127.0.0.1"
Jul 20 12:41:27 MediaCenter nginx: 2023/07/20 12:41:27 [error] 11593#11593: *164 open() "/usr/local/emhttp/status/format/json" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /status/format/json HTTP/1.1", host: "127.0.0.1"
Jul 20 12:41:29 MediaCenter nginx: 2023/07/20 12:41:29 [error] 11593#11593: *173 open() "/usr/local/emhttp/basic_status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /basic_status HTTP/1.1", host: "127.0.0.1"
Jul 20 12:41:29 MediaCenter nginx: 2023/07/20 12:41:29 [error] 11593#11593: *174 open() "/usr/local/emhttp/stub_status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /stub_status HTTP/1.1", host: "localhost"
Jul 20 12:41:29 MediaCenter nginx: 2023/07/20 12:41:29 [error] 11593#11593: *175 open() "/usr/local/emhttp/stub_status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /stub_status HTTP/1.1", host: "127.0.0.1"
Jul 20 12:41:29 MediaCenter nginx: 2023/07/20 12:41:29 [error] 11593#11593: *176 open() "/usr/local/emhttp/nginx_status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /nginx_status HTTP/1.1", host: "127.0.0.1"
Jul 20 12:41:29 MediaCenter nginx: 2023/07/20 12:41:29 [error] 11593#11593: *177 open() "/usr/local/emhttp/status" failed (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /status HTTP/1.1", host: "127.0.0.1"
Jul 20 12:41:29 MediaCenter nginx: 2023/07/20 12:41:29 [error] 11593#11593: *178 "/usr/local/emhttp/api/index.html" is not found (2: No such file or directory), client: 127.0.0.1, server: , request: "GET /api/ HTTP/1.1", host: "127.0.0.1"
 

[Teknoclown]

In this Most recent SHutdown I really can't tell what happened. 

 

This is the Syslog output:

 

Jul 21 00:14:01 MediaCenter root: Fix Common Problems Version 2023.07.16
Jul 21 00:14:01 MediaCenter root: Fix Common Problems: Warning: Docker Application netdata has an update available for it
Jul 21 00:14:01 MediaCenter root: Fix Common Problems: Warning: Docker Application nextcloud has an update available for it
Jul 21 00:14:17 MediaCenter root: Fix Common Problems: Other Warning: unRAID6-Sanoid Support Updated
Jul 21 03:38:57 MediaCenter kernel: mdcmd (36): set md_write_method 1
Jul 21 03:38:57 MediaCenter kernel: 
Jul 21 03:38:58 MediaCenter cache_dirs: Arguments=
Jul 21 03:38:58 MediaCenter cache_dirs: Max Scan Secs=10, Min Scan Secs=1
Jul 21 03:38:58 MediaCenter cache_dirs: Scan Type=adaptive
Jul 21 03:38:58 MediaCenter cache_dirs: Min Scan Depth=4
Jul 21 03:38:58 MediaCenter cache_dirs: Max Scan Depth=none
Jul 21 03:38:58 MediaCenter cache_dirs: Use Command='find -noleaf'

 

What is happening between 00:14 and 03:38?  I don't get it, what am I missing?

[Teknoclown]

I read another thread that said the NGINX error was NetData, so shut that down.  Additionally I have tried removing and reinstalling UnRaid Connect (No Change with it on or off).  Currently I have NerdTools removed, although I can't remember if I required that for something, so we will see if any of the Docker stuff breaks..

[JorgeB]

16 minutes ago, Teknoclown said:

I really can't tell what happened. 

There's nothing logged, if the server is rebooting on it's own it could be a hardware issue.

[Teknoclown]

yeah I think I have ruled out most software.  I rolled back to 6.11.5, rebuilt Docker Image, Ran with no plugins, still same results in general.  I wonder if its glitchy power or something like that.  My UPS just seems to lose connection to the Server, not like its causing a shutdown, but loses connection and then a restart happens.  Any ideas where to start on the hardware side? Like I don't know what would cause random reboots

[JorgeB]

PSU would IMHO be a good place to start.

[Teknoclown]

So PSU was newish in the last 3 months, what I didn't do was replace the some of the cables from the old PSU.  Both were EVGA and everything fit fine, but that may have been my issue. After replacing any of the power cables, I'm at 5 days of uptime now, on 12.3, no issues, things running as they should.  I hope that was it

[JonathanM]

4 hours ago, Teknoclown said:

what I didn't do was replace the some of the cables from the old PSU.

You are very lucky, there have been multiple instances of people blowing up components by reusing cables they were SURE were compatible.

 

Even from the same brand isn't a guarantee, because brands don't always make their own product, they relabel from a few giant manufacturers.