I've been having some weird issues with my server the last couple of days. 

 

* I've had 2 of 4 cores showing pinned at 100% utilization

* CA is set to auto-update all dockers and plugins, but they haven't been updating

* I cannot open the WebUI of any of my dockers

* I cannot generate a Diagnostics .zip file by clicking the option in the Tools menu

 

I thought to click the Log button on the WebGUI, and this is what it shows:

 

Quote

Dec 9 19:49:34 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:34 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:35 NAS nginx: 2019/12/09 19:49:35 [error] 6786#6786: *4234766 limiting requests, excess: 20.842 by zone "authlimit", client: 192.168.1.6, server: , request: "GET /webGui/images/close.png HTTP/1.1", host: "nas", referrer: "http://nas/Tools/Diagnostics"
Dec 9 19:49:35 NAS nginx: 2019/12/09 19:49:35 [error] 6786#6786: *4234767 limiting requests, excess: 20.842 by zone "authlimit", client: 192.168.1.6, server: , request: "GET /webGui/images/sort-asc.png HTTP/1.1", host: "nas", referrer: "http://nas/Tools/Diagnostics"
Dec 9 19:49:35 NAS nginx: 2019/12/09 19:49:35 [error] 6786#6786: *4234766 limiting requests, excess: 20.838 by zone "authlimit", client: 192.168.1.6, server: , request: "GET /webGui/images/file-types.png HTTP/1.1", host: "nas", referrer: "http://nas/Tools/Diagnostics"
Dec 9 19:49:37 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:37 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:43 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:43 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:44 NAS root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: wrong csrf_token
Dec 9 19:49:50 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:50 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:51 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:51 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:51 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:53 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:53 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:53 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:54 NAS root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: wrong csrf_token
Dec 9 19:49:55 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:49:55 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:00 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:00 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:04 NAS root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: wrong csrf_token
Dec 9 19:50:05 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:05 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:06 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:06 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:06 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:10 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:10 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:10 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:14 NAS root: error: /plugins/dynamix.system.temp/include/SystemTemp.php: wrong csrf_token
Dec 9 19:50:15 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:15 NAS root: error: /webGui/include/DashUpdate.php: wrong csrf_token
Dec 9 19:50:28 NAS kernel: mdcmd (805): set md_write_method 1
Dec 9 19:50:28 NAS kernel:
Dec 9 19:50:42 NAS nginx: 2019/12/09 19:50:42 [error] 6786#6786: *4234264 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/Download.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Tools/Diagnostics"

 

As I was posting this, I thought I should reboot the server first, just to see if that would fix things. I clicked "Restart" in the GUI, and now I'm getting a 504 Gateway Time-out when trying to reach the WebGUI, but dockers are responsive. Since the above output from the Log window was captured, the following has been added:

 

Quote

Dec 9 19:53:38 NAS nginx: 2019/12/09 19:53:38 [error] 6786#6786: *4235171 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:53:54 NAS nginx: 2019/12/09 19:53:54 [error] 6786#6786: *4234681 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:54:09 NAS nginx: 2019/12/09 19:54:09 [error] 6786#6786: *4235159 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:54:14 NAS php-fpm[6760]: [WARNING] [pool www] server reached max_children setting (20), consider raising it
Dec 9 19:54:24 NAS nginx: 2019/12/09 19:54:24 [error] 6786#6786: *4235168 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:54:39 NAS nginx: 2019/12/09 19:54:39 [error] 6786#6786: *4235172 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:54:54 NAS nginx: 2019/12/09 19:54:54 [error] 6786#6786: *4235169 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:55:53 NAS nginx: 2019/12/09 19:55:53 [error] 6786#6786: *4235171 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:56:09 NAS nginx: 2019/12/09 19:56:09 [error] 6786#6786: *4234681 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:56:24 NAS nginx: 2019/12/09 19:56:24 [error] 6786#6786: *4235159 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:56:39 NAS nginx: 2019/12/09 19:56:39 [error] 6786#6786: *4235168 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:56:54 NAS nginx: 2019/12/09 19:56:54 [error] 6786#6786: *4235172 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:56:54 NAS nginx: 2019/12/09 19:56:54 [error] 6786#6786: *4235169 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:56:56 NAS nginx: 2019/12/09 19:56:56 [error] 6786#6786: *4235439 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/Notify.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:57:00 NAS nginx: 2019/12/09 19:57:00 [error] 6786#6786: *4235479 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:57:00 NAS nginx: 2019/12/09 19:57:00 [error] 6786#6786: *4235530 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:57:03 NAS nginx: 2019/12/09 19:57:03 [error] 6786#6786: *4236002 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /plugins/dynamix.system.temp/include/SystemTemp.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:57:05 NAS nginx: 2019/12/09 19:57:05 [error] 6786#6786: *4236008 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:57:53 NAS nginx: 2019/12/09 19:57:53 [error] 6786#6786: *4235171 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /plugins/dynamix.apcupsd/include/UPSstatus.php?all=false HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:58:09 NAS nginx: 2019/12/09 19:58:09 [error] 6786#6786: *4234681 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:58:24 NAS nginx: 2019/12/09 19:58:24 [error] 6786#6786: *4235159 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:58:39 NAS nginx: 2019/12/09 19:58:39 [error] 6786#6786: *4235168 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:58:54 NAS nginx: 2019/12/09 19:58:54 [error] 6786#6786: *4235172 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/Notify.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:58:54 NAS nginx: 2019/12/09 19:58:54 [error] 6786#6786: *4235169 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /plugins/dynamix.system.temp/include/SystemTemp.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:58:54 NAS nginx: 2019/12/09 19:58:54 [error] 6786#6786: *4236216 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /Tools/Diagnostics HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Tools"
Dec 9 19:58:56 NAS nginx: 2019/12/09 19:58:56 [error] 6786#6786: *4235439 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /plugins/dynamix.apcupsd/include/UPSstatus.php?all=false HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:59:00 NAS nginx: 2019/12/09 19:59:00 [error] 6786#6786: *4235479 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:59:00 NAS nginx: 2019/12/09 19:59:00 [error] 6786#6786: *4235530 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:59:03 NAS nginx: 2019/12/09 19:59:03 [error] 6786#6786: *4236002 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:59:05 NAS nginx: 2019/12/09 19:59:05 [error] 6786#6786: *4236008 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 19:59:53 NAS nginx: 2019/12/09 19:59:53 [error] 6786#6786: *4235171 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:00:02 NAS speedtest: Internet bandwidth test started
Dec 9 20:00:09 NAS nginx: 2019/12/09 20:00:09 [error] 6786#6786: *4234681 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:00:24 NAS nginx: 2019/12/09 20:00:24 [error] 6786#6786: *4235159 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /plugins/dynamix.local.master/include/LocalMaster.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:00:38 NAS speedtest: Host: AT&T (Indianapolis, IN) [47.90 km]
Dec 9 20:00:38 NAS speedtest: Ping (Lowest): 44.981 ms | Download (Max): 55.27 Mbit/s | Upload (Max): 5.97 Mbit/s
Dec 9 20:00:38 NAS speedtest: Internet bandwidth test completed
Dec 9 20:00:39 NAS nginx: 2019/12/09 20:00:39 [error] 6786#6786: *4235168 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:00:54 NAS nginx: 2019/12/09 20:00:54 [error] 6786#6786: *4236216 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Tools/Diagnostics"
Dec 9 20:00:56 NAS nginx: 2019/12/09 20:00:56 [error] 6786#6786: *4235439 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:01:08 NAS nginx: 2019/12/09 20:01:08 [error] 6786#6786: *4235169 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/Boot.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:03:08 NAS nginx: 2019/12/09 20:03:08 [error] 6786#6786: *4235169 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/webGui/include/Boot.php"
Dec 9 20:06:13 NAS nginx: 2019/12/09 20:06:13 [error] 6786#6786: *4237031 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /plugins/dynamix.system.temp/include/SystemTemp.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:06:13 NAS nginx: 2019/12/09 20:06:13 [error] 6786#6786: *4237035 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:06:13 NAS nginx: 2019/12/09 20:06:13 [error] 6786#6786: *4237036 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:06:13 NAS nginx: 2019/12/09 20:06:13 [error] 6786#6786: *4237039 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:06:13 NAS nginx: 2019/12/09 20:06:13 [error] 6786#6786: *4237033 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:06:13 NAS nginx: 2019/12/09 20:06:13 [error] 6786#6786: *4237034 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "POST /webGui/include/DashUpdate.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:08:13 NAS nginx: 2019/12/09 20:08:13 [error] 6786#6786: *4237031 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /Dashboard HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Tools/Diagnostics"
Dec 9 20:10:13 NAS nginx: 2019/12/09 20:10:13 [error] 6786#6786: *4237031 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:11:15 NAS nginx: 2019/12/09 20:11:15 [error] 6786#6786: *4237578 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /Dashboard HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Tools/Diagnostics"
Dec 9 20:13:15 NAS nginx: 2019/12/09 20:13:15 [error] 6786#6786: *4237578 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:16:13 NAS nginx: 2019/12/09 20:16:13 [error] 6786#6786: *4238102 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /Dashboard HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Tools/Diagnostics"
Dec 9 20:18:13 NAS nginx: 2019/12/09 20:18:13 [error] 6786#6786: *4238102 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"
Dec 9 20:18:22 NAS kernel: mdcmd (806): spindown 1
Dec 9 20:18:32 NAS kernel: mdcmd (807): spindown 2
Dec 9 20:18:45 NAS kernel: mdcmd (808): spindown 3
Dec 9 20:18:57 NAS kernel: mdcmd (809): spindown 4
Dec 9 20:19:09 NAS kernel: mdcmd (810): spindown 5
Dec 9 20:19:18 NAS kernel: mdcmd (811): spindown 6
Dec 9 20:19:59 NAS kernel: usb 4-2: new low-speed USB device number 2 using ohci-pci
Dec 9 20:19:59 NAS kernel: input: Microsoft Natural® Ergonomic Keyboard 4000 as /devices/pci0000:00/0000:00:13.0/usb4/4-2/4-2:1.0/0003:045E:00DB.0002/input/input5
Dec 9 20:19:59 NAS kernel: microsoft 0003:045E:00DB.0002: input,hidraw1: USB HID v1.11 Keyboard [Microsoft Natural® Ergonomic Keyboard 4000] on usb-0000:00:13.0-2/input0
Dec 9 20:19:59 NAS kernel: input: Microsoft Natural® Ergonomic Keyboard 4000 as /devices/pci0000:00/0000:00:13.0/usb4/4-2/4-2:1.1/0003:045E:00DB.0003/input/input6
Dec 9 20:20:00 NAS kernel: microsoft 0003:045E:00DB.0003: input,hidraw2: USB HID v1.11 Device [Microsoft Natural® Ergonomic Keyboard 4000] on usb-0000:00:13.0-2/input1
Dec 9 20:20:28 NAS kernel: mdcmd (812): set md_write_method 0
Dec 9 20:20:28 NAS kernel:
Dec 9 20:22:53 NAS nginx: 2019/12/09 20:22:53 [error] 6786#6786: *4238791 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /Dashboard HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Tools/Diagnostics"
Dec 9 20:22:58 NAS nginx: 2019/12/09 20:22:58 [error] 6786#6786: *4238802 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /Tools/Diagnostics HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Tools"
Dec 9 20:23:07 NAS nginx: 2019/12/09 20:23:07 [error] 6786#6786: *4238820 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.6, server: , request: "GET /Main HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "nas", referrer: "http://nas/Dashboard"

 

The "Spindown" messages are a bit odd since I believe that shutting down the array first requires that all devices be spun up. There are about 3 minutes between the last message in the first group and the first message in the second group. The "new USB device" was me plugging in a keyboard - usually I do have a monitor and keyboard plugged in, but I didn't at this particular point. I do now have both the monitor & keyboard plugged in

 

Would the Log window have survived a reboot? None of the text from the first batch were cleared from that window as though it had attempted a refresh, received nothing (because the server was rebooting), then cleared itself; instead it seems to have been continually displaying the log file tail and doesn't seem to have indicated that the server went down at all.

 

I can log into the console.

I have a PuTTY session that didn't disconnect (hence, the server hasn't rebooted), and is responsive.

I can browse server shares from a Windows Explorer.

 

What suggestions from here?

Edited December 13, 20196 yr by FreeMan

Thank you, I'll take that under advisement, @BRiT.  (At this point I'm sort of switching from Firefox to Brave, so a lot of things are open in both. I opened Chrome wondering if there was some sort of issue with the Chromium engine all of a sudden, so I had 3 connections open, all of which are getting the 504 error.)

 

What's the suggested fix to get the WebUI working again?

I attempted to shutdown the server from the console with:

powerdown -r

The server is still running.

 

I've attempted to collect diagnostics from the console with:

root@NAS:/var/log# diagnostics
Starting diagnostics collection...

It's been sitting there for 10+ minutes with no return to the prompt.

 

I managed to:

cp /var/log/syslog* /boot/logs/

and pulled the 3 attached syslog* files from my flash drive (I have it shared. Yes... I know...)

 

The 3 "Accepted password" lines at the end are from 3 different logins via PuTTy after I did the powerdown. The man page for shutdown indicates there are no options to force a shutdown as was suggested in this forum post 

but unRAID uses powerdown, not shutdown, so I'd imagine it has some special sauce prior to issuing the actual shutdown command. Is there a Force parameter I could try sending?

Any other suggestions before I push the power button?

syslog syslog.1 syslog.2

To shut the server down when commands (GUI or Command Line) won't work, try these suggestions. 

 

First, press the power switch on the case for about one second.  This should start a normal shutdown (that you can watch happening on the console monitor). If it finishes, this will be a clean shutdown. 

 

Second, if that doesn't work, push and hold the power switch for longer than five seconds.  This will force a unclean shutdown and a parity check will occur on restart. 

Thanks@Frank1940. I did the short press this morning before heading out the door. I heard disk spinning and heads moving, but the server was still running 10 min later and nothing appeared on the console.

I'll grab a fresh copy of the syslog when I get back home.

Are there any other files I can post that may help identify what's going on since diagnostics isn't working from the command line either?

Sent from Tapatalk

If something happened after the short press and the server never powered down, it is hung up.  You can look at the console screen, take a picture and see if anyone has any thoughts on where it is hung up at.  (By the way, what appears on the monitor is in a video buffer on the MB so you can connect a monitor at any time to display that buffer!)

 

You are probably going to have to force a shutdown with a long push.  It is not what we like to do but sometimes it is the only solution.  Make sure you have shutdown all browsers that you were using to access the server.  (That would include any that are on mobile devices.)  Then start the server back up and see where you are after that.  It will start a parity check.  It may find and correct errors (That often occurs in these situations) but let the check finish.

Edited December 11, 20196 yr by Frank1940

Yeah, I've had to do a hard power down in the past, too. More than once. 

 

Fortunately, unRAID is more than stable enough that when I have issues, I don't remember how to attempt to force it to shut down short of pulling the plug (figuratively).

 

Parity check is running now. I'd expect that the wonky issues I was having will go away now, too.

Parity check complete. No errors. No signs of previous weirdness, for now at least. Updated to 6.8.0.

 

I guess I'll close this out.