bpage Posted December 9, 2019 Share Posted December 9, 2019 Not sure exactly what is going on. The only thing I did was remove the libvirtwol plugin and after reboot all went to hell. It looks like nginx is not loading upon boot. When I do manually start it, I get tons of errors in /var/log.nginx/error.log. They are below. When running in safe mode, the gui and the array load fine. Any ideas? I am on 6.8.0-rc7. 2019/12/09 11:26:45 [crit] 19929#19929: *1 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:26:45 [error] 19929#19929: *1 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:26:45 [crit] 19929#19929: *3 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:26:45 [error] 19929#19929: *3 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:26:48 [crit] 19929#19929: *10 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:26:48 [error] 19929#19929: *10 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:26:50 [crit] 19929#19929: *13 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:26:50 [error] 19929#19929: *13 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:26:52 [crit] 19929#19929: *17 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:26:52 [error] 19929#19929: *17 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:26:54 [crit] 19929#19929: *21 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.2.2, server: , request: "GET /Main HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:26:54 [error] 19929#19929: *21 auth request unexpected status: 502 while sending to client, client: 10.0.2.2, server: , request: "GET /Main HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:26:54 [crit] 19929#19929: *23 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.2.2, server: , request: "GET /favicon.ico HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10", referrer: "http://10.0.1.10/Main" 2019/12/09 11:26:54 [error] 19929#19929: *23 auth request unexpected status: 502 while sending to client, client: 10.0.2.2, server: , request: "GET /favicon.ico HTTP/1.1", host: "10.0.1.10", referrer: "http://10.0.1.10/Main" 2019/12/09 11:26:56 [crit] 19929#19929: *27 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:26:56 [error] 19929#19929: *27 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:26:59 [crit] 19929#19929: *32 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:26:59 [error] 19929#19929: *32 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:27:00 [crit] 19929#19929: *35 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:27:00 [error] 19929#19929: *35 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:27:03 [crit] 19929#19929: *41 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:27:03 [error] 19929#19929: *41 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:27:04 [crit] 19929#19929: *43 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" 2019/12/09 11:27:04 [error] 19929#19929: *43 auth request unexpected status: 502 while sending to client, client: 10.0.1.35, server: , request: "GET /sub/dockerload HTTP/1.1", host: "10.0.1.10" 2019/12/09 11:27:07 [crit] 19929#19929: *51 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 10.0.1.35, server: , request: "GET /sub/var?last_event_id=1575906911%3A3 HTTP/1.1", subrequest: "/auth_request.php", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "10.0.1.10" Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 Ok. So after manually starting php-fpm via ssh I can get to the webgui, but, the array is stuck on "Array Starting * Mounting Disks" Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 Find diagnostics attached. pageraid-diagnostics-20191209-1637.zip Quote Link to comment
trurl Posted December 9, 2019 Share Posted December 9, 2019 Why are you running the webUI on a non-standard port. Your go file: #!/bin/bash # Start the Management Utility /usr/local/sbin/emhttp -p 2785& cp /boot/custom/docker-shell /usr/local/bin Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 6 minutes ago, trurl said: Why are you running the webUI on a non-standard port. Your go file: #!/bin/bash # Start the Management Utility /usr/local/sbin/emhttp -p 2785& cp /boot/custom/docker-shell /usr/local/bin I am not sure why that is in go file, as I typically hit the webgui on port 80.... Quote Link to comment
trurl Posted December 9, 2019 Share Posted December 9, 2019 And what about that cp line? What is the purpose of that? Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 The cp line is to add an easy way to get into the docker shell. That has been working for a few years at this point. I have removed the -p config. I am trying a reboot now. As seen here: #!/bin/bash # easily "docker exec" into a running Docker container # latest version: https://gist.github.com/ljm42/2b3bfd8ff886015bbce8 # for unRAID, place this script on your flash drive as /boot/custom/docker-shell # then add this to your go script (without the leading pound sign): # cp /boot/custom/docker-shell /usr/local/bin CONTAINERS=`docker ps | awk 'NR==1 {offset=index($0,"NAMES")};NR>1{print substr($0,offset)}' | sort -f | tr "\n" " "` echo echo "Choose a Docker container:" i=1 for container in $CONTAINERS do echo "$((i++)) : $container" done read MYCHOICE CHOSEN=`echo $CONTAINERS | cut -d' ' -f$MYCHOICE 2>/dev/null` if [ "$CHOSEN" = "" ]; then echo Invalid option exit fi # try running bash, if exit code 126 then try running sh for SHELL in bash sh do clear echo " " echo -e '\E[30;42m'"\033[5m $CHOSEN - $SHELL \033[0m" echo " " tput sgr0 docker exec -it $CHOSEN $SHELL if [ $? -ne 126 ]; then break fi done Quote Link to comment
trurl Posted December 9, 2019 Share Posted December 9, 2019 1 minute ago, bpage said: The cp line is to add an easy way to get into the docker shell. You can easily get a shell inside any of your containers by simply clicking on its icon on the Dashboard or Docker page and selecting >_ Console. Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 Just now, trurl said: You can easily get a shell inside any of your containers by simply clicking on its icon on the Dashboard or Docker page and selecting >_ Console. That's true and I don't really use that shell script anymore, now that I have gotten better using direct docker commands. But again, that had been there for a long time and my server was just fine. I am happy to remove it, but I do not suspect that is causing the issue. Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 rebooting after commenting out the -p flag and the cp command resulted in the same issue. NGINX does not start. Booting into Safe mode still works fine. Quote Link to comment
trurl Posted December 9, 2019 Share Posted December 9, 2019 5 minutes ago, bpage said: Booting into Safe mode still works fine That normally indicates a plugin issue. Post a new diagnostic. Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 attached pageraid-diagnostics-20191209-1728.zip Quote Link to comment
trurl Posted December 9, 2019 Share Posted December 9, 2019 Nothing obvious. Delete all .plg files in config/plugins and see if it boots in normal mode with no plugins. Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 I see lots of folders as well as .plg files. Should I leave the folders and just delete the file? Quote Link to comment
trurl Posted December 9, 2019 Share Posted December 9, 2019 3 minutes ago, bpage said: I see lots of folders as well as .plg files. Should I leave the folders and just delete the file? Nothing from the folders will be loaded without the corresponding .plg file Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 3 minutes ago, trurl said: Nothing from the folders will be loaded without the corresponding .plg file Ok. Trying now. I should also mentioned, I have a backup of the flash drive from a couple of days ago. I used Duplicacy to backup and just restored to a folder on my desktop. Not sure if I could just replace the contents and reboot.... It seems the kernel is older though. 5.3.12 vs 4.19.56 Quote Link to comment
trurl Posted December 9, 2019 Share Posted December 9, 2019 Not familiar with that method to backup flash. You can just download a zipped copy of flash from Main - Boot Device - Flash - Flash Backup. Different version of kernel would of course be a different version of Unraid. Usually a new install with the config folder copied from backup should get things going as they were, but let's see what happens without plugins. Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 Ok. So without the plugins, the server boots correctly and the the array starts. Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 One other thing to note, I did last week install a 10gbe network card and I wonder if that may have been causing this. I say this because when I booted the server after removing the plugins, I did not have any internet access to the outside world. I believe this may have been because the 10gbe network (different subnet) did not have WAN access. I am not exactly sure how to have the 10gbe network ONLY work for internal storage sharing, but not for unRAID internet access. I am not exactly sure I am explaining that correctly... Quote Link to comment
bpage Posted December 9, 2019 Author Share Posted December 9, 2019 Regardless, I much appreciate the help @trurl ! I was growing more concerned by the minute. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.