6.8.x NGINX Upstream Error Brings Unraid To Its Knees


Recommended Posts

I keep running into issues with Unraid 6.8.0 and now 6.8.1 releases which ultimately causes me to rollback to a previous build.  

What im seeing in the syslog are nginx errors over and over and over. Once I see one error, unraid starts to become extremely unstable to the point of web gui becoming unresponsive and then the entire system goes unresponsive to the point where ssh into the machine isnt even possible :o 

 

I've seen a few other posts with others having similar issues but no solution has been discovered.

Does anyone have any idea where i can even start to dig into the root cause?  

 

 

Error that i see multiple times in syslog:

Jan 17 10:26:47 NEXUS nginx: 2020/01/17 10:26:47 [error] 10374#10374: *17 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.104, server: , request: "POST /webGui/include/DashUpdate.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "_________.unraid.net", referrer: "https://_______.unraid.net/Dashboard"

nginx.conf attached

 

Screen Shot 2020-01-17 at 10.54.24 AM.png

Edited by drventure31
Link to comment

Changed to safe mode via main > boot device > syslinux config and selected unRAID OS Safe Mode (no plugins, no GUI).  

Rebooted the system and then it ended up hanging so had to hard shutdown.   

On boot (with array off) i see no nginx erros SWEET!  


However, syslog does show some other errors. Most peculuar is the one regarding a floppy and kernel bug related to drivers. 

Jan 17 12:28:13 NEXUS kernel: print_req_error: I/O error, dev fd0, sector 0
Jan 17 12:28:13 NEXUS kernel: floppy: error 10 while reading block 0
Jan 17 12:28:16 NEXUS kernel: ------------[ cut here ]------------
Jan 17 12:28:16 NEXUS kernel: kernel BUG at drivers/ata/sata_mv.c:2118!
Jan 17 12:28:16 NEXUS kernel: invalid opcode: 0000 [#1] SMP NOPTI
Jan 17 12:28:16 NEXUS kernel: CPU: 4 PID: 1466 Comm: scsi_eh_32 Not tainted 4.19.94-Unraid #1

 

 

Link to comment
2 minutes ago, drventure31 said:

Jan 17 12:28:13 NEXUS kernel: print_req_error: I/O error, dev fd0, sector 0 Jan 17 12:28:13 NEXUS kernel: floppy: error 10 while reading block 0

Disable floppy on board BIOS

 

2 minutes ago, drventure31 said:

Jan 17 12:28:16 NEXUS kernel: kernel BUG at drivers/ata/sata_mv.c:2118!

These are likely Supermicro SAT2-MV8 controllers, they have known issues with recent devices, both HDD e SSDs, driver not likely to be maintained any longer, best bet if it's not working correctly would be to replace them with an LSI HBA.

Link to comment

Sounds good, Ill jump out there and flip those off in bios. Could the controllers be triggering issues with nginx?  

Since the controllers i have right now are old and unsupported in 6.8.x it could be possible that the files nginx process is digging for are just unavailable? Just spitballing here..

side note: Just got the nginx error while waiting for the array to try to start (it never started fyi).  

Jan 17 12:37:21 NEXUS nginx: 2020/01/17 12:37:21 [error] 2205#2205: *842 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.104, server: , request: "POST /update.htm HTTP/2.0", upstream: "http://unix:/var/run/emhttpd.socket/update.htm", host: "_____.unraid.net", referrer: "https://_____.unraid.net/Main"

 

I've previously looked at new controllers but decided to wait to upgrade... guess the time has come :D   

Anyone have any recommendations? Otherwise, im just going to research through the list posted here: https://wiki.unraid.net/Hardware_Compatibility 

 

 

Link to comment

Any other ideas on things to check for the nginx errors that seem to be crippling unraid 6.8.x?  

 

  • I/O error 'floppy'
    • FIX: try and disable in BIOS 
  • kernel BUG at drivers/ata/sata_mv.c:2118!
    • FIX: Drivers for (Marvell MV88SX6081 8port SATA II PCI-X) likely not being maintained, recommend upgrading controllers if on Unraid 6.8.x 
  • nginx error - upstream timed out while reading response header (see above for full error) .

    • Workaround: roll back to non 6.8.x versions of unraid.  

Link to comment

is it possible this nginx error could be related to the same one im experiencing? erros look different, but nginx seems to be a common denominator. i havent found a solution yet either. could try rolling back to 6.7 and see... but i never had any problems before 6.8 either, and no other major changes since the upgrade aside from wireguard install / use. 

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.