• nginx 404 - 6.10 Upgrade


    m00f
    • Urgent

    After upgrading to 6.10 , 

    GUI stopped working , same in safe mode..

     

    i'm kinda stuck atm, cant work ;/

     

    is it even possible to revert updates ? 

    image.png.1df44f3e09dafa2aacdd8f8a76502d3f.png

    b0x-diagnostics-20220518-1356.zip




    User Feedback

    Recommended Comments



    On 5/18/2022 at 5:44 PM, ljm42 said:

     

    OK we figured it out. There was a bug in 6.9.2 that allowed you to use https://servername.local even though it was not valid for the hash.unraid.net certificate the server was configured to use. And since self-signed certificates also throw errors the browser probably didn't make it clear exactly which error you were accepting. So it would have been tough for you to know that the system was misconfigured in 6.9.2.

     

    We are going to tweak the upgrade process so that anyone upgrading from 6.9.2 with USE_SSL=auto will change to USE_SSL=yes in 6.10.0. That will eliminate this problem, and put users in control of whether they want the more secure/restrictive USE_SSL=auto setting.

    This is going to take a while so bear with me:

     

    I used to have two ways that worked to access my server-GUI for this server and one that didn't work (and that I should have fixed but didn't get around to, I know...):

    1. https://hanebijters2.duckdns.org:47438/Dashboard (remote): worked
    2. https://192.168.1.223:2443/ (local): worked
    3. https://hash.unraid.net (didn't work)

     

    If is set <use_ssl no> I get to the login with http://192.168.1.223/login, but I cannot log in! No errors, but no cigar either.

    (NB! I know username and pwd are correct because I can SSH into the server with those)

    So the situation now is as follows:

    1.  https://hanebijters2.duckdns.org:47438/Dashboard gives me a 404 error
    2. as does https://192.168.1.223:2443: 404
    3. https://hash.unraid.net gives me a 'can't connect (as before the upgrade from 6.9.2)
    4. https://192.168.1.223:1443/ gets me to the nginx server in 'swag', cf. the well known configuration I 'borrowed' from SpaceInvaderOne.  Although he was using the Letsencrypt (as swag was formerly known) plugin, the functionality is exactly the same. All GUI's for docker plugins (Syncthing, Nextcloud, Jellyfin, etc.)  are perfectly accessible
    5. http://192.168.1.223/login brings me to the login screen, but I can't login! No errors, just nothing.

     

    Of course it's 5 that is the most annoying.

     

    Port-forwarding is as follows:

    • UnRaid_http 80 180 (as suggested by SpaceInvaderOne)

    • UnRaid_https 443 1443 (idem)

    • UnRaid Remote access 47438 2443 (for the unraid.net login)

     

    Changing <use_ssl no> to <ussl_ssl yes> (or USE_SSL=yes) and v.v. makes no difference, the results are exactly the same

     

    So: I cannot send you my system_diagnostics file, because I still don't have access to the GUI. Is there a command that would create that file?

     

    So, basically, I'm stuck right now.

     

     

     

    Link to comment
    6 minutes ago, RikStigter said:

    http://192.168.1.223/login brings me to the login screen, but I can't login! No errors, just nothing.

     

    Trying to get you back up and running as quickly as possible, so just addressing this issue for now:

     

    Try http://192.168.1.223/login in your browser's private/incognito mode. 

    • Like 1
    Link to comment
    23 hours ago, ljm42 said:

     

    Trying to get you back up and running as quickly as possible, so just addressing this issue for now:

     

    Try http://192.168.1.223/login in your browser's private/incognito mode. 

    Bingo! That works!

     

    Cleared the cache for that adress and now it works in 'normal' browser-mode as well.

     

    Thank you!

     

    Now for the rest of the URL's I have used. I'm including the diagnostics

    tower-diagnostics-20220520-0632.zip

    Edited by RikStigter
    Link to comment
    17 hours ago, RikStigter said:

    Bingo! That works!

     

    Cleared the cache for that adress and now it works in 'normal' browser-mode as well.

     

    OK great! Glad we're past that.

     

    Unraid 6.9.2 had a behavior/bug which allowed you to use https://whateveryouwant.com even though the SSL certificate did not match the url. Unraid 6.10.0 corrects that and will only respond when the url matches the certificate, otherwise it gives a 404. (actually there is one exception to that, if Use SSL is "yes" then it will also respond to https://ipaddress using a self-signed cert)

     

    So: 
    1) https://hanebijters2.duckdns.org:47438 will give a 404 unless you set the server name to hanebijters2 and the Local TLD to duckdns.org.  Probably not what you want to do.

     

    1a) However, the My Servers remote access url does work: https://www.hash.unraid.net:47438  Depending on how your router is configured, this will probably only work from outside your network.

     

    The best way to access this url is via the My Servers Dashboard ( https://forums.unraid.net/my-servers/ ) - if it detects you are on the same network as the server it will show the Local Access url, if it detects you are on a different network it will show you the Remote Access url.

     

    BTW once you get past this you might want to change the remote access port to something else, just to keep script kiddies away from your server now that it has been published here.

     

    2) https://192.168.1.223:2443 will respond using a self-signed cert if you set Use SSL to Yes. It will not work if Use SSL is No or Auto.  (Think of "auto" as "extremely strict" mode)

     

    3) https://hash.unraid.net:2443 . The server is setup for this to work, but DNS Rebinding Protection is enabled on your network, so hash.unraid.net is unable to resolve to your internal IP. If you can disable DNS Rebinding Protection then this will work as well (try googling "nameOfYourRouter disable DNS rebinding protection") Until you resolve that, leave Use SSL on No or Yes, not Auto.

    • Like 1
    • Thanks 1
    Link to comment
    8 hours ago, ljm42 said:

     

    OK great! Glad we're past that.

     

    Unraid 6.9.2 had a behavior/bug which allowed you to use https://whateveryouwant.com even though the SSL certificate did not match the url. Unraid 6.10.0 corrects that and will only respond when the url matches the certificate, otherwise it gives a 404. (actually there is one exception to that, if Use SSL is "yes" then it will also respond to https://ipaddress using a self-signed cert)

     

    So: 
    1) https://hanebijters2.duckdns.org:47438 will give a 404 unless you set the server name to hanebijters2 and the Local TLD to duckdns.org.  Probably not what you want to do.

     

    1a) However, the My Servers remote access url does work: https://www.hash.unraid.net:47438  Depending on how your router is configured, this will probably only work from outside your network.

     

    The best way to access this url is via the My Servers Dashboard ( https://forums.unraid.net/my-servers/ ) - if it detects you are on the same network as the server it will show the Local Access url, if it detects you are on a different network it will show you the Remote Access url.

     

    BTW once you get past this you might want to change the remote access port to something else, just to keep script kiddies away from your server now that it has been published here.

     

    2) https://192.168.1.223:2443 will respond using a self-signed cert if you set Use SSL to Yes. It will not work if Use SSL is No or Auto.  (Think of "auto" as "extremely strict" mode)

     

    3) https://hash.unraid.net:2443 . The server is setup for this to work, but DNS Rebinding Protection is enabled on your network, so hash.unraid.net is unable to resolve to your internal IP. If you can disable DNS Rebinding Protection then this will work as well (try googling "nameOfYourRouter disable DNS rebinding protection") Until you resolve that, leave Use SSL on No or Yes, not Auto.

     Right! A couple of observations:

     

    1) Most definitely not! ;-)

    1a) Locally that works. When accessed through the My Servers Dashboard ( https://forums.unraid.net/my-servers/ )It rewrites to http://tower.local/login and that is OK. From outside my network it doesn't work: It rewrites to https://www.hash.unraid.net and I reach the base-page of nginx (the 'My Server' page). If I add the newly configured port (https://www.hash.unraid.net:xxxxx it does work. I have changed ports.🙃

     

    For me this is totally workable and secure.

     

    Now it gets interesting:

    2) Nope, still get 404 with <USE_SSL=yes>

    3) DNS rebind is not active on my network as far as I can tell (just checked my router). So I cannot disable it and I still get a 'Server not found'.

     

    Now there are a few things that might influence this since I'm running a caching DNS-server (Unbound), I'm running a DShield-honeypot and I'm running IPv6. I'll have to figure out how that might influence the situation, but that requires a lot of hard thought (and lots of paper) and it's still early over here (the Netherlands) as I write this (I'm not putting this to 'it has been a while since I did this kind of stuff on a daily basis', because that would make me feel very old <g>). I'll keep you posted as I go along, but it may take a while... I feel the need for a few espresso's. 🙂

     

    We'll get there. Many thanks and have a pleasant weekend!

     

    Rik

     

     

    Edited by RikStigter
    Typo's and additional info
    Link to comment
    23 hours ago, RikStigter said:

    -snip-

     

    2) Nope, still get 404 with <USE_SSL=yes>

    3) DNS rebind is not active on my network as far as I can tell (just checked my router). So I cannot disable it and I still get a 'Server not found'.

     

    Now there are a few things that might influence this since I'm running a caching DNS-server (Unbound), I'm running a DShield-honeypot and I'm running IPv6. I'll have to figure out how that might influence the situation, but that requires a lot of hard thought (and lots of paper) and it's still early over here (the Netherlands) as I write this (I'm not putting this to 'it has been a while since I did this kind of stuff on a daily basis', because that would make me feel very old <g>). I'll keep you posted as I go along, but it may take a while... I feel the need for a few espresso's. 🙂

     

    We'll get there. Many thanks and have a pleasant weekend!

     

    Rik

     

     

    Ok, so I think I know why it didn't work.

     

    1. I had made a bit of a hash of my port-forwarding table. Too many entries and no sorting of any kind so barely readable. That's fixed now. One does get a bit complacent using IPv6, I guess. Not a real acceptable reason, but you know... It's a hobby network, mostly. 🤪
    2. Some of my rules were not adjusted correctly to the settings in 'Management Access'. My guess is that I had experimented so much that I forgot some of the things I had tried.

     

    <Sigh> I would have had something (tough) to say if that happened during my professional career phase and not just to myself!

     

    Now what still does not work is accessing the servers through my own domain with the correct port-adresses, e.g. https://hanebijters2.duckdns.org:xxxxx. That results in a 404 on both servers. Now I could try to find out why, but I have to admit that since it is now working perfectly through https://forums.unraid.net/my-servers/ (and I've bookmarked the new addresses) I'm inclined to think that it isn't worth the trouble anymore. At least not at this time. Other things are waiting for my attention. 😅

     

    Another big thanks to ljm42 for your help. I hope that others may have the same smooth experience with their problems!

     

    Rik

     

    Edited by RikStigter
    Link to comment

    Great! Glad you were able to sort that out. I have run into the same where you try so many things you forget what you've changed :) 

     

    6 hours ago, RikStigter said:

    https://hanebijters2.duckdns.org:xxxxx. That results in a 404 on both servers

     

    A 404 in this case is correct. In 6.10, Unraid will only respond to urls in the Subject line of the certificate, so https://servername.localTLD or variations of (my)unraid.net, not  https://anythingatall.com 

     

    Unraid 6.10 supports two certs:

     

    To use a cert for hanebijters2.duckdns.org at /boot/config/ssl/certs/[servername]_unraid_bundle.pem you would have to change the servername and localTLD of the server to match the cert.

     

    If you don't want to use My Servers Remote Access or (my)unraid.net cert, you could put a cert for hanebijters2.duckdns.org at /boot/config/ssl/certs/certificate_bundle.pem. I would call that semi-supported, as in... I'm pretty sure it will work today (not tested) but it might break in the future because we aren't really expecting people to do that. 

     

    My recommendation would be to drop the duckdns url and continue with My Servers Remote Access.

    • Upvote 1
    Link to comment

    Quick question--I was previously using the old single-host Let's Encrypt SSL cert on 6.9.x.

    I always accessed my server using https://<local-ip>/ and it would simply redirect to the <hash>.unraid.net URL.
    Upon upgrading to 6.10, https://<local-ip>/ no longer redirects...it simply brings me to the login page with the browser complaining the connection isn't secure (due to the self signed cert). I should also add that https://<hostname>.localTLD/ behaves the same way...no redirect, just an insecure connection.

     

    Was this behavior intentionally changed? I assume nginx on 6.9.x was automatically redirecting where as it is not in 6.10. If not intentional, any idea how to fix this? It was really nice to simply enter my IP rather than having to make sure I always had a bookmark on the device for the <hash>.unraid.net URL.

    Edited by dcarpenter85
    Link to comment

    This has happened to me. I also created a bug report but updated that all I had to do was close all my browsers then load up only one instance of the WebUI and it worked. If I tried to open up another tab while one was already opened I would get that 404 NOT FOUND NGINX. I am using non secure mode since I do not open my LAN to the outside world.

     

    This is the link to my bug report -> 

     

     

    I do not know if unraid itself is blocking to only have one instance of the Webgui running, but that's what it seems. The error populated in my syslog when this happens is below. My server is 192.168.1.200.

     

    May 19 21:12:58 SUN nginx: 2022/05/19 21:12:58 [error] 7633#7633: *829549 open() "/usr/local/emhttp/main" failed (2: No such file or directory) while sending to client, client: 192.168.1.122, server: , request: "GET /main HTTP/1.1", host: "192.168.1.200"

     

    root@SUN:/# ls -al /usr/local/emhttp/webGui
    lrwxrwxrwx 1 root root 15 Jun 19  2016 /usr/local/emhttp/webGui -> plugins/dynamix/

     

    ls -al /usr/local/emhttp/webGui/Main.page

    -rw-r--r-- 1 root root 49 Aug  4  2021 /usr/local/emhttp/webGui/Main.page

    Edited by opentoe
    Link to comment
    12 hours ago, ljm42 said:

    -snip-

     

    My recommendation would be to drop the duckdns url and continue with My Servers Remote Access.

    That is my conclusion as well.

     

    Two final questions: Would it be safe to remove the current 'certificate_bundle.pem'? Considering the file's date I'd guess a resounding 'NO' is most likely. The other .pem-file in that directory is old and can probably be removed. I've back upped both, just to be on the sure side.

     

    This being a 'hobby'-network I need to make sure I don't have too many complex installation-setups. I'd say that that is probably true for most of us. Now that I have phased out my other servers (Ubuntu LTS and a OMV6-testbed server) I don't have to worry about all the different management-tools anymore. KISS, as it is. 😀

    Now I'll be updating my documentation to reflect this wisdom!

    Edited by RikStigter
    Link to comment
    9 hours ago, RikStigter said:

    Would it be safe to remove the current 'certificate_bundle.pem'? Considering the file's date I'd guess a resounding 'NO' is most likely. The other .pem-file in that directory is old and can probably be removed.

    certificate_bundle.pem contains your unraid.net certificate, so you don't want to delete it.

     

    [servername]_unraid_bundle.pem contains the self-signed cert which is still used, so you don't want to delete that either.

     

    10 hours ago, RikStigter said:

    This being a 'hobby'-network I need to make sure I don't have too many complex installation-setups. I'd say that that is probably true for most of us. Now that I have phased out my other servers (Ubuntu LTS and a OMV6-testbed server) I don't have to worry about all the different management-tools anymore. KISS, as it is. 😀

    Now I'll be updating my documentation to reflect this wisdom!

     

    Glad you are liking Unraid!  And I'm impressed you have documented your setup, that can be really helpful

     

    Link to comment

    https://wiki.unraid.net/Manual/Release_Notes/Unraid_OS_6.10.0#Moving_to_Let.27s_Encrypt_wildcard_SSL_certificates.
     

    Quote

     

    Finally, we have set up nginx such that the URL's:

    http://<server-name>.<local-tld>/

    or

    https://<server-name>.<local-tld>/

    will redirect to https://[lan-ip].[hash].myunraid.net

     

     

    Can anyone confirm for me that this is indeed the case? Because this is not working for me. https://<server-name>.<local-tld>/ does not redirect. It simply goes to the login page using a self signed cert.

     

    I am using a new wildcard cert and I have tried all permutations of setting SSL to off/on/auto. Auto shows a 404 instead of redirecting, on or off both go to the login page using either no ssl or a self signed cert.

    Link to comment
    1 hour ago, dcarpenter85 said:

    Can anyone confirm for me that this is indeed the case? Because this is not working for me. https://<server-name>.<local-tld>/ does not redirect. It simply goes to the login page using a self signed cert.

     

    I am using a new wildcard cert and I have tried all permutations of setting SSL to off/on/auto. Auto shows a 404 instead of redirecting, on or off both go to the login page using either no ssl or a self signed cert.

     

    We might need to work on that wording.

     

    If you access this url: http://<server-name>.<local-tld>/

      * If Use SSL is no - the login screen will render at that url (http, no cert)

      * If Use SSL is yes - you will be redirected to https://<server-name>.<local-tld>/ (https, self-signed cert)

      * If use SSL is auto - you will be redirected to https://[lan-ip].[hash].myunraid.net (https, full cert)

     

    If at any point you try logging in and can't even though you are using the right password, clear your browser's cache. Browsers get confused when you enable/disable SSL.

    Link to comment
    3 minutes ago, ljm42 said:

     

    We might need to work on that wording.

     

    If you access this url: http://<server-name>.<local-tld>/

      * If Use SSL is no - the login screen will render at that url (http, no cert)

      * If Use SSL is yes - you will be redirected to https://<server-name>.<local-tld>/ (https, self-signed cert)

      * If use SSL is auto - you will be redirected to https://[lan-ip].[hash].myunraid.net (https, full cert)

     

    If at any point you try logging in and can't even though you are using the right password, clear your browser's cache. Browsers get confused when you enable/disable SSL.


    Oddly enough, after upgrading to 6.10.1 yesterday from 6.9.2, SSL set to auto was showing a 404 if I went to http://<server-name>.<local-tld>/ or http://<server-ip>/. Just tested again and it redirects as expected. 

    It looks like https://<server-name>.<local-tld>/ or https://<server-ip>/ do not redirect and just go to the login page using the self-signed cert. Regardless, my issue seems to have resolved itself and the behavior I was used to in 6.9.x is back. Thank you!

    Link to comment
    2 minutes ago, dcarpenter85 said:


    Oddly enough, after upgrading to 6.10.1 yesterday from 6.9.2, SSL set to auto was showing a 404 if I went to http://<server-name>.<local-tld>/ or http://<server-ip>/. Just tested again and it redirects as expected. 

    It looks like https://<server-name>.<local-tld>/ or https://<server-ip>/ do not redirect and just go to the login page using the self-signed cert. Regardless, my issue seems to have resolved itself and the behavior I was used to in 6.9.x is back. Thank you!

     

    Not sure between yesterday and today, it may have been a browser caching issue causing confusion

    Link to comment
    On 5/21/2022 at 2:32 PM, RikStigter said:

    <Sigh> I would have had something (tough) to say if that happened during my professional career phase and not just to myself!

     

    Don't be so tough on yourself: I had the same issues and thanks to this thread I found my router rebind protection configuration and whitelisted myunraid.net, additional to unraid.net that was whitelisted some years ago, but who remembered that? 😀

    Link to comment

    Well.... I've skipped reboot after some update of the lastest. now when I finally did a reboot (because of some auto-start/ mods to pci-e pass) I got stuck.... and if I wouldn't have found this thread and the recommendation of:

     

    Try http://192.168.1.223/login in your browser's private/incognito mode. 

     

    I would have lost the rest of my sanity. 

     

    Thanks. P.s - this should go to release/ update notes :P

    Link to comment



    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.