(SOLVED) SSL Certificate No Longer Working in 6.10.0-rc1

[ehbush]

Hey ya'll,

 

I recently upgraded to 6.10.0-rc1. Previously (for the past ~2 years) I have been using a custom SSL certificate for the webgui (not just for unraid, but for most of my internal and external services). I have multiple widlcard SSL certificates for each of my relevant domains. Anyhow, as of 6.10.0, unraid has decided to discard my custom SSL certificate, and instead revert to using a self-signed SSL certificate.

 

I have attempted to fix this, by re-uploading my certificate bundle to the config/ssl/ directory, but no matter, upon reboot of nginx (either manually, or rebooting the entire server) it re-creates a self-signed cert and uses that as the default. I do not wish to use that, nor do I wish to use LetsEncrypt.

 

What is the proper way to solve this issue, and what/why was specifically changed in 6.10 that resorted in this unintended behaviour?

Edited August 15, 2021 by ehbush
Known Bug. Issue solved thanks to the advice and will be fixed in rc2

[glennv]

There is a bug in /etc/rc.d/rc.nginx in handling wildcard certificates. I reported it to@ljm42 and will be fixed in rc2.
For now:
around line 354 you see the code :

SUBJECT=$(openssl x509 -noout -subject -nameopt multiline -in $SSL/ce.... 

Right after that, add this:

SUBJECT=${SUBJECT/\*/$LANNAME} # support wildcard certs

put your cert in place and restart nginx with :

/etc/rc.d/rc.nginx restart

[ljm42]

7 hours ago, ehbush said:

What is the proper way to solve this issue, and what/why was specifically changed in 6.10 that resorted in this unintended behaviour?

 

Thank you for helping to test a prerelease version of Unraid. Congrats! you found a bug  This will be resolved in 6.10.0-rc2.

 

One request - please post prerelease bug reports in the prerelease board so they don't cause confusion for everyone on stable:

  https://forums.unraid.net/bug-reports/prereleases/

 

Here is more info on testing prereleases:

  https://forums.unraid.net/bug-reports/prereleases/how-to-install-prereleases-and-report-bugs-r8/ 

 

 

And for anyone who stumbles on this thread, I'll mention that release notes are in the same prerelease board mentioned above. Here is the direct link to the release notes for 6.10.0-rc1:

  https://forums.unraid.net/bug-reports/prereleases/unraid-os-version-6100-rc1-available-r1514/

 

 

 

 

[ehbush]

Thanks so much @glennv and @ljm42. Sorry, I had assumed something changed, not that there was a bug! Oops 🙂

Thanks for the quick response(s), and noted about proper forum etiquette!

[Kopernikus]

Hi,

 

I must be doing something wrong...

So I edit /etc/rc.d/rc.nginx from the terminal, adding the extra line.

Put my wilcard cert

Then /etc/rc.d/rc.nginx restart but nginx says gracefull shutdown but doesn't restart

After a reboot the extra line in /etc/rc.d/rc.nginx is disapeared

 

Edited August 23, 2021 by Kopernikus

[glennv]

29 minutes ago, Kopernikus said:

Hi,

 

I must be doing something wrong...

So I edit /etc/rc.d/rc.nginx from the terminal, adding the extra line.

Put my wilcard cert

Then /etc/rc.d/rc.nginx restart but nginx says gracefull shutdown but doesn't restart

After a reboot the extra line in /etc/rc.d/rc.nginx is disapeared

 

Either wait for RC2 or add some code to your /boot/config/go file to do it on the fly.

Without resorting to some code kunfu, the easiest would be :

1. edit the /etc/rc.d/rc.nginx to make it work  (make sure nginx can restart properly and your code did not break it !!!)

2. make a copy of it to for example  /boot/config/rc.nginx.tempfix

3. add a line to the go file to copy /boot/config/rc.nginx.tempfix back to  /etc/rc.d/rc.nginx

 

- Make sure you can always access your server via ssh (in case anything breaks the nginx gui)

- Remember to remove the extra line in the go file "before" "any" unraid upgrade !!!!

 

 

Edited August 23, 2021 by glennv

[Kopernikus]

Hi,

 

So like this:

 

# regenerate self-signed cert if local TLD changes */
  if [[ -f $SSL/certs/${LANNAME}_unraid_bundle.pem ]]; then
    SUBJECT=$(openssl x509 -noout -subject -nameopt multiline -in $SSL/certs/${LANNAME}_unraid_b>
    SUBJECT=${SUBJECT/\*/$LANNAME} # support wildcard certs
    [[ $SUBJECT != $LANFQDN ]] && rm -f $SSL/certs/${LANNAME}_unraid_bundle.pem
  fi

 

Update:

 

Did all steps and the change is now "permanent", however my servername_unraid_bundle.pem is still recreated whenever nginx is starting and overwriting my custom servername_unraid_bundle.pem

Edited August 23, 2021 by Kopernikus

[glennv]

are you using this literaly or did you replace the file with your actual unraid servername as part of the name ?
MYSERVER_unraid_bundle.pem

Its also case sensitive btw

 

Also test out the command to check your ssl cert . eg in my case (my hostname is TACH-UNRAID )

> hostname
TACH-UNRAID

> openssl x509 -noout -subject -nameopt multiline -in /boot/config/ssl/certs/TACH-UNRAID_unraid_bundle.pem
subject=
    commonName                = *.tachyon-consulting.com

and under management acces i have configured my domain:

 

 

The rc.nginx script compares the results of the above ssl command with  the Local TLD and if not equal it removes the file and replaces it with a regenerated one. The patch (extra line) removes the * from 1st command so they now match.

Maybe you have not filled in the Local TLD field or your SSL cert is not correct. So check both.

 

Edited August 23, 2021 by glennv

[maxstevens2]

So I decided to also take a shot on this, and I hacked my trough it!
What did I used to do?:
I just replaced the /boot/config/ssl/certs/(Your_ServerName)_unraid_bundle.pem with my own one.

How do I do it now?:
Well exact same script, but an edited rc.nginx config.
At 356 (or close to it) I removed the removal of the 'old' file by disabling the rm command.
at 365 (or close to it) I disabled the whole line, so it won't get overwritten.

This way my /boot/config/ssl/certs/Unraid_unraid_bundle.pem does not get overwritten, and I can use my own cert!

[ljm42]

The issue with wildcard certs in 6.10.0-rc1 should be resolved in rc2. Please see the docs for details on setting up a custom certificate:

https://wiki.unraid.net/Manual/Security#Securing_webGui_connections_.28SSL.29

[ReddotCleaner]

just updated to 6.10.facing the same problem

[BurntOC]

1 hour ago, ReddotCleaner said:

just updated to 6.10.facing the same problem

 

I'm not sure if my issue is exactly the same but I just upgraded one of my two servers to 6.10 just now as well and I'm now getting the certificate is untrusted message, too.

 

Before they both showed secure, and the one on 6.9.2 remains to be fine.  Hopefully we can resolve this so I don't have to roll back.

 

EDIT - to be clear, I don't use a custom cert, either.  I use Let's Encrypt, mainly wildcard for most everything.  My 6.9.2 is still showing LE cert and working as I haven't touched it.

 

EDIT2 - I thought maybe this had to do with not having updated the cert via the MyServers plugin, or whatever it is called.  I thought I recalled on one of the early RCs I recalled being told I had to do that at least once.  In any case, I tried that and "upgraded my cert" which did appear to generate a new cert, but it didn't fix the issue and I would prefer to use the LE certs anyway....

 

EDIT3 - Looks like we're getting bitten by the changes noted here https://unraid.net/blog/6-10-stable.  I may need to roll back until I understand it better.  I know I should read the release notes before upgrading, but this server is my most basic install and I'm not doing anything particularly special.  I also ran the Update Assistant and FCP beforehand as well as applied all updates.  I think things like this that will affect the vast majority of users probably ought to be copied to an acknowledgement popup in the upgrade workflow.

Edited May 18, 2022 by BurntOC

[ljm42]

43 minutes ago, BurntOC said:

I thought maybe this had to do with not having updated the cert via the MyServers plugin, or whatever it is called.

 

The My Servers plugin does not have anything to do with SSL, other than to tell you to enable it if you want to use the optional Remote Access feature. Installing / uninstalling the My Servers plugin will have no effect on the url or certificate used to access your server. That is all provided by Unraid itself.

 

 

I am not sure what your current status is, are you able to login to the webgui? If so, upload your diagnostics.zip (from Tools -> Diagnostics). If not, use SSH or connect a local keyboard / monitor and type "diagnostics". Note where it says it put the diagnostics.zip file on your flash drive, then get that file and upload it here.

[ReddotCleaner]

6 hours ago, ljm42 said:

 

The My Servers plugin does not have anything to do with SSL, other than to tell you to enable it if you want to use the optional Remote Access feature. Installing / uninstalling the My Servers plugin will have no effect on the url or certificate used to access your server. That is all provided by Unraid itself.

 

 

I am not sure what your current status is, are you able to login to the webgui? If so, upload your diagnostics.zip (from Tools -> Diagnostics). If not, use SSH or connect a local keyboard / monitor and type "diagnostics". Note where it says it put the diagnostics.zip file on your flash drive, then get that file and upload it here.

here is my status: every time I copy my working bundle.pem(yes, I have a backup of  6.9.2) file to /certs, then when i restart/ reload nginx, it will regenerate the self-signed ssl. So I have to click provision to get a ssl then edit to my owm ssl to get it work normally.

[ljm42]

4 hours ago, ReddotCleaner said:

here is my status: every time I copy my working bundle.pem(yes, I have a backup of  6.9.2) file to /certs, then when i restart/ reload nginx, it will regenerate the self-signed ssl. So I have to click provision to get a ssl then edit to my owm ssl to get it work normally.

 

I don't see a diagnostics.zip file, but I'll take a guess... you are trying to use your own custom certificate, but the url on the certificate does not match the servername.localTLD that the server is configured to use. This is causing Unraid to delete your certificate and create a self-signed one that matches Unraid's settings.

 

You have two options, either provide a certificate that matches the server's settings, or change the server's settings to match the certificate you want to use.

 

Take a look here:

  https://wiki.unraid.net/Manual/Security#Securing_webGui_connections_.28SSL.29

Start with the "A few details before we begin" section so you know how things are defined, then skip down to the "Custom Certificates" section.  Probably a good idea to reboot first though to undo any manual edits.

 

If that doesn't help, please provide your diagnostics.zip file (from Tools -> Diagnostics) and tell me what url you want to use to access the server.

[ReddotCleaner]

2 hours ago, ljm42 said:

 

I don't see a diagnostics.zip file, but I'll take a guess... you are trying to use your own custom certificate, but the url on the certificate does not match the servername.localTLD that the server is configured to use. This is causing Unraid to delete your certificate and create a self-signed one that matches Unraid's settings.

 

You have two options, either provide a certificate that matches the server's settings, or change the server's settings to match the certificate you want to use.

 

Take a look here:

  https://wiki.unraid.net/Manual/Security#Securing_webGui_connections_.28SSL.29

Start with the "A few details before we begin" section so you know how things are defined, then skip down to the "Custom Certificates" section.  Probably a good idea to reboot first though to undo any manual edits.

 

If that doesn't help, please provide your diagnostics.zip file (from Tools -> Diagnostics) and tell me what url you want to use to access the server.

yeah, bcz my cert is not a wildcard cert. Thats why it dosent match the servername.localTLD. I will try get a wildcard cert to test it and get back to u.

[ljm42]

23 minutes ago, ReddotCleaner said:

yeah, bcz my cert is not a wildcard cert. Thats why it dosent match the servername.localTLD. I will try get a wildcard cert to test it and get back to u.

 

There is no need for it to be a wildcard cert. However, the subject of the cert must match servername.localTLD

 

If you have an existing cert you want to use, tell me what the "subject" of the cert is and I will tell you how to set your server name and localTLD

[ReddotCleaner]

2 hours ago, ljm42 said:

 

There is no need for it to be a wildcard cert. However, the subject of the cert must match servername.localTLD

 

If you have an existing cert you want to use, tell me what the "subject" of the cert is and I will tell you how to set your server name and localTLD

i dont really understand what subject is. but my cert is for 'reddotcleaner.tk'  server name is 'unraid'.

[ljm42]

OK if you want this to be your url:
  https://unraid.reddotcleaner.tk

First, make sure that there is a DNS entry which resolves unraid.reddotcleaner.tk to your server's ip address.

Then per the instructions here:

  https://wiki.unraid.net/Manual/Security#Securing_webGui_connections_.28SSL.29

you'll want to:

• Go to Settings -> Management Access and set "Use SSL/TLS" to "No" if it isn't already, then Apply.
• Go to Settings -> Identification and set the server name to "unraid".
• Go to Settings -> Management Access and set "Local TLD" to "reddotcleaner.tk"
• Obtain a certificate for this specific url: "unraid.reddotcleaner.tk" and place it on your flash drive as config/ssl/certs/unraid_unraid_bundle.pem  ( Note that capitalization matters when naming the file, so if you name the server "Unraid" then the filename should be Unraid_unraid_bundle.pem )
• Go to Settings -> Management Access and set "Use SSL/TLS" to "Yes" and Apply to use the certificate.
• If the certificate gets deleted and replaced when you enable SSL, that means the url the certificate was created for does not match exactly "unraid.reddotcleaner.tk". You'll need to get a new certificate for this exact url.  Note that "Subject Alternative Name" is ignored, only the main "Subject" field of the certificate is used.

[fluisterben]

This is still not working. I have just updated to 6.10.3 and the ssl is not loaded in nginx.

Here's what I run after booting;

#!/bin/bash

cp -af /mnt/user/nxt/live/somename.org/fullchain.pem /boot/config/ssl/certs/somename_unraid_bundle.pem
cat /mnt/user/nxt/live/somename.org/privkey.pem >> /boot/config/ssl/certs/somename_unraid_bundle.pem

/etc/rc.d/rc.nginx reload
/etc/rc.d/rc.php-fpm reload

 

This runs without errors, it starts nginx, but after this (and after rebooting the entire server as well), it still does not load the new cert.

Honestly, this entire concept is broken in your config, as far as I would say. People create their own certs now, and there's no way I can properly work that into unraid's OS setup.

Please allow us to have it use a custom location for the cert(s), and NOT recreate one every time we update the OS. Simply allow that to skip all your ssl coding, and let the user put in a replacement for the /etc/nginx/conf.d/servers.conf ssl path in the UI and you'd be done with this time-wasting support on ssl certs..

Edited June 18, 2022 by fluisterben

[ljm42]

2 hours ago, fluisterben said:

This runs without errors, it starts nginx, but after this (and after rebooting the entire server as well), it still does not load the new cert.

Honestly, this entire concept is broken in your config, as far as I would say. People create their own certs now, and there's no way I can properly work that into unraid's OS setup.

 

I am extremely confused by your comments, so let's start from the top and see if we can figure this out.

 

Unraid 6.10 has vastly improved SSL support in order to provide better security. You are welcome to provide your own cert, but you must make sure the cert matches the server's settings. Once your settings match the cert, simply drop the file on your flash drive and change "Use SSL/TLS" from No to Yes. As long as the setting match the cert it will remain on the flash drive.  No need to do write any scripts that do anything special at boot.

 

If the system is deleting your certificate, then your settings do not match the certificate. 

 

Specifically, what is your servername and what is your Local TLD? 

The server's url is expected to be https://servername.localTLD . That exact url (not including any ports) must be specified in your certificate. As long as that is the case, the certificate will not be deleted.

 

For more details see the wiki:

  https://wiki.unraid.net/Manual/Security#Securing_webGui_connections_.28SSL.29

 

(actually, there is one thing I need to update in the wiki.  Previously, Unraid would only look for the url in the the "Subject" of the cert, starting with 6.10.3 you can also include the url in one of the Subject Alternative Names.)

 

Just to mention... if you are changing the certificate you must make sure that you have a single tab open to the server. Your browser will get confused if you have one tab open using the old cert and then you try to use a new cert in a second tab.

 

Normally these types of things are done using the webgui, but if you do ever run `rc.nginx reload` in order to change the cert, be sure that you do it from a real SSH application and not the web terminal. If you change the certificate while the web terminal is open you'll have the same issue as mentioned above about multiple tabs.

[0rca]

I hope it is ok to post in this thread, since it is the only one covering wildcard certs and I don't have to write down too much context.

Our certificate is has a Subject as well as a Common Name of "*.domain.com" and the Subject Alternative Names are "*.domain.com domain.com".

We are using the same cert on many servers and I can verify the above with SSLLabs.

 

It is not recognized as valid in Unraid though - Certificate URL: Certificate not valid for tower.domain.com even though the cert itself is ok:

Certificate issuer: C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA

It is replaced again and again, whatever I try. Can someone help us out?

 

[ljm42]

On 7/17/2023 at 6:12 AM, 0rca said:

I hope it is ok to post in this thread, since it is the only one covering wildcard certs and I don't have to write down too much context.

Our certificate is has a Subject as well as a Common Name of "*.domain.com" and the Subject Alternative Names are "*.domain.com domain.com".

We are using the same cert on many servers and I can verify the above with SSLLabs.

 

It is not recognized as valid in Unraid though - Certificate URL: Certificate not valid for tower.domain.com even though the cert itself is ok:

Certificate issuer: C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA

It is replaced again and again, whatever I try. Can someone help us out?

 

 

Have you set the server's name and localTLD to match the cert? Read the intro here https://docs.unraid.net/unraid-os/manual/security#securing-webgui-connections-ssl then scroll down to "Custom Certificates"

 

If you still have questions please post the diagnostics.zip (from Tools -> Diagnostics)