[Support] Linuxserver.io - SWAG - Secure Web Application Gateway (Nginx/PHP/Certbot/Fail2ban)


Recommended Posts

20 hours ago, Anthop said:

I'm working on my first unRAID setup, but am stuck getting the Let's Encrypt docker to work properly.  I've been following a few tutorials (namely the CyanLabs), but I seem to be missing a step or something.  I'm using the Let's Encrypt docker provided by linuxserver.  Please redirect me if this is not the correct place to post.

 

I get the following errors in the Docker log:


Performing the following challenges:
http-01 challenge for skybox.mydomain.org
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. skybox.mydomain.org (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching https://Paravati.local:443/.well-known/acme-challenge/T5M9XzFvy-wl_UkG9dspNJ6oUcg-7DVm6X8YnBEYt8Y: Error getting validation data
IMPORTANT NOTES:
- The following errors were reported by the server:

Domain: skybox.mydomain.org
Type: connection
Detail: Fetching
https://Paravati.local:443/.well-known/acme-challenge/T5M9XzFvy-wl_UkG9dspNJ6oUcg-7DVm6X8YnBEYt8Y:
Error getting validation data

It looks like the issue is two fold:

  • It looks like Nginx is redirecting my requests to Paravati.local:443, for some reason.  I'm not sure if this is the intended behavior.  I confirmed this by issuing a GET command via curl to https://skybox.mydomain.org over the Internet (not the same LAN) and getting back a 302 response that takes me to https://paravati.local:443.
  • Paravati.local:443 is the unRAID GUI, and it doesn't appear that the ACME challenge files/keys are actually hosted here.

anthop@NATSUKI-3:~$ curl -kv https://skybox.mydomain.org
* Rebuilt URL to: https://skybox.mydomain.org/
*   Trying XXX.XXX.XXX.XXX...
* TCP_NODELAY set
* Connected to skybox.mydomain.org (XXX.XXX.XXX.XXX) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /home/anthop/anaconda3/ssl/cacert.pem
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: O=Self-signed; OU=unRAID; CN=Paravati.local
*  start date: May 29 06:58:56 2018 GMT
*  expire date: May 26 06:58:56 2028 GMT
*  issuer: O=Self-signed; OU=unRAID; CN=Paravati.local
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET / HTTP/1.1
> Host: skybox.mydomain.org
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Fri, 01 Jun 2018 00:59:37 GMT
< Content-Type: text/html
< Content-Length: 154
< Connection: keep-alive
< Location: Main
<
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Curl_http_done: called premature == 0
* Connection #0 to host skybox.mydomain.org left intact

I'm running unRAID 6.5.2.  Paravati is the name of the unRAID tower.  I have the Let's Encrypt docker running on ports 1080 and 1443 (and port forwarding is enabled on the router to those locations).  I'm not entirely clear if I'm supposed to be touching the Nginx config file, but none of the tutorials have mentioned needing to do this, so I'm wondering if I'm missing something obvious.

Any help would be greatly appreciated!

 

Nginx is not doing anything wrong, nginx (in this container) is not even up during validation.

 

What you're seeing here is that when the letsencrypt server tries to connect to your domain on port 443 to validate, it gets a response from the unraid server's gui instead of the webserver in the container. It means your port forwarding is not correct.

 

 

Link to comment
On 5/28/2018 at 10:29 AM, dalben said:

I have an issue where when this docker stops and restarts (nightly backup, etc) the port forwarding from my router (USG using the LSIO Unifi Docker) no longer works.  I have to restart the Unifi container/controller for the port forwarding to work.

 

Does anyone else have this combo of containers and experiencing the same problem ?

 

No one else uses this combo of containers and seeing this problem?  Myself and one other than I have seen so far.  I'm surprised as they are both popular containers.

Link to comment
10 hours ago, CHBMB said:

@aptalca

 

EDIT: Also bear in mind Unraid's webui is running on Nginx as well.

 

 

You're right, this was the piece of information I was missing; assumed since Nginx was showing up, I was forwarded to the correct spot.  Fixed this, and now at least the Let's Encrypt is working.  Thanks!

Link to comment
11 hours ago, dalben said:

 

No one else uses this combo of containers and seeing this problem?  Myself and one other than I have seen so far.  I'm surprised as they are both popular containers.

 

I use unifi, but only for APs. The router is pfsense

Link to comment

Hi,

 

I fiddled some time again with the LE Docker.

One thing/wish is that I can use DNS Validation via ACME-Challenge/TXT Record. Websites like zerossl.com can give me the option to generate a certificate but it would still be more convenient to integrate this one in this docker to run every 3 months. any possibility that this one will make it into your LE Docker?

Link to comment

I have a DuckDNS hostname that supports DNS txt records for Let's Encrypt DNS validation, as my ISP blocks WAN ports 80 and 443.

 

I would like to use this LSIO Let's Encrypt docker image but I could not figure out how to use it with DNS validation for DuckDNS.

 

As a workaround, I'm using the LSIO Nginx-only docker image with the tweaks outlined here to make it as close as possible to the LE image and I installed acme.sh on the host which supports DNS validation for DuckDNS and creates the correct certs for the Nginx docker to use.

 

However, like the LE image, I would much prefer to use a single docker image to perform this task and keep the host as clean as possible.

 

The Let's Encrypt docker looks ideal, is it possible to use DuckDNS DNS validation with this image? It appears to support DNS validation for other providers, but I can't see a way to use it with DuckDNS.

  • Like 1
Link to comment
4 hours ago, Aluavin said:

Hi,

 

I fiddled some time again with the LE Docker.

One thing/wish is that I can use DNS Validation via ACME-Challenge/TXT Record. Websites like zerossl.com can give me the option to generate a certificate but it would still be more convenient to integrate this one in this docker to run every 3 months. any possibility that this one will make it into your LE Docker?

 

This container already supports dns validation

Link to comment
4 hours ago, Conners said:

I have a DuckDNS hostname that supports DNS txt records for Let's Encrypt DNS validation, as my ISP blocks WAN ports 80 and 443.

 

I would like to use this LSIO Let's Encrypt docker image but I could not figure out how to use it with DNS validation for DuckDNS.

 

As a workaround, I'm using the LSIO Nginx-only docker image with the tweaks outlined here to make it as close as possible to the LE image and I installed acme.sh on the host which supports DNS validation for DuckDNS and creates the correct certs for the Nginx docker to use.

 

However, like the LE image, I would much prefer to use a single docker image to perform this task and keep the host as clean as possible.

 

The Let's Encrypt docker looks ideal, is it possible to use DuckDNS DNS validation with this image? It appears to support DNS validation for other providers, but I can't see a way to use it with DuckDNS.

 

We support the official dns plugins maintained by letsencrypt (certbot) and unfortunately there isn't one for duckdns yet. 

 

Perhaps you can submit a request to them: https://github.com/certbot/certbot

Link to comment

 

5 hours ago, aptalca said:

 

This container already supports dns validation

 

Well.it supports some of the dns providers but not a generic one. My Provider doesn't offer any api_key/secrettoken.

I'm talkling about a real dns validation (TXT Record( like ZeroSSL/LE Validation does) or CNAME (like AWS does for their certs) for answering the DNS challenge).

even  rfc2136  isn't any helpful since I can't find any documentation on my Hosters Website.

 

Link to comment
2 hours ago, Aluavin said:

 

 

Well.it supports some of the dns providers but not a generic one. My Provider doesn't offer any api_key/secrettoken.

I'm talkling about a real dns validation (TXT Record( like ZeroSSL/LE Validation does) or CNAME (like AWS does for their certs) for answering the DNS challenge).

even  rfc2136  isn't any helpful since I can't find any documentation on my Hosters Website.

 

 

They all do it through txt records. The reason api keys or tokens are needed is so letsencrypt (certbot) can create the txt records itself for automation, rather than asking the users to do it manually. 

 

You can always point your name servers to a supported free dns service like cloudflare and use their plugin. 

Link to comment

After hours of trying every possible config I have found it impossible to configure DuckDNS + LE to access dockers outside my LAN. I set up a DDNS domain, forwarded the ports on my router, configured LE, and I can access the default web page via the domain, but cannot access ANY apps via their port. Ombi or Nextcloud for example, are useless without WAN access. 

 

Have followed many 3rd party guides, but they don't work. The devs for these projects should provide tutorials, or else all their hard work is wasted.

 

Any help appreciated

Link to comment
37 minutes ago, Odessa said:

After hours of trying every possible config I have found it impossible to configure DuckDNS + LE to access dockers outside my LAN. I set up a DDNS domain, forwarded the ports on my router, configured LE, and I can access the default web page via the domain, but cannot access ANY apps via their port. Ombi or Nextcloud for example, are useless without WAN access. 

 

Have followed many 3rd party guides, but they don't work. The devs for these projects should provide tutorials, or else all their hard work is wasted.

 

Any help appreciated

 

The devs of these projects have provided all the information one needs to set this up. There are even preset reverse proxy configs for common apps, most of which only require a simple rename of a file to activate (including both ombi and nextcloud):  https://github.com/linuxserver/docker-letsencrypt/blob/master/README.md#setting-up-the-application

Edited by aptalca
Link to comment
46 minutes ago, Odessa said:

@aptalca Thank you, I will read through this documentation, although it does not appear to be a step by step guide. For example I have no idea what this  means and no further instructions are provided: "

  • --cap-add=NET_ADMIN is required for fail2ban to modify iptables"

 

I'm sorry but we're all adults here and I can't hold your hand. There are 3 references to --cap-add on that page and if you read all three, you'll realize what it means. There is also google search that provides a lot of answers to common questions.

 

Everybody's environment and needs are different. A step by step guide may work for someone in a specific situation, but not another person. Some people like subfolder, some like subdomain methods for reverse proxy, some want to password protect, some don't, some want to use organizr as their homepage, some prefer heimdall, etc. 

 

We provide the tools and the instructions on how to use them. You need to put in some effort to try and understand how things fit together rather than blindly follow some guide that may or may not work for what you're trying to achieve. 

 

We already made reverse proxy as easy as possible with the preset configs. I honestly don't know what a step by step guide would add since you already have letsencrypt set up and can access the homepage. That's usually the hard part due to port forwarding. 

Link to comment

@aptalca Tbh I have spent about 6 hours trying to get LE to work before posting here. This is extremely difficult without complete instructions. If you google the issue you will find many people trying unsuccessfully to configure LE / Duckdns / Ombi etc, with multiple different solutions, unfortunately none of which worked for me. 

 

I will pay $50 to anyone who can provide working instructions. 

Edited by Odessa
Link to comment
1 hour ago, Odessa said:

@aptalca Tbh I have spent about 6 hours trying to get LE to work before posting here. This is extremely difficult without complete instructions. If you google the issue you will find many people trying unsuccessfully to configure LE / Duckdns / Ombi etc, with multiple different solutions, unfortunately none of which worked for me. 

 

I will pay $50 to anyone who can provide working instructions. 

 

There are complete documentation on the internet if you make a search. The best one is the nginx documentation. 

Our github for LE also holds the information to get LE running. 

To blame us and the app makers for your inability to read and understand how to get it running, I find rude! 

Link to comment
[mention=7767]aptalca[/mention] Tbh I have spent about 6 hours trying to get LE to work before posting here. This is extremely difficult without complete instructions. If you google the issue you will find many people trying unsuccessfully to configure LE / Duckdns / Ombi etc, with multiple different solutions, unfortunately none of which worked for me. 
 
I will pay $50 to anyone who can provide working instructions. 
6 hours is not a huge amount of time, not in the grand scheme of things, and it's a drop in the ocean compared to how long@aptalca has spent optimising this process to make it as easy as possible.

But one thing to ponder, this is a community, but it presents a skewed view of things, only those with a problem post, those that get it working first time, do not.

As this is an open source project, anybody can write a tutorial, not just the devs, so maybe when you've finished, you might like to write it up to help others and act as a personal aide memoire to yourself. We are quite happy to publish decent articles on our website.

Please remember that we all do this in our spare time, and it does tend to irk a little when people suggest we do even more work because their time is so valuable.

Docker and Unraid already take the vast majority of the leg work away from server admin and bring it to the masses in my opinion and it's wonderful, but their still needs to be some personal learning that goes on, this is a fully fledged webserver, you're going to be responsible to maintain it and keep it secure, so a degree of understanding about the process is required.

Things break, that's computing, to all intents and purposes you're the sysadmin of this system.
Link to comment
18 hours ago, aptalca said:

 

They all do it through txt records. The reason api keys or tokens are needed is so letsencrypt (certbot) can create the txt records itself for automation, rather than asking the users to do it manually. 

 

You can always point your name servers to a supported free dns service like cloudflare and use their plugin. 

Of course, I can do that, but tbh a workaround is not that satisfying.

That's why I hand in a wish/feature request covering DNS validation manual mode. This would require more variables (acme challenge key)

Unfortunately, I am not able to put a decent amount of time into digging into certain dockers (either LE or ZeroSSL) to take care of it.

Link to comment
[mention=63089]saarg[/mention] [mention=6219]CHBMB[/mention] I do not blame the developers, I just need more specific documentation. If I can't figure this out by following multiple guides, I know many others can't either. It's reasonable to request help. Again, I will pay for a helpful response that solves the issue.
Link me to where you've posted your logs and docker run command (look in the docker FAQ for how to get those)
Link to comment
5 hours ago, Aluavin said:

Of course, I can do that, but tbh a workaround is not that satisfying.

That's why I hand in a wish/feature request covering DNS validation manual mode. This would require more variables (acme challenge key)

Unfortunately, I am not able to put a decent amount of time into digging into certain dockers (either LE or ZeroSSL) to take care of it.

 

That's just the problem. Acme challenge key is generated on the fly by letsencrypt servers. So you can't really do that with a variable. Manual validation process goes like this:

1) user types certbot to start process (can include variables like url, email, validation type, etc) 

2) Letsencrypt generates and outputs a key, starts waiting

3) User goes on their dns provider's interface and creates a new txt record with that key

4) User tells certbot they created it and it propagated

5) Letsencrypt checks the dns records to confirm txt record and provides user the cert

 

As you can see, the above process requires user interaction, which presents two problems, it is hard to do in docker start up, and it cannot be automated. You have to go through the manual process every 90 days. 

 

The dns plugins automate the process by taking care of steps 3 and 4,but it requires a specific login for each dns provider as their apis for updating records are vastly different. 

 

If you're willing to do that manual process, you can use our nginx image instead, and run certbot in it manually. 

 

As for the feature request, please submit it here: 

https://github.com/certbot/certbot

 

PS. I personally don't think using cloudflare is a workaround. It delivers about 10% of the internet (cdn, proxy) and I bet it is a lot more reliable as a dns provider than any hosting company

Edited by aptalca
Link to comment
Quote

Link me to where you've posted your logs and docker run command (look in the docker FAQ for how to get those)

 

@CHBMB Thank you very much, here is my LE run command:

https://pastebin.com/sTnfSFkk

 

My LE log:

https://pastebin.com/9DawkVww

[personal info obfuscated]

 

I see the error regarding port in the log, but 443 does not show as used in my docker mappings. Note I cannot currently access the default web page from outside, I get "ERR_CONNECTION_REFUSED" in browser.

 

Edited by Odessa
Link to comment
1 hour ago, Odessa said:

 

@CHBMB Thank you very much, here is my LE run command:

https://pastebin.com/sTnfSFkk

 

My LE log:

https://pastebin.com/9DawkVww

[personal info obfuscated]

 

I see the error regarding port in the log, but 443 does not show as used in my docker mappings. Note I cannot currently access the default web page from outside, I get "ERR_CONNECTION_REFUSED" in browser.

 

 

 

nginx: [emerg] a duplicate default server for 0.0.0.0:443 in /config/nginx/site-confs/default - Copy:13

That needs fixing.  Look at line 13 in that file

 

You now need to post your default file.

Edited by CHBMB
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.