[Support] Linuxserver.io - SWAG - Secure Web Application Gateway (Nginx/PHP/Certbot/Fail2ban)


Recommended Posts

Ok, I think I found the problem. It's a problem with the way certbot uses the nsone api apparently. It tries to set two records and the first works but the second doesn't, because nsone's api has to be called a different way I guess.

Here's the bug for certbot: https://github.com/certbot/certbot/issues/5735

I guess I have to wait for this bug to be fixed if I want to keep using nsone with certbot/letsencrypt.

I tried just running certbot from the command line to get a wildcard cert and I got the same error. The reason the error says TXT record is wrong is because it's looking for the second record that was set and it was never set, and it's just reading the first one. (It does successfully delete the TXT record it had set so nothing extra is left in my DNS.)

 

I know cloudflare is free and supports wildcard certs, but when I last looked you couldn't set a wildcard A DNS record (e.g., <anything>.<domain>.<tld> points to my server) at least in the free version. Has this changed?

 

Thanks for your help you guys.

 

Aptalca, your work has been making my life so much better for months. Appreciate it.

Link to comment

Hello all!

 

I have set up the letsencrypt docker, but when I try to access it via a web-browser on the local network I get an ERR_CONNECTION_REFUSED.

 

I am in the process of setting up Nextcloud using this guide.

The guide does not include the setup for letsencrypt, it assumes you have that already, so I am following this guide for a dynamic DNS letsencrypt reverse proxy (I think that's the correct terminology).

It uses DuckDNS to do the dynamic DNS.


If I run the following on the unRAID command line: tail -20 /var/log/nginx/error.log

I get the following:

2018/03/28 22:40:53 [error] 4246#4246: *41724 user "XXXXX" was not found in "/etc/nginx/htpasswd", client: 192.XXX.XXX.XXX, server: , request: "GET /Main HTTP/1.1", host: "192.XXX.XXX.YYY"

I dug around for an htpasswd file to look at, but couldn't find one, even when using Krusader to search all of /mnt.

 

According to the guide, I should be able to navigate to the letsencrypt installtion via server:81 and should see the following:

1.png


Attached is a log for the letsencrypt docker, a this is its run command:

root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='LETSENCRYPT-ReverseProxy-' --net='bridge' --privileged=true -e TZ="America/Los_Angeles" -e HOST_OS="unRAID" -e 'EMAIL'='[email protected]' -e 'URL'='duckdns.org' -e 'SUBDOMAINS'='quillnextcloud' -e 'ONLY_SUBDOMAINS'='true' -e 'DHLEVEL'='2048' -e 'VALIDATION'='http' -e 'DNSPLUGIN'='' -e 'PUID'='99' -e 'PGID'='100' -p '81:80/tcp' -p '444:443/tcp' -v '/mnt/user/appdata/letsencrypt':'/config':'rw' 'linuxserver/letsencrypt'
35d44450e249fec834b7849d1324df5a393bdc2d63e40ce3ee166d541094ed64

 

Can anyone provide some guidance?

letsencrypt log.txt

Link to comment
Failed authorization procedure. *****nextcloud.duckdns.org (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://*****nextcloud.duckdns.org/.well-known/acme-challenge/pBh4tVjEdh2LbJ1O-PTVjE5gCs14g91uSp5JocABmm8: Timeout

IMPORTANT NOTES:
- The following errors were reported by the server:

Domain: *****nextcloud.duckdns.org
Type: connection
Detail: Fetching
http://*****nextcloud.duckdns.org/.well-known/acme-challenge/pBh4tVjEdh2LbJ1O-PTVjE5gCs14g91uSp5JocABmm8:
Timeout

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.
ERROR: Cert does not exist! Please see the validation error above. The issue may be due to incorrect dns or port forwarding settings. Please fix your settings and recreate the container

That's why it's not working.  Sure your router ports are setup correctly?

Edited by CHBMB
Link to comment
42 minutes ago, Ruthalas said:

Hello all!

 

I have set up the letsencrypt docker, but when I try to access it via a web-browser on the local network I get an ERR_CONNECTION_REFUSED.

 

I am in the process of setting up Nextcloud using this guide.

The guide does not include the setup for letsencrypt, it assumes you have that already, so I am following this guide for a dynamic DNS letsencrypt reverse proxy (I think that's the correct terminology).

It uses DuckDNS to do the dynamic DNS.


If I run the following on the unRAID command line: tail -20 /var/log/nginx/error.log

I get the following:


2018/03/28 22:40:53 [error] 4246#4246: *41724 user "XXXXX" was not found in "/etc/nginx/htpasswd", client: 192.XXX.XXX.XXX, server: , request: "GET /Main HTTP/1.1", host: "192.XXX.XXX.YYY"

I dug around for an htpasswd file to look at, but couldn't find one, even when using Krusader to search all of /mnt.

 

According to the guide, I should be able to navigate to the letsencrypt installtion via server:81 and should see the following:

1.png


Attached is a log for the letsencrypt docker, a this is its run command:


root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='LETSENCRYPT-ReverseProxy-' --net='bridge' --privileged=true -e TZ="America/Los_Angeles" -e HOST_OS="unRAID" -e 'EMAIL'='[email protected]' -e 'URL'='duckdns.org' -e 'SUBDOMAINS'='quillnextcloud' -e 'ONLY_SUBDOMAINS'='true' -e 'DHLEVEL'='2048' -e 'VALIDATION'='http' -e 'DNSPLUGIN'='' -e 'PUID'='99' -e 'PGID'='100' -p '81:80/tcp' -p '444:443/tcp' -v '/mnt/user/appdata/letsencrypt':'/config':'rw' 'linuxserver/letsencrypt'
35d44450e249fec834b7849d1324df5a393bdc2d63e40ce3ee166d541094ed64

 

Can anyone provide some guidance?

letsencrypt log.txt

 

"If I run the following on the unRAID command line: tail -20 /var/log/nginx/error.log"

That is not the log of this container's nginx. That is the unraid web interface's log. The container app logs are in your /config folder

 

"According to the guide, I should be able to navigate to the letsencrypt installtion via server:81 and should see the following:"

You need to visit https://yoursubdomain.duckdns.org to visit the interface since that is the address your cert covers. If your router blocks access to it from your lan due to nat loopback, try it from a cell phone over cellular connection to test.

 

If you're still confused, you should post in the thread for the external guide.

Link to comment
39 minutes ago, CHBMB said:

That's why it's not working.  Sure your router ports are setup correctly?

 

My port forwarding looks like this:

ports.PNG.d47300b9e634b927a423077a5dd9a152.PNG

I believe that is appropriate for the docker, as the docker is configured like so:

ports2.PNG.988c51ca12d091e84cb0989b73142bef.PNG

Does anything seem to be awry there?

 

14 minutes ago, aptalca said:

 

"If I run the following on the unRAID command line: tail -20 /var/log/nginx/error.log"

That is not the log of this container's nginx. That is the unraid web interface's log. The container app logs are in your /config folder

 

"According to the guide, I should be able to navigate to the letsencrypt installtion via server:81 and should see the following:"

You need to visit https://yoursubdomain.duckdns.org to visit the interface since that is the address your cert covers. If your router blocks access to it from your lan due to nat loopback, try it from a cell phone over cellular connection to test.

 

If you're still confused, you should post in the thread for the external guide.

Thanks for the clarification!
The docker-specific log is attached to the first post.

When I try from outside the local network I get ERR_CONNECTION_RESET.

 

I belive the port forwarding is appropriately configured (see above). 

I will post in the external help next if you feel that is a better option.

Edited by Ruthalas
Link to comment

Interesting. This router does have that functionality. (Good call!)

 

I have enabled it and cannot even access the router's webUI from the new asus domain.

 

I will move to the external access thread. Can you direct me to it? I searched around and couldn't find an official thread for that.

Link to comment
3 minutes ago, Ruthalas said:

I will move to the external access thread. Can you direct me to it? I searched around and couldn't find an official thread for that.

 

Turn off the Asuscomm domain functionality.

 

Not sure what thread you're talking about to be honest?

Link to comment

The asuscomm functionality was off to start with. I enabled it to see if it would change anything.

I have turned it back off.

 

Sorry, it was the other fellow who recommended I go elsewhere:

1 hour ago, aptalca said:

If you're still confused, you should post in the thread for the external guide.

 

Edit: Ah! My phone had defaulted to my work wifi, rather than 4G as I thought.

On 4G I once again receive the same 'connection refused' error. (Rather than a 'connection reset' error.)
I get connection refused when I access via local network or via XXX.duckdns.org

Edited by Ruthalas
Link to comment
3 minutes ago, jonathanm said:

To which web server? Docker or router? Who is your ISP? Are you sure they don't block port 80?

 

When requesting the page from outside the local network, the prefix 'https://' must be used.

 

Comcast is my ISP.

 

It does not appear that they are blocking port 80, as I can now access the base letsencrypt page externally.

Link to comment
20 minutes ago, Ruthalas said:

 

When requesting the page from outside the local network, the prefix 'https://' must be used.

 

Comcast is my ISP.

 

It does not appear that they are blocking port 80, as I can now access the base letsencrypt page externally.

Your initial logs showed no certificate was generated.  Seems that has been resolved.  It's working fine now, out the box, nginx isn't configured to respond on port 80 (http)

A lot of this could have been avoided by looking at the logs.  Would have helped remove a lot of guesswork, but at least you got it working.

Link to comment

I am slowly learning which logs are where, and how to read them. 

 

(Just as a side note, I've been reading through this thread, and several people have tried to mix in parts of this other guy's tutorial:

https://cyanlabs.net/tutorials/the-complete-unraid-reverse-proxy-duck-dns-dynamic-dns-and-letsencrypt-guide/

The reason for that is that you don't cover setting up letsencrypt in your (otherwise excellent) guide, and that guide is one of the results when searching for setting up letsencrypt.)

 

I am currently untangling the same mess via some older posts.

Link to comment
1 hour ago, Ruthalas said:

I am slowly learning which logs are where, and how to read them. 

 

(Just as a side note, I've been reading through this thread, and several people have tried to mix in parts of this other guy's tutorial:

https://cyanlabs.net/tutorials/the-complete-unraid-reverse-proxy-duck-dns-dynamic-dns-and-letsencrypt-guide/

The reason for that is that you don't cover setting up letsencrypt in your (otherwise excellent) guide, and that guide is one of the results when searching for setting up letsencrypt.)

 

I am currently untangling the same mess via some older posts.

 

You should have followed my directions above more closely. I specifically told you to use the https address.

 

The default site config is only listening over port 443 (https). But there are instructions in there to enable listening on port 80.

 

This image requires some knowledge on how to set up nginx. The container sets up the webserver and its environment, but the user has to customize the config files to serve their content. 

Link to comment

Is there currently any kind of tutorial or work-around for making this work with an ISP that blocks all traffic on port 80 inbound?

Since they disabled TLS-SNI validation and now require either HTTP (only works with port 80 open) I'm stuck.

I did look into DNS validation but having that update dynamically seems non-trivial.

 

Anyone? I miss using nextcloud as I was able to get at my whole server, securely, from anywhere in the world.

Link to comment
30 minutes ago, commander-flatus said:

Is there currently any kind of tutorial or work-around for making this work with an ISP that blocks all traffic on port 80 inbound?

Since they disabled TLS-SNI validation and now require either HTTP (only works with port 80 open) I'm stuck.

I did look into DNS validation but having that update dynamically seems non-trivial.

 

Anyone? I miss using nextcloud as I was able to get at my whole server, securely, from anywhere in the world.

 

If you have your own domain name, get a free cloudflare account, point your nameservers from your domain name provider to cloudflare and set cloudflare to "dns only". 

 

Then in the config folder, edit the cloudflare.ini file and enter your email and global api key. DNS validation will take care of everything automatically.

 

On cloudflare it is super easy to create new aliases for subdomains.

 

Your cert can even cover all subdomains via a wildcard cert.

Link to comment
 
If you have your own domain name, get a free cloudflare account, point your nameservers from your domain name provider to cloudflare and set cloudflare to "dns only". 
 
Then in the config folder, edit the cloudflare.ini file and enter your email and global api key. DNS validation will take care of everything automatically.
 
On cloudflare it is super easy to create new aliases for subdomains.
 
Your cert can even cover all subdomains via a wildcard cert.


Thanks. Appreciate your rapid response. Everything works now.

In case anyone comes across this response please note that you have to put “cloudflare” in the DNS plugin box for the docker.


Sent from my iPhone using Tapatalk
Link to comment

I am having issues with my install.  It was working a few months ago, but then it was turned off for awhile and I upgraded the OS twice and updated the Docker.  It was no longer working,  Everything seems to be set up correctly, and I never changed any of the settings.  I tried recreating the docker and removing the appdata folder.  


 

 

le1.PNG

le3.PNG

le4.PNG

Edited by clause
Link to comment
On 3/27/2018 at 8:03 PM, fivestones said:

I know cloudflare is free and supports wildcard certs, but when I last looked you couldn't set a wildcard A DNS record (e.g., <anything>.<domain>.<tld> points to my server) at least in the free version. Has this changed?

 

I went back and looked at cloudflare again, and while I'm pretty sure that a few months ago when I was trying it it wouldn't let me use *.mydomain.com to make a DNS A record, now it does. They say that using a wildcard DNS like this will make the wildcard subdomains not be protected by the cloudflare network (unless you pay for the enterprise version), but it will still point to your server as intended.

 

So I set it up for my domain, made the wildcard subdomain in cloudflare, and then set the letsencrypt docker to make a wildcard cert, and it all works! Now I can go to any random subdomain random.mydomain.com and it points to mydomain.com if nothing is specified in letsencrypt config/nginx/site-confs/default. Or if I specify something in that file random.mydomain.com can point to a particular port on my server like ghost or plex.

 

I'm so excited to see it all working! Thanks for the tip on cloudflare.

Link to comment
8 hours ago, fivestones said:

 

I went back and looked at cloudflare again, and while I'm pretty sure that a few months ago when I was trying it it wouldn't let me use *.mydomain.com to make a DNS A record, now it does. They say that using a wildcard DNS like this will make the wildcard subdomains not be protected by the cloudflare network (unless you pay for the enterprise version), but it will still point to your server as intended.

 

So I set it up for my domain, made the wildcard subdomain in cloudflare, and then set the letsencrypt docker to make a wildcard cert, and it all works! Now I can go to any random subdomain random.mydomain.com and it points to mydomain.com if nothing is specified in letsencrypt config/nginx/site-confs/default. Or if I specify something in that file random.mydomain.com can point to a particular port on my server like ghost or plex.

 

I'm so excited to see it all working! Thanks for the tip on cloudflare.

 

Glad to hear. 

 

Just so you know, in the nginx site config, you can define a default_server directive, one for each listening port and any request that doesn't match a specific server block will go to the defined default

Link to comment

@clause I am having the exact same issue recently. Used to work just fine, now I get that error.

 

For some reason, sub-domains is activated and stuck upon updating container. I had to completely remove the option, and re-remove it after each update, for cert generation to work.

Edited by d2dyno
Link to comment
2 hours ago, d2dyno said:

@clause I am having the exact same issue recently. Used to work just fine, now I get that error.

 

For some reason, sub-domains is activated and stuck upon updating container. I had to completely remove the option, and re-remove it after each update, for cert generation to work.

 

What do you mean by sub-domains activated and stuck? 

 

Did you forget to forward port 80 on your router? 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.