[Support] Linuxserver.io - SWAG - Secure Web Application Gateway (Nginx/PHP/Certbot/Fail2ban)


Recommended Posts

8 hours ago, lusitopp said:

im new to linux systems, but im eager to learn, this is the output i get

 


root@0d9237f2d370:/config/www/wordpress/wp-content# ls -la
total 8
drwxr-xr-x 1 root root   67 Jul  2 07:59 .
drwxr-xr-x 1 root root 4096 Jul  2 07:50 ..
-rw-r--r-- 1 root root   28 Jul  1 17:36 index.php
drwxr-xr-x 1 root root   80 Jul  2 18:00 plugins
drwxr-xr-x 1 root root  108 Jul  2 18:00 themes
drwxr-xr-x 1 abc  abc    54 Jul  1 18:03 uploads

 

Restart the container and it should fix the permissions

Link to comment

Just started using this instead of having my server handle the SSL certificate directly. Now that this is running, my server's access log shows all requests as having come from the reverse proxy. Is there an access log on the reverse proxy where I can see the outside addresses using the server?

Link to comment
3 hours ago, draeh said:

Just started using this instead of having my server handle the SSL certificate directly. Now that this is running, my server's access log shows all requests as having come from the reverse proxy. Is there an access log on the reverse proxy where I can see the outside addresses using the server?

You need to provide more context. Are you reverse proxying the server? And by server do you mean unraid?

Link to comment
1 hour ago, aptalca said:

You need to provide more context. Are you reverse proxying the server? And by server do you mean unraid?

Sorry if I didn't make that clear.

 

I have an existing apache server that my firewall pointed to. That server managed a letsencrypt certificate. I decided to employ the letsencrypt reverse proxy docker on my unraid server to manage the certificate to make it easier to host multiple named servers and subdomains. As a first step I simply used the docker to reverse proxy the original server which is working great, but I've lost the ability to audit my server in the original way that I did. I would audit the apache access logs for undesired behavior and sometimes blacklist other domains or ips based on the addresses listed in those logs. Now the apache server's access logs only show the unraid server's ip address as the one making the requests. Is there somewhere within the reverse proxy docker where I can view a kind of access log that will show me what internet addresses are trying to access the proxy?

Edited by draeh
clarity... hopefully.
Link to comment
1 hour ago, draeh said:

Sorry if I didn't make that clear.

 

I have an existing apache server that my firewall pointed to. That server managed a letsencrypt certificate. I decided to employ the letsencrypt reverse proxy docker on my unraid server to manage the certificate to make it easier to host multiple named servers and subdomains. As a first step I simply used the docker to reverse proxy the original server which is working great, but I've lost the ability to audit my server in the original way that I did. I would audit the apache access logs for undesired behavior and sometimes blacklist other domains or ips based on the addresses listed in those logs. Now the apache server's access logs only show the unraid server's ip address as the one making the requests. Is there somewhere within the reverse proxy docker where I can view a kind of access log that will show me what internet addresses are trying to access the proxy?

Nginx logs in letsencrypt will show you all the connections. They're in the config folder.

 

Also, if you reverse proxied with all the correct headers, letsencrypt will pass the original ip in there. You may have to tell apache to trust those headers. For nginx, you do it via "real ip" module and settings. Not sure what apache needs

Link to comment

A while back my docker stopped working, dont know why. Maybe when i switch to unraid-nvidia.
 

Quote

Challenge failed for domain www.xxxxx.se
http-01 challenge for www.xxxxx.se
Cleaning up challenges
Some challenges have failed.
IMPORTANT NOTES:
- The following errors were reported by the server:

Domain: www.xxxxx.se
Type: connection
Detail: Fetching
http://www.xxxxxxx.se/.well-known/acme-challenge/fnxgQnxxxxxxxxxxxxxxxyIUdNPHG6qtmKQnReKc:
Timeout during connect (likely firewall problem)

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.
ERROR: Cert does not exist! Please see the validation error above. The issue may be due to incorrect dns or port forwarding settings. Please fix your settings and recreate the container


Portforwarding works.
I run unraid-server on eth0(192.168.1.6) and dockers on eth3 (192.168.4.xxx), eth1 & eth2 inactive.
 

 

dockersettings.PNG

dockers.PNG

Edited by capt.shitface
Link to comment
4 hours ago, capt.shitface said:

A while back my docker stopped working, dont know why. Maybe when i switch to unraid-nvidia.
 


Portforwarding works.
I run unraid-server on eth0(192.168.1.6) and dockers on eth3 (192.168.4.xxx), eth1 & eth2 inactive.
 

letsencryptsettings.PNG

dockersettings.PNG

dockers.PNG

Can you also show us the port forward?

Link to comment
2 hours ago, saarg said:

Can you also show us the port forward?

 

port forward2.PNG

 

If i change the port forward to point at Nextcloud-docker(192.168.4.4) Portscanner can se i have open ports and working.ncpf.PNG.ea90d8a1b96c595fb1b4cceb7e7b7d1c.PNG
 

If i change back to Letsencrypt i says the dont respond to ports.

ngpf.png.985f0cb7612b1273df7184450e641f8c.png

 


 

Edited by capt.shitface
Link to comment
2 hours ago, capt.shitface said:

 

port forward2.PNG

 

If i change the port forward to point at Nextcloud-docker(192.168.4.4) Portscanner can se i have open ports and working.ncpf.PNG.ea90d8a1b96c595fb1b4cceb7e7b7d1c.PNG
 

If i change back to Letsencrypt i says the dont respond to ports.

ngpf.png.985f0cb7612b1273df7184450e641f8c.png

 


 

You can't test the port forwarding to letsencrypt as nginx isn't started until a cert is created. You can use our nginx container to test.

Use this blog post for troubleshooting https://blog.linuxserver.io/2019/07/10/troubleshooting-letsencrypt-image-port-mapping-and-forwarding/

Edited by saarg
Link to comment

Hi,

I just started with unraid and have been following Spaceinvader1 on youtube.  I was attempting to create a reverse proxy and followed his instructions exactly.  However I am getting an error that the challenges have failed and that a cert does not exist.  

 

I started using my own domain name and then cname to point to duckdns.org.  At this point to troubleshoot I removed that and am just trying to  get it to work through duckdns only.  I can't find any info on how to solve this issue.  I'm not sure if it's telling me my port forwarding is not working or not.  I set it up the same as in the video but I don't have the ability to select http and https for the destination.  That is the only difference.

 

This is what I see:

rought to you by linuxserver.io
-------------------------------------

To support the app dev(s) visit:
Let's Encrypt: https://letsencrypt.org/donate/

To support LSIO projects visit:
https://www.linuxserver.io/donate/
-------------------------------------
GID/UID
-------------------------------------

User uid: 99
User gid: 100
-------------------------------------

[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-config: executing...
[cont-init.d] 20-config: exited 0.
[cont-init.d] 30-keygen: executing...
using keys found in /config/keys
[cont-init.d] 30-keygen: exited 0.
[cont-init.d] 50-config: executing...
Variables set:
PUID=99
PGID=100
TZ=America/Los_Angeles
URL=duckdns.org
SUBDOMAINS=xxxxxxxserver
EXTRA_DOMAINS=
ONLY_SUBDOMAINS=true
VALIDATION=http
DNSPLUGIN=
[email protected]
STAGING=

SUBDOMAINS entered, processing
SUBDOMAINS entered, processing
Only subdomains, no URL in cert
Sub-domains processed are: -d xxxxserver.duckdns.org
E-mail address entered: [email protected]
http validation is selected
Generating new certificate
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for xxxxxxserver.duckdns.org
Waiting for verification...
Challenge failed for domain xxxxxxserver.duckdns.org

http-01 challenge for xxxxxxxserver.duckdns.org
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
- The following errors were reported by the server:

Domain: xxxxxxxxserver.duckdns.org
Type: connection
Detail: Fetching
http://xxxxxxxserver.duckdns.org/.well-known/acme-challenge/HoaCFK90SDgQaw2iuma2cx4BtMENmLm5vgXzS39iybw:
Timeout during connect (likely firewall problem)

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.
ERROR: Cert does not exist! Please see the validation error above. The issue may be due to incorrect dns or port forwarding settings. Please fix your settings and recreate the container

[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

 

I'm not sure what else to try at this point.  I am on an ATT network using their router so I don't have a lot of control over it.

 

Ideally I would love to type in my own domain and get redirected to the Heimdall page that has all my apps easily ready to click on.  If not I can still do things through the webui's

 

thanks!

Link to comment
4 hours ago, cosmicrelish said:

Hi,

I just started with unraid and have been following Spaceinvader1 on youtube.  I was attempting to create a reverse proxy and followed his instructions exactly.  However I am getting an error that the challenges have failed and that a cert does not exist.  

 

I started using my own domain name and then cname to point to duckdns.org.  At this point to troubleshoot I removed that and am just trying to  get it to work through duckdns only.  I can't find any info on how to solve this issue.  I'm not sure if it's telling me my port forwarding is not working or not.  I set it up the same as in the video but I don't have the ability to select http and https for the destination.  That is the only difference.

 

This is what I see:


rought to you by linuxserver.io
-------------------------------------

To support the app dev(s) visit:
Let's Encrypt: https://letsencrypt.org/donate/

To support LSIO projects visit:
https://www.linuxserver.io/donate/
-------------------------------------
GID/UID
-------------------------------------

User uid: 99
User gid: 100
-------------------------------------

[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-config: executing...
[cont-init.d] 20-config: exited 0.
[cont-init.d] 30-keygen: executing...
using keys found in /config/keys
[cont-init.d] 30-keygen: exited 0.
[cont-init.d] 50-config: executing...
Variables set:
PUID=99
PGID=100
TZ=America/Los_Angeles
URL=duckdns.org
SUBDOMAINS=xxxxxxxserver
EXTRA_DOMAINS=
ONLY_SUBDOMAINS=true
VALIDATION=http
DNSPLUGIN=
[email protected]
STAGING=

SUBDOMAINS entered, processing
SUBDOMAINS entered, processing
Only subdomains, no URL in cert
Sub-domains processed are: -d xxxxserver.duckdns.org
E-mail address entered: [email protected]
http validation is selected
Generating new certificate
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for xxxxxxserver.duckdns.org
Waiting for verification...
Challenge failed for domain xxxxxxserver.duckdns.org

http-01 challenge for xxxxxxxserver.duckdns.org
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
- The following errors were reported by the server:

Domain: xxxxxxxxserver.duckdns.org
Type: connection
Detail: Fetching
http://xxxxxxxserver.duckdns.org/.well-known/acme-challenge/HoaCFK90SDgQaw2iuma2cx4BtMENmLm5vgXzS39iybw:
Timeout during connect (likely firewall problem)

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.
ERROR: Cert does not exist! Please see the validation error above. The issue may be due to incorrect dns or port forwarding settings. Please fix your settings and recreate the container

[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

 

I'm not sure what else to try at this point.  I am on an ATT network using their router so I don't have a lot of control over it.

 

Ideally I would love to type in my own domain and get redirected to the Heimdall page that has all my apps easily ready to click on.  If not I can still do things through the webui's

 

thanks!

Use the guide linked in the post above yours

Link to comment

Thanks for the link.  I read through it and tried  many different things.  Nothing is working.  I have port 80 mapped to port 180 and 443 mapped to 1443 on my router.  The forwarding appears to be working when I use another docker on those ports.  In letsencrypt docker I have the host ports set correctly and the container ports are 80 and 443.  I have checked the subdomain name and the domain names.  set only subdomains to false.  I am still receiving the same error that the challenge failed and that it thinks its a firewall problem.

Link to comment
18 hours ago, saarg said:

You have the gateway on 192.168.4.1?

Your A record and cnames are correct?

Ohhh! i found the problem!
After weeks of troubleshooting, reinstalled routers and support-tickets to my ISP i found the problem!

I use DynDNS on OPNsense to update my ip to loopia.se and my subdomain www.mydomain.se was not in there! Just the other subdomains (nextcloud, plex etc...)
I added www to the dyndns-client and now it works!

Thanks for the help, im gonna remove my pics and domain info now from the thread just to be safe :)

Again thanks for your time and help!

Link to comment
1 hour ago, capt.shitface said:

Ohhh! i found the problem!
After weeks of troubleshooting, reinstalled routers and support-tickets to my ISP i found the problem!

I use DynDNS on OPNsense to update my ip to loopia.se and my subdomain www.mydomain.se was not in there! Just the other subdomains (nextcloud, plex etc...)
I added www to the dyndns-client and now it works!

Thanks for the help, im gonna remove my pics and domain info now from the thread just to be safe :)

Again thanks for your time and help!

Glad to hear you figured it out, but it sounds like you didn't follow the troubleshooting guide properly as that test would tell you the IP was not correct for that subdomain

Link to comment

I finally got things working.  I don't know how, just came in the following day and now letsencrypt is validating fine.  However I now have a new issue.

When I go to my subdomain.domain.com I am getting a few things.

First, radarr.domain.com comes up with a bg of radarr but it just loads and loads.  Checked the webui and it is working fine.

(EDIT - not sure why but radarr just starting working but the rest remain as described)

Second, for other subs like sonarr.domain.com it pulls up a folder hierarchy with a cgi folder displayed.

Third, I just got nextcloud running and set that up the way spaceinvader taught in his video and I get sent to a 502 bad gateway page.

 

I was able to get heimdall to come up with no problems so I know it's sort of working

 

Any ideas where I went wrong?

Edited by cosmicrelish
Update
Link to comment
13 hours ago, cosmicrelish said:

I finally got things working.  I don't know how, just came in the following day and now letsencrypt is validating fine.  However I now have a new issue.

When I go to my subdomain.domain.com I am getting a few things.

First, radarr.domain.com comes up with a bg of radarr but it just loads and loads.  Checked the webui and it is working fine.

(EDIT - not sure why but radarr just starting working but the rest remain as described)

Second, for other subs like sonarr.domain.com it pulls up a folder hierarchy with a cgi folder displayed.

Third, I just got nextcloud running and set that up the way spaceinvader taught in his video and I get sent to a 502 bad gateway page.

 

I was able to get heimdall to come up with no problems so I know it's sort of working

 

Any ideas where I went wrong?

With that info, we can't say what is wrong, but you have not set it up correctly.

How did you set it up? Following space Invaders  video is not a valid answer.

Link to comment

I just set this up a couple of days ago to reverse proxy to my emby server. I am having a strange issue where responses logged for browser traffic show the traffic is using https and where the traffic is going, but for devices (like rokus, xbox, android) it just shows a dash ("-") instead.


For example, this is what is being logged now for devices:

[Devices IP Address] - - [Timestamp] "POST /emby/Sessions/Playing/Progress HTTP/2.0" 204 0 "-" "Roku/DVP-9.30 (deviceID)"

 

If I connect from a browser the log shows this:

[Devices IP Address] - - [Timestamp] "POST /emby/Sessions/Playing/Progress HTTP/2.0" 204 0 "https://[mydomain.com]/web/index.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36"

 

So my concern is, can I customize/fix the log to show the same information for devices as in browsers? I just want to verify that all of the traffic is being sent and received as encrypted https traffic.

 

Any help/insight would be greatly appreciated here! 

Link to comment
2 hours ago, lukeoslavia said:

I just set this up a couple of days ago to reverse proxy to my emby server. I am having a strange issue where responses logged for browser traffic show the traffic is using https and where the traffic is going, but for devices (like rokus, xbox, android) it just shows a dash ("-") instead.


For example, this is what is being logged now for devices:

[Devices IP Address] - - [Timestamp] "POST /emby/Sessions/Playing/Progress HTTP/2.0" 204 0 "-" "Roku/DVP-9.30 (deviceID)"

 

If I connect from a browser the log shows this:

[Devices IP Address] - - [Timestamp] "POST /emby/Sessions/Playing/Progress HTTP/2.0" 204 0 "https://[mydomain.com]/web/index.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36"

 

So my concern is, can I customize/fix the log to show the same information for devices as in browsers? I just want to verify that all of the traffic is being sent and received as encrypted https traffic.

 

Any help/insight would be greatly appreciated here! 

All traffic between the client and letsencrypt are using ssl if you are using our preset configs. They only listen for traffic on port 443.

Link to comment

I'm trying to get binhex-mineos-node webui accessible via the nginx reverse proxy. However, when I try to connect, I'm getting a 502 error.

 

I've created a mineos.subdomain.conf file based on another template that works. I've also already added the mineos subdomain to the docker variables so the cert is generated correctly. I can also ping the mineos docker from inside the lets-encrpyt one, using the docker name.

 

This is the error in the error.log file for ngnix:

2020/07/09 11:24:41 [error] 932#932: *1 upstream prematurely closed connection while reading response header from upstream, client: ##.###.##.##, server: mineos.*, request: "GET / HTTP/2.0", upstream: "http://172.18.0.12:8443/", host: "mineos.domain.tech"                                                                                                      
2020/07/09 11:24:41 [error] 932#932: *1 upstream prematurely closed connection while reading response header from upstream, client: ##.###.##.##, server: mineos.*, request: "GET /favicon.ico HTTP/2.0", upstream: "http://172.18.0.12:8443/favicon.ico", host: "mineos.domain.tech"   

 

When I look up this error I see suggestions about increasing timeouts, but this happens instantly so I don't think any timeout is happening.


Here's my config file.

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name mineos.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_mineos binhex-mineos-node;
        proxy_pass http://$upstream_mineos:8443;
    }
}

 

Anyone got suggestions where I can look. I don't think it's a .conf file problem, but I'm not sure what else would need to change.

Link to comment
On 7/9/2020 at 2:37 PM, MattTheQuaker said:

I'm trying to get binhex-mineos-node webui accessible via the nginx reverse proxy. However, when I try to connect, I'm getting a 502 error.

 

I've created a mineos.subdomain.conf file based on another template that works. I've also already added the mineos subdomain to the docker variables so the cert is generated correctly. I can also ping the mineos docker from inside the lets-encrpyt one, using the docker name.

 

This is the error in the error.log file for ngnix:


2020/07/09 11:24:41 [error] 932#932: *1 upstream prematurely closed connection while reading response header from upstream, client: ##.###.##.##, server: mineos.*, request: "GET / HTTP/2.0", upstream: "http://172.18.0.12:8443/", host: "mineos.domain.tech"                                                                                                      
2020/07/09 11:24:41 [error] 932#932: *1 upstream prematurely closed connection while reading response header from upstream, client: ##.###.##.##, server: mineos.*, request: "GET /favicon.ico HTTP/2.0", upstream: "http://172.18.0.12:8443/favicon.ico", host: "mineos.domain.tech"   

 

When I look up this error I see suggestions about increasing timeouts, but this happens instantly so I don't think any timeout is happening.


Here's my config file.


server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name mineos.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_mineos binhex-mineos-node;
        proxy_pass http://$upstream_mineos:8443;
    }
}

 

Anyone got suggestions where I can look. I don't think it's a .conf file problem, but I'm not sure what else would need to change.

modify server name, add binhex- before mineos.*; it should be server_name binhex-mineos.*;

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.