[Support] Linuxserver.io - SWAG - Secure Web Application Gateway (Nginx/PHP/Certbot/Fail2ban)


Recommended Posts

Hey guys,

sonive been pulling my hair out all week. I have this set up with minio and I can access it no problem. 
I am trying to use this with rclone as an off site backup. Both surveys are on the local network for now 
it seems that when I use the https addres in rclone the speed becomes unusable basically 400 KB/s. And sometimes I get a 403 error in rclone. 
when I set the local IP address for minio in rclone, the speed is back to normal. I have a 100/100 connection. So it seems like it has something to do with letsencrypt because when the local IP address is used for minio the transfer speed is fine. 
 

I even tried to use a VPN and transfer from my MacBook and same issue when using https duckdns ip. 
Inset it up using spaceinvader one’s video with some changes for minio. 
I can access the minio webui without any issues using https... would really appreciate some help guys 

Below is my config file

----------------------

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name name of my subdomain form duck dns.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        proxy_max_temp_file_size 2048m;
        # Below is where you add the local IP and port that minio runs on
        proxy_pass http://192.168.1.55:9768;
    }
}

Edited by maxse
Link to comment
On 12/26/2019 at 2:21 PM, drawmonster said:

Just had my certificates expire. Restarted the LE container several times, but it never tried to renew the cert. I also backup my appdata every night, so the container gets restarted nightly. Ended up having to run the renew command manually. 

 

I had gotten some emails saying the certificates were set to expire today, but I just assumed that was normal and that they would get renewed automatically. Guess not.

 


cronjob running on Sun Nov 17 02:08:00 CST 2019
Running certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/mydomain.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not yet due for renewal

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

The following certs are not due for renewal yet:
  /etc/letsencrypt/live/mydomain.com/fullchain.pem expires on 2019-12-26 (skipped)
No renewals were attempted.
No hooks were run.

I found this repeated over and over in the letsencrypt logs. So it new it was expiring, but never renewed it.

 

Is there anything glaringly obvious that would keep the LE container from renewing the certificates automatically?

Log entry is from November 17, which states that the expiration date is December 26 and thus not due for renewal. That's correct and expected behavior.

 

Read the readme to find out how renewals are handled and what to do when you receive the expiration email.

Link to comment
15 hours ago, RossEm said:

hello! am i able to use lets encrypt over a diffrent port? i have a wordpress inside the www folder and run nextcloud. is it able i can run bitwarden aswell? preferably on port 443 but port 8080 will also work.

Why not assign them different subdomains and run everything on 443?

Link to comment

I'm using the External storage support 1.7.0 App to mount local Unraid shares via Docker mapping into Nextcloud 16.0.7. I'm not using SMB to map them because it is more than one share and SMB turned out to be much slower in navigating into them via Nextcloud. 

When Syncing data to these mapped shares via Web DAV, or when I'm creating a folder via Web GUI,

  • permissions of created folders are not 777 (drwxrwxrwx) but 755 (drwxr-xr-x).
  • Permissions of a txt file created via Web GUI are 644 (-rw-r--r--) and not 666 (-rw-rw-rw-) as expected for Unraid. 

Is there any option to set the propper UMASK to make Nextcloud respect Unraid's permission scheme?

Edited by Diggewuff
Link to comment

I have just started using Unraid and trying to get nextcloud access using letsencrypt. I have followed spaceinvaderone's tutorial to the letter. I get server ready in my letsencrypt logs. 

When I try accessing from external using https:// i get

 

This site can’t be reached

nextcloud.xxxxxxx.xxx refused to connect.

Try:

Checking the connection

Checking the proxy and the firewall

ERR_CONNECTION_REFUSED

 

If I try using http://

I get my modem log in page and can log in and access my modem.

 

Any thoughts?

Link to comment
  • Just wanted to give a heads up that the name in the bitwarden proxy config file is set to "bitwarden" but the name when installing the docker defaults to "bitwardenrs". Changing the containers name to "bitwarden" was easy enough and fixed the 502 issue but since linuxserver doesn't control that particular container or it's default name I'm wondering if maybe you should update the bitwarden.subdomain.conf.sample file.
Link to comment
10 hours ago, pinion said:
  • Just wanted to give a heads up that the name in the bitwarden proxy config file is set to "bitwarden" but the name when installing the docker defaults to "bitwardenrs". Changing the containers name to "bitwarden" was easy enough and fixed the 502 issue but since linuxserver doesn't control that particular container or it's default name I'm wondering if maybe you should update the bitwarden.subdomain.conf.sample file.

Since bitwarden and bitwardenrs are 2 different things, perhaps adding the bitwardenrs.subdomain.conf.sample would be a better idea.

  • Like 1
Link to comment
13 hours ago, pinion said:
  • Just wanted to give a heads up that the name in the bitwarden proxy config file is set to "bitwarden" but the name when installing the docker defaults to "bitwardenrs". Changing the containers name to "bitwarden" was easy enough and fixed the 502 issue but since linuxserver doesn't control that particular container or it's default name I'm wondering if maybe you should update the bitwarden.subdomain.conf.sample file.

Clarifying that in the proxy conf. Thanks for the heads up: https://github.com/linuxserver/reverse-proxy-confs/pull/108

  • Thanks 1
Link to comment

Hello - 

 

Hopefully someone can help. I am following a video from Spaceinvader One to create Lets Encrypt and use it with OwnCloud but when I install the LetsEncrypt Docker and move it over to the new network advised in the video (which i have created, and change the setting to yes to keep the network) I get the below error. I have even rebooted the unraid box but still no joy. 

 

The video I am following is 

 

root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='letsencrypt' --net='proxynet' --log-opt max-size='50m' --log-opt max-file='1' --privileged=true -e TZ="Europe/London" -e HOST_OS="Unraid" -e 'EMAIL'='email' -e 'URL'='domain' -e 'SUBDOMAINS'='downloads,' -e 'ONLY_SUBDOMAINS'='true' -e 'DHLEVEL'='2048' -e 'VALIDATION'='http' -e 'DNSPLUGIN'='' -e 'PUID'='99' -e 'PGID'='100' -p '180:80/tcp' -p '1443:443/tcp' -v '/mnt/user/appdata/letsencrypt':'/config':'rw' 'linuxserver/letsencrypt' fabcbb598147c5e26db3e08cdf0c57261a347dceb06897ac4a8445b7083c0907 /usr/bin/docker: Error response from daemon: network proxynet not found.

 

 

 

***** UPDATE***** 

Deleting the docker network and re creating it worked and its now worked as expected. 

Edited by IKWeb
working now.
Link to comment
On 12/28/2019 at 4:59 PM, Diggewuff said:

I'm using the External storage support 1.7.0 App to mount local Unraid shares via Docker mapping into Nextcloud 16.0.7. I'm not using SMB to map them because it is more than one share and SMB turned out to be much slower in navigating into them via Nextcloud. 

When Syncing data to these mapped shares via Web DAV, or when I'm creating a folder via Web GUI,

  • permissions of created folders are not 777 (drwxrwxrwx) but 755 (drwxr-xr-x).
  • Permissions of a txt file created via Web GUI are 644 (-rw-r--r--) and not 666 (-rw-rw-rw-) as expected for Unraid. 

Is there any option to set the propper UMASK to make Nextcloud respect Unraid's permission scheme?

Any advice on this?

Link to comment
On 3/18/2017 at 3:12 PM, dmacias said:


The original setup for letsencrypt/fail2ban only monitors the access.log and error.log of the front end nginx of the LE docker. Unless you have something like this in your nextcloud location block which would require users to login first before actually getting to the nextcloud login.


auth_basic "Restricted";
auth_basic_user_file /config/nginx/.htpasswd;


If not then all nextcloud traffic is proxypass to backend nginx of nextcloud docker. Any failed logins are only seen in nextcloud and nextcloud nginx access/error logs. Fail2ban never sees this unless you add the location of the NC logs to the LE docker and setup a fail2ban filter to monitor those logs.

Here's my setup. So for the LE docker I added
Container Path: /logs/emby with Host Path: /mnt/user/appdata/emby/logs/
and
Container Path: /logs/nc_nginx with Host Path: /mnt/user/appdata/nextcloud/log/nginx/

Container Path: /logs/nextcloud with Host Path: /mnt/user/nextcloud/

My ../appdata/letsencrypt/fail2ban/jail.local looks like this.


# This is the custom version of the jail.conf for fail2ban
# Feel free to modify this and add additional filters
# Then you can drop the new filter conf files into the fail2ban-filters
# folder and restart the container

[DEFAULT]

# "bantime" is the number of seconds that a host is banned.
bantime  = 600

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
findtime  = 600

# "maxretry" is the number of failures before a host get banned.
maxretry = 5


[ssh]

enabled = false

[nginx-http-auth]

enabled  = true
filter   = nginx-http-auth
port     = http,https
logpath  = /config/log/nginx/error.log


[nginx-badbots]

enabled  = true
port     = http,https
filter   = nginx-badbots
logpath  = /config/log/nginx/access.log
maxretry = 2


[nginx-botsearch]

enabled  = true
port     = http,https
filter   = nginx-botsearch
logpath  = /config/log/nginx/access.log


[emby]

enabled  = true
port     = http,https
filter   = emby
logpath  = /logs/emby/server-*.txt


[nc-nginx-http-auth]

enabled  = true
filter   = nginx-http-auth
port     = http,https
logpath  = /logs/nc_nginx/error.log


[nc-nginx-badbots]

enabled  = true
port     = http,https
filter   = nginx-badbots
logpath  = /logs/nc_nginx/access.log
maxretry = 2


[nc-nginx-botsearch]

enabled  = true
port     = http,https
filter   = nginx-botsearch
logpath  = /logs/nc_nginx/access.log


[nextcloud]

enabled  = true
port     = http,https
filter   = nextcloud
logpath = /logs/nextcloud/nextcloud.log

 

I added ../fail2ban/filter.d/emby.conf


# Fail2Ban filter for emby
#

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# common.local
before = common.conf


[Definition]

_daemon = emby-server

failregex = Info HttpServer: HTTP Response 401 to <HOST>.*authenticatebyname
            Info HttpServer: HTTP Response 500 to <HOST>.*mediabrowser/Users/None

ignoreregex =

# DEV Notes:
#
#       Matching on http 401 with a trailing url including 'authenticatebyname' to catch incorrect passwords
#       Matching on http 500 with a trailing url including 'mediabrowser/Users/None' to catch incorrect usernames
#
# Author: [email protected]
I also added ../fail2ban/filter.d/nextcloud from here https://github.com/hailthemelody/nextcloud-fail2ban

# Fail2Ban filter for Nextcloud

[INCLUDES]
before = common.conf

[Definition]
failregex = Login failed.*Remote IP.*'<HOST>'
ignoreregex =

 

Hi, Happy new year everyone,

 

I need some help please, i'm trying to get fail2ban to block IP address.  I have followed above, i am not getting any IP ban's listed when

console into docker and running fail2ban-client status nextcloud.

Link to comment
On 3/18/2017 at 8:12 AM, dmacias said:


The original setup for letsencrypt/fail2ban only monitors the access.log and error.log of the front end nginx of the LE docker. Unless you have something like this in your nextcloud location block which would require users to login first before actually getting to the nextcloud login.


auth_basic "Restricted";
auth_basic_user_file /config/nginx/.htpasswd;


If not then all nextcloud traffic is proxypass to backend nginx of nextcloud docker. Any failed logins are only seen in nextcloud and nextcloud nginx access/error logs. Fail2ban never sees this unless you add the location of the NC logs to the LE docker and setup a fail2ban filter to monitor those logs.

Here's my setup. So for the LE docker I added
Container Path: /logs/emby with Host Path: /mnt/user/appdata/emby/logs/
and
Container Path: /logs/nc_nginx with Host Path: /mnt/user/appdata/nextcloud/log/nginx/

Container Path: /logs/nextcloud with Host Path: /mnt/user/nextcloud/

My ../appdata/letsencrypt/fail2ban/jail.local looks like this.


# This is the custom version of the jail.conf for fail2ban
# Feel free to modify this and add additional filters
# Then you can drop the new filter conf files into the fail2ban-filters
# folder and restart the container

[DEFAULT]

# "bantime" is the number of seconds that a host is banned.
bantime  = 600

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds.
findtime  = 600

# "maxretry" is the number of failures before a host get banned.
maxretry = 5


[ssh]

enabled = false

[nginx-http-auth]

enabled  = true
filter   = nginx-http-auth
port     = http,https
logpath  = /config/log/nginx/error.log


[nginx-badbots]

enabled  = true
port     = http,https
filter   = nginx-badbots
logpath  = /config/log/nginx/access.log
maxretry = 2


[nginx-botsearch]

enabled  = true
port     = http,https
filter   = nginx-botsearch
logpath  = /config/log/nginx/access.log


[emby]

enabled  = true
port     = http,https
filter   = emby
logpath  = /logs/emby/server-*.txt


[nc-nginx-http-auth]

enabled  = true
filter   = nginx-http-auth
port     = http,https
logpath  = /logs/nc_nginx/error.log


[nc-nginx-badbots]

enabled  = true
port     = http,https
filter   = nginx-badbots
logpath  = /logs/nc_nginx/access.log
maxretry = 2


[nc-nginx-botsearch]

enabled  = true
port     = http,https
filter   = nginx-botsearch
logpath  = /logs/nc_nginx/access.log


[nextcloud]

enabled  = true
port     = http,https
filter   = nextcloud
logpath = /logs/nextcloud/nextcloud.log

 

I added ../fail2ban/filter.d/emby.conf


# Fail2Ban filter for emby
#

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# common.local
before = common.conf


[Definition]

_daemon = emby-server

failregex = Info HttpServer: HTTP Response 401 to <HOST>.*authenticatebyname
            Info HttpServer: HTTP Response 500 to <HOST>.*mediabrowser/Users/None

ignoreregex =

# DEV Notes:
#
#       Matching on http 401 with a trailing url including 'authenticatebyname' to catch incorrect passwords
#       Matching on http 500 with a trailing url including 'mediabrowser/Users/None' to catch incorrect usernames
#
# Author: [email protected]
I also added ../fail2ban/filter.d/nextcloud from here https://github.com/hailthemelody/nextcloud-fail2ban

# Fail2Ban filter for Nextcloud

[INCLUDES]
before = common.conf

[Definition]
failregex = Login failed.*Remote IP.*'<HOST>'
ignoreregex =

 

I am having an issues when I set the container paths/hosts paths in letsencrypt. When I go to view these paths in the  letsencrypt folder the are empty. Yet, when I am setting them I can see that I am in the correct directories. 

Edited by lazydaze
Link to comment
1 hour ago, phyzical said:

Hey there,

 

is there any guidance in support for more than 1 top level domain? based on the install of the unraid app it isnt as simple as the docker config, but i assume it should be possible via manual config edits?

 

thanks in advance!

You add that in the extra domain field. If it isn't i the template, then add a variable in the template checking the correct name in the github link in the first post.

Link to comment
26 minutes ago, saarg said:

You add that in the extra domain field. If it isn't i the template, then add a variable in the template checking the correct name in the github link in the first post.

thanks for the speedy reply i will give it a crack, just to confirm does this create a separate ssl cert or simply add the additional domain to the cert that gets generated. or is that okay? from what i know i would think its generally better to have a separate ssl cert per domain

 

thanks again!

Link to comment

okay looks like it only generates the one cert but seems to work find for both domains when providing the EXTRA_DOMAINS var. thanks for all that.

 

only final quick question this docker image doesn't support dyndns updates does it? i currently just use ddclient docker image. from what i have read so do most people, just felt that was the one thing missing from this combo docker image. (unless im blind :D)

 

works great besides that look forward to using the reverse proxy functionality when i move my main server across to unraid later in the month :D

thanks!

Link to comment
13 hours ago, phyzical said:

okay looks like it only generates the one cert but seems to work find for both domains when providing the EXTRA_DOMAINS var. thanks for all that.

 

only final quick question this docker image doesn't support dyndns updates does it? i currently just use ddclient docker image. from what i have read so do most people, just felt that was the one thing missing from this combo docker image. (unless im blind :D)

 

works great besides that look forward to using the reverse proxy functionality when i move my main server across to unraid later in the month :D

thanks!

This container is only doing letsencrypt/nginx. No dynamic ip stuff.

Link to comment
13 hours ago, phyzical said:

okay looks like it only generates the one cert but seems to work find for both domains when providing the EXTRA_DOMAINS var. thanks for all that.

 

only final quick question this docker image doesn't support dyndns updates does it? i currently just use ddclient docker image. from what i have read so do most people, just felt that was the one thing missing from this combo docker image. (unless im blind :D)

 

works great besides that look forward to using the reverse proxy functionality when i move my main server across to unraid later in the month :D

thanks!

No, there is no dyn dns update. We recommend ddclient or router based solutions

 

Edit: looks like saarg beat me to it

Edited by aptalca
Link to comment

I'm having a problem. I switched my ISP recently, which means different WAN IP, and for some reason, I can't connect to my NextCloud Server via reverse proxy.

 

QSa8Joy.jpg

 

Everything looks to be in order, as it was before my ISP change:

 

qD837jj.jpg

 

And the nginx log file doesn't show any errors:

 

[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-config: executing...
[cont-init.d] 20-config: exited 0.
[cont-init.d] 30-keygen: executing...
using keys found in /config/keys
[cont-init.d] 30-keygen: exited 0.
[cont-init.d] 50-config: executing...
Variables set:
PUID=99
PGID=100
TZ=(my location)
URL=duckdns.org
SUBDOMAINS=my subdomain
EXTRA_DOMAINS=
ONLY_SUBDOMAINS=true
DHLEVEL=2048
VALIDATION=http
DNSPLUGIN=
EMAIL=my email
STAGING=

2048 bit DH parameters present
SUBDOMAINS entered, processing
SUBDOMAINS entered, processing
Only subdomains, no URL in cert
Sub-domains processed are: -d my subdomain
E-mail address entered: my email
http validation is selected
Certificate exists; parameters unchanged; starting nginx
[cont-init.d] 50-config: exited 0.
[cont-init.d] 99-custom-files: executing...
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-files: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
nginx: [alert] detected a LuaJIT version which is not OpenResty's; many optimizations will be disabled and performance will be compromised (see https://github.com/openresty/luajit2 for OpenResty's LuaJIT or, even better, consider using the OpenResty releases from https://openresty.org/en/download.html)

Nor do any of the other containers.

Edited by Stubbs
Link to comment
4 hours ago, norsemanGrey said:

Hi. I have a registered domain at Domainnameshop / Domeneshop for which I would like to set up a SSL wildcard certificate. Domeneshop has a DNS authentication plugin for Certbot (https://github.com/domeneshop/certbot-dns-domeneshop). How would I go about using this with the linuxserver/letsencrypt image?

Open a github issue for a feature request and we'll add it when we get to it

  • Like 1
Link to comment

Hi All,

 

After two days of googling,  a bit of an exaggeration, I can't figure out why I'm failing certification.  I think my port forwarding is set up correctly,  running an edgerouter x sfp.  ***** the domain name but it's filled out properly I think.  

 

here's the log output,  and attached is a screen shot of my port forwarding.  

 

2048 bit DH parameters present
SUBDOMAINS entered, processing
SUBDOMAINS entered, processing
Only subdomains, no URL in cert
Sub-domains processed are: -d *****
E-mail address entered: *****
http validation is selected
Generating new certificate
/usr/lib/python3.8/site-packages/jmespath/visitor.py:32: SyntaxWarning: "is" with a literal. Did you mean "=="?
if x is 0 or x is 1:
/usr/lib/python3.8/site-packages/jmespath/visitor.py:32: SyntaxWarning: "is" with a literal. Did you mean "=="?
if x is 0 or x is 1:
/usr/lib/python3.8/site-packages/jmespath/visitor.py:34: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif y is 0 or y is 1:
/usr/lib/python3.8/site-packages/jmespath/visitor.py:34: SyntaxWarning: "is" with a literal. Did you mean "=="?
elif y is 0 or y is 1:
/usr/lib/python3.8/site-packages/jmespath/visitor.py:260: SyntaxWarning: "is" with a literal. Did you mean "=="?
if original_result is 0:
/usr/lib/python3.8/site-packages/digitalocean/LoadBalancer.py:19: SyntaxWarning: "is" with a literal. Did you mean "=="?
if type is 'cookies':
/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py:65: SyntaxWarning: "is" with a literal. Did you mean "=="?
if self.email is '' or self.token is '':
/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py:65: SyntaxWarning: "is" with a literal. Did you mean "=="?
if self.email is '' or self.token is '':
/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py:89: SyntaxWarning: "is" with a literal. Did you mean "=="?
if self.email is '' or self.token is '':
/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py:89: SyntaxWarning: "is" with a literal. Did you mean "=="?
if self.email is '' or self.token is '':
/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py:113: SyntaxWarning: "is" with a literal. Did you mean "=="?
if self.certtoken is '' or self.certtoken is None:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for *****
Waiting for verification...
Challenge failed for domain *****
http-01 challenge for *****
Cleaning up challenges
Some challenges have failed.
IMPORTANT NOTES:
- The following errors were reported by the server:

Domain: *****281407197_ScreenShot2020-01-10at6_45_01PM.thumb.png.a02384dd9e4a189bf89f696ca3159f02.png
Type: unauthorized
Detail: Invalid response from
http://*****/.well-known/acme-challenge/P_1kowh6nWwToCI-ORAGFWGYL3TfRmq28Znn3o6Q5IA
[162.241.225.183]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
2.0//EN\">\n<html><head>\n<title>404 Not
Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.
ERROR: Cert does not exist! Please see the validation error above. The issue may be due to incorrect dns or port forwarding settings. Please fix your settings and recreate the container

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.