Jump to content
linuxserver.io

[Support] Linuxserver.io - SWAG - Secure Web Application Gateway (Nginx/PHP/Certbot/Fail2ban)

5146 posts in this topic Last Reply

Recommended Posts

Here's an interesting one.

 

I would like to run MineOS through this too so I can close the ports I need open for it. I don't care about the web interface, just so ports 25565-25575 get passed on to the docker, no URL stuff. I don't know nearly enough about nginx to come up with the solution on my own and I can't find much for references to base it off of. Any ideas?

Share this post


Link to post
19 hours ago, aptalca said:

Could be DNS issue (check IP), could be port forwarding (router setting looks correct as long as you moved unraid to a different port) or maybe your ISP just started blocking ports 80 and 443.

 

See here to further troubleshoot (article not published yet but here's a preview): https://blog.linuxserver.io/p/3959a1d3-d70d-4d1d-a4ef-4dcdc0bcfd94/

I've manage to resolve it. Turns out port 80 was working fine but 443 wasn't (I have a redirect to https). After searching every option on my router I've found out, that one of the Windows app I was testing used UPnP and somehow "reserved" port 443. After removing this record everything went back to normal.

 

Thanks for your help.

Share this post


Link to post
3 hours ago, Essaith said:

I've manage to resolve it. Turns out port 80 was working fine but 443 wasn't (I have a redirect to https). After searching every option on my router I've found out, that one of the Windows app I was testing used UPnP and somehow "reserved" port 443. After removing this record everything went back to normal.

 

Thanks for your help.

Ah, I always turn off upnp on the router. Don't want some rogue apps opening ports to the outside without my knowledge

Share this post


Link to post
On 7/3/2019 at 12:57 AM, saarg said:

 

If the only thing you changed was the modem/router, then the problem is there.

 Post a screenshot of the port forwarding and we'll check if it's correct.

 

It doesn't really make sense that your sister can access ombi,but you can't.

 

You have updated your domain to point to your new IP?

think it was the only think I changed, at least at first, but I can't be certain. I made some other changes after the fact (had to regenerate the cert, possibly more), lost track of what I did and so am now starting over with only those assumptions I mentioned. Networking isn't my specialty (nor my favorite) but I have a Security+ so I'm pretty competent with networking, for the most part. Here is my router config:
image.png.ee6adab65f6c9d114c231a527e7fe873.png

There is also a built in "app port forwarding" feature, which is just a bunch of predefined port forwarding settings for common apps and protocols. I tried the one for webserver (forwards 80 and 443) and that didn't work either. 192.168.1.2 is my UnRAID server. I tried one of those websites to check for open ports and 80 and 443 are both open from outside my network. My router/modem is a Zyxel C3000Z from Centurylink. Previously I was using an older modem with the router function disabled and used my Linksys WRT1900AC with DD-WRT. I'm wonder if the port forwarding function on the Centurylink router is just broken. It wouldn't surprise me. I've known routers, especially ones issued from ISPs, to have some of the more advanced features not implemented correctly or not working.

 

As far as my sister only being able to connect, believe me, I know it makes no sense and if I were you I'd be telling me I'm wrong or mistaken because that's impossible... But the site had been down for over a week, I didn't have time to fix it, other family members asked why it was down... Then out of the blue my Ombi phone app said it got a request from my sister. I thought that impossible, maybe and old request got stuck and just went through. I messaged her, she said she just requested it a few minutes ago. The Ombi logs say the same. She confirmed that she still can load the page just fine. I don't remember if she uses the app or website but that shouldn't make a difference. My sister is by no means a computer expert but she is very smart, good with technology, and I trust that she knows enough about computers that she isn't mistaken about it. She is still the only one that can access it. So unless I was sleep walking one night and set up a firewall rule in my router to only allow connections from her IP on that port (which I checked anyway), that shouldn't be possible. That's mostly why I am leaning towards a fault in the router firmware, which is up to date.

I have manually updated my domain to point to my IP (can't get crappy ISP router to do it with my provider just yet) but using the IP address doesn't work either.

 

On 7/3/2019 at 6:16 AM, aptalca said:

If it works from outside the lan, your issue is hairpin nat. Google how to enable it for your new router

My router doesn't have hairpin NAT in the settings, just NAT and a simple enable/disable that is already enabled. The thing is it's not that it only works from outside my LAN, it that's it only works for my sister, who is outside my LAN. No one else outside my LAN can access it. See the above more more detail.

Edited by bobbintb

Share this post


Link to post
On 7/2/2019 at 6:24 PM, aptalca said:

Check line 4 of the default site config

Thanks I got it working :)

Share this post


Link to post
22 hours ago, bobbintb said:

think it was the only think I changed, at least at first, but I can't be certain. I made some other changes after the fact (had to regenerate the cert, possibly more), lost track of what I did and so am now starting over with only those assumptions I mentioned. Networking isn't my specialty (nor my favorite) but I have a Security+ so I'm pretty competent with networking, for the most part. Here is my router config:
image.png.ee6adab65f6c9d114c231a527e7fe873.png

There is also a built in "app port forwarding" feature, which is just a bunch of predefined port forwarding settings for common apps and protocols. I tried the one for webserver (forwards 80 and 443) and that didn't work either. 192.168.1.2 is my UnRAID server. I tried one of those websites to check for open ports and 80 and 443 are both open from outside my network. My router/modem is a Zyxel C3000Z from Centurylink. Previously I was using an older modem with the router function disabled and used my Linksys WRT1900AC with DD-WRT. I'm wonder if the port forwarding function on the Centurylink router is just broken. It wouldn't surprise me. I've known routers, especially ones issued from ISPs, to have some of the more advanced features not implemented correctly or not working.

 

As far as my sister only being able to connect, believe me, I know it makes no sense and if I were you I'd be telling me I'm wrong or mistaken because that's impossible... But the site had been down for over a week, I didn't have time to fix it, other family members asked why it was down... Then out of the blue my Ombi phone app said it got a request from my sister. I thought that impossible, maybe and old request got stuck and just went through. I messaged her, she said she just requested it a few minutes ago. The Ombi logs say the same. She confirmed that she still can load the page just fine. I don't remember if she uses the app or website but that shouldn't make a difference. My sister is by no means a computer expert but she is very smart, good with technology, and I trust that she knows enough about computers that she isn't mistaken about it. She is still the only one that can access it. So unless I was sleep walking one night and set up a firewall rule in my router to only allow connections from her IP on that port (which I checked anyway), that shouldn't be possible. That's mostly why I am leaning towards a fault in the router firmware, which is up to date.

I have manually updated my domain to point to my IP (can't get crappy ISP router to do it with my provider just yet) but using the IP address doesn't work either.

 

My router doesn't have hairpin NAT in the settings, just NAT and a simple enable/disable that is already enabled. The thing is it's not that it only works from outside my LAN, it that's it only works for my sister, who is outside my LAN. No one else outside my LAN can access it. See the above more more detail.

 

Since I'm on the phone and lazy, I'm not going to try to find out if you already posted your letsencrypt docker run command, so please post it.

Share this post


Link to post

Hi,

 

Can anyone please explain how I can htaccess protect my sites.

I've tried docker exec -it letsencrypt htpasswd -c /config/nginx/.htpasswd

In the console area for letsencrypt and no joy.

 

Thanks

 

Share this post


Link to post
3 hours ago, Lien1454 said:

Hi,

 

Can anyone please explain how I can htaccess protect my sites.

I've tried docker exec -it letsencrypt htpasswd -c /config/nginx/.htpasswd

In the console area for letsencrypt and no joy.

 

Thanks

 

That command creates the credentials file. Then edit your site config files to enable it for whatever location you want

Share this post


Link to post

I'm playing around with GeoIP2, but would it be possible to include the geoipupdate package in the image?

This to make it possible to automatically update the GeoIP2 database.

Share this post


Link to post

Hey guys, is there anyway we can get php7-ldap integration? Or if its already possible to be used can someone explain how to turn it on? Here is my usecase:

 

I have a Wordpress site that I am trying to use LDAPforPlex (that was just added to CA) in order to allow my PLEX users to log in using the Plex accounts. Unfortunately the Wordpress plugin I am trying to use is telling me that the LDAP-extension is not loaded and that without it I would not be able to query a LDAP server.

 

Any help is greatly appreciated as always.

Share this post


Link to post
On 7/12/2019 at 2:51 PM, saarg said:

 

Since I'm on the phone and lazy, I'm not going to try to find out if you already posted your letsencrypt docker run command, so please post it.

docker run -d --name='letsencrypt' --net='bridge' --privileged=true -e TZ="America/Denver" -e HOST_OS="Unraid" -e 'EMAIL'='myemail@gmail.com' -e 'URL'='mydomain.com' -e 'SUBDOMAINS'='mysubdomain' -e 'ONLY_SUBDOMAINS'='true' -e 'DHLEVEL'='4096' -e 'VALIDATION'='http' -e 'DNSPLUGIN'='' -e 'PUID'='99' -e 'PGID'='100' -e 'HTTPVAL'='true' -p '8008:80/tcp' -p '443:443/tcp' -v '/mnt/cache/Docker/letsencrypt':'/config':'rw' 'linuxserver/letsencrypt'

 

Although, now that I look in the log, I'm getting an error I hadn't before:
 

ERROR: Cert does not exist! Please see the validation error above. The issue may be due to incorrect dns or port forwarding settings. Please fix your settings and recreate the container

Not sure what I changed to get that. I'm losing my mind.

Share this post


Link to post
2 hours ago, bobbintb said:

docker run -d --name='letsencrypt' --net='bridge' --privileged=true -e TZ="America/Denver" -e HOST_OS="Unraid" -e 'EMAIL'='myemail@gmail.com' -e 'URL'='mydomain.com' -e 'SUBDOMAINS'='mysubdomain' -e 'ONLY_SUBDOMAINS'='true' -e 'DHLEVEL'='4096' -e 'VALIDATION'='http' -e 'DNSPLUGIN'='' -e 'PUID'='99' -e 'PGID'='100' -e 'HTTPVAL'='true' -p '8008:80/tcp' -p '443:443/tcp' -v '/mnt/cache/Docker/letsencrypt':'/config':'rw' 'linuxserver/letsencrypt'

 

Although, now that I look in the log, I'm getting an error I hadn't before:
 


ERROR: Cert does not exist! Please see the validation error above. The issue may be due to incorrect dns or port forwarding settings. Please fix your settings and recreate the container

Not sure what I changed to get that. I'm losing my mind.

See the error above

Share this post


Link to post

I'm starting to loose my mind. 

 

I had this working before but my unraid BTRFS cache drive was corrupted and I'm not able to get my bitwarden working with letsencrypt anymore.

 

I'm using the default configuration in letsencrypt and I didn't do anything special with Bitwarden but I keep getting 502 Bad Gateway.

 

I get the following error in the nginx error.log

2019/07/17 22:00:59 [error] 371#371: *2 bitwarden could not be resolved (3: Host not found), client: 192.168.2.1, server: bitwarden.*, request: "GET /favicon.ico HTTP/2.0", host: "bitwarden.xxx.com", referrer: "https://bitwarden.xxx.com/"

 

I have both letsencrypt and bitwarden on the same network "proxynet".

 

I have other services like NextCloud and phpmyadmin set up and working correctly but I can't seem to get bitwarden to work anymore.

 

Thanks.

Share this post


Link to post
1 hour ago, aptalca said:

See the error above

I don't see what post you are referring to. I checked canyouseeme.org and now it seems my port forward is not working. It was working previously but now isn't I'm pretty sure I didn't change anything, the port forward settings are still there and correct. Redid them and rebooted. I'm starting to think my new router is a dud. Given the unexplainable behavior I mentioned with only my sister being able to access the site, that would make sense. In my experience, extremely bizarre/impossible behavior usually indicates a hardware failure.

Share this post


Link to post
7 hours ago, bobbintb said:

docker run -d --name='letsencrypt' --net='bridge' --privileged=true -e TZ="America/Denver" -e HOST_OS="Unraid" -e 'EMAIL'='myemail@gmail.com' -e 'URL'='mydomain.com' -e 'SUBDOMAINS'='mysubdomain' -e 'ONLY_SUBDOMAINS'='true' -e 'DHLEVEL'='4096' -e 'VALIDATION'='http' -e 'DNSPLUGIN'='' -e 'PUID'='99' -e 'PGID'='100' -e 'HTTPVAL'='true' -p '8008:80/tcp' -p '443:443/tcp' -v '/mnt/cache/Docker/letsencrypt':'/config':'rw' 'linuxserver/letsencrypt'

 

Although, now that I look in the log, I'm getting an error I hadn't before:
 


ERROR: Cert does not exist! Please see the validation error above. The issue may be due to incorrect dns or port forwarding settings. Please fix your settings and recreate the container

Not sure what I changed to get that. I'm losing my mind.

You haven't forwarded Port 80 to the correct port you set for let's encrypt that is 8008.

Share this post


Link to post
10 hours ago, bobbintb said:

I don't see what post you are referring to. I checked canyouseeme.org and now it seems my port forward is not working. It was working previously but now isn't I'm pretty sure I didn't change anything, the port forward settings are still there and correct. Redid them and rebooted. I'm starting to think my new router is a dud. Given the unexplainable behavior I mentioned with only my sister being able to access the site, that would make sense. In my experience, extremely bizarre/impossible behavior usually indicates a hardware failure.

Referring to the post I quoted. Your error literally says see the error message above but you didn't post the error message above.

Share this post


Link to post

Also, for anyone who's trying to troubleshoot ports. . .

Can you see me only works if you already have a service that's listening on that port. If you have a cert error and you never get the "server ready" message in the log, there is nothing listening. Nginx is down because it doesn't start until certs are successfully retrieved.

 

See here to troubleshoot:

https://blog.linuxserver.io/2019/07/10/troubleshooting-letsencrypt-image-port-mapping-and-forwarding/

Share this post


Link to post
11 hours ago, aptalca said:

Referring to the post I quoted. Your error literally says see the error message above but you didn't post the error message above.

Oops, sorry. It basically says the same thing. "Timeout during connect (likely firewall problem)."

 

 

11 hours ago, aptalca said:

Also, for anyone who's trying to troubleshoot ports. . .

Can you see me only works if you already have a service that's listening on that port. If you have a cert error and you never get the "server ready" message in the log, there is nothing listening. Nginx is down because it doesn't start until certs are successfully retrieved.

 

See here to troubleshoot:

https://blog.linuxserver.io/2019/07/10/troubleshooting-letsencrypt-image-port-mapping-and-forwarding/

I realized that a while after the fact. Thanks for the reminder. I'm all over the place lately.

 

18 hours ago, saarg said:

You haven't forwarded Port 80 to the correct port you set for let's encrypt that is 8008.

I did have it at one point. I added it. Now I have 443, 80, and 8008 forwarded. Still getting the error.

Share this post


Link to post

Follow up to my post from yesterday.

 

Still getting a 502 Bad Gateway for Bitwarden. I forced update on both Letsencrypt and Bitwarden and still nothing.

 

I am able to connect by using the IP address (in my case: http://192.168.2.242:8343/#/) but this is of limited value as I need to be able to connect to bitwarden remotely. I tried to change the bitwarden in the configuration file to upper case B with no change.

 

Below is the conf I'm using for Bitwarden, I have no problems with any other servers at this time.

 

# make sure that your dns has a cname set for bitwarden and that your bitwarden container is not using a base url

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name bitwarden.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 128M;

    # enable for ldap auth, fill in ldap details in ldap.conf
    #include /config/nginx/ldap.conf;

    location / {
        # enable the next two lines for http auth
        #auth_basic "Restricted";
        #auth_basic_user_file /config/nginx/.htpasswd;

        # enable the next two lines for ldap auth
        #auth_request /auth;
        #error_page 401 =200 /login;

        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_bitwarden bitwarden;
        proxy_pass http://$upstream_bitwarden:80;
    }

    location /notifications/hub {
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_bitwarden bitwarden;
        proxy_pass http://$upstream_bitwarden:80;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
    }

    location /notifications/hub/negotiate {
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_bitwarden bitwarden;
        proxy_pass http://$upstream_bitwarden:80;
    }

}
 

 

As stated above, I continue to get the following errors when I try to connect 

 

2019/07/18 22:25:47 [error] 380#380: *1 bitwarden could not be resolved (3: Host not found), client: 192.168.2.1, server: bitwarden.*, request: "GET /favicon.ico HTTP/2.0", host: "bitwarden.XXX.com", referrer: "https://bitwarden.XXX.com/"

Share this post


Link to post
44 minutes ago, Patb said:

Follow up to my post from yesterday.

 

Still getting a 502 Bad Gateway for Bitwarden. I forced update on both Letsencrypt and Bitwarden and still nothing.

 

I am able to connect by using the IP address (in my case: http://192.168.2.242:8343/#/) but this is of limited value as I need to be able to connect to bitwarden remotely. I tried to change the bitwarden in the configuration file to upper case B with no change.

 

Below is the conf I'm using for Bitwarden, I have no problems with any other servers at this time.

 

# make sure that your dns has a cname set for bitwarden and that your bitwarden container is not using a base url

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name bitwarden.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 128M;

    # enable for ldap auth, fill in ldap details in ldap.conf
    #include /config/nginx/ldap.conf;

    location / {
        # enable the next two lines for http auth
        #auth_basic "Restricted";
        #auth_basic_user_file /config/nginx/.htpasswd;

        # enable the next two lines for ldap auth
        #auth_request /auth;
        #error_page 401 =200 /login;

        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_bitwarden bitwarden;
        proxy_pass http://$upstream_bitwarden:80;
    }

    location /notifications/hub {
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_bitwarden bitwarden;
        proxy_pass http://$upstream_bitwarden:80;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
    }

    location /notifications/hub/negotiate {
        include /config/nginx/proxy.conf;
        resolver 127.0.0.11 valid=30s;
        set $upstream_bitwarden bitwarden;
        proxy_pass http://$upstream_bitwarden:80;
    }

}
 

 

As stated above, I continue to get the following errors when I try to connect 

 

2019/07/18 22:25:47 [error] 380#380: *1 bitwarden could not be resolved (3: Host not found), client: 192.168.2.1, server: bitwarden.*, request: "GET /favicon.ico HTTP/2.0", host: "bitwarden.XXX.com", referrer: "https://bitwarden.XXX.com/"

Make sure the container name is bitwarden and that both containers are in the same non-default bridge network

Share this post


Link to post

They are both in a network called "proxynet" and I verified that the names match. Still no success.

Share this post


Link to post

ok, this is strange, I renamed the container to bitwarden (lower case b) and it now works...

 

Thanks

Share this post


Link to post
6 hours ago, bobbintb said:

Oops, sorry. It basically says the same thing. "Timeout during connect (likely firewall problem)."

 

 

I realized that a while after the fact. Thanks for the reminder. I'm all over the place lately.

 

I did have it at one point. I added it. Now I have 443, 80, and 8008 forwarded. Still getting the error.

I think you don't fully understand port forwarding.

You are not forwarding port 8008. You are forwarding port 80 on the Wan side to port 8008 on the container. So on the wsn side it's always 443 and 80, but in the actual router/firewall you use the Wan side ports as the source ports and the ports you set in the container template you set as the destination ports.

From looking at your screenshot, you are just opening the ports, and not port forwarding.

Share this post


Link to post

Alright, I've been reading through this forum getting all the answers to my problems so far. I was able to figure it out and get everything in a working state. 

I just have just a few questions now -- 

 

On startup I see the alert about the LuaJIT version issue, is that a problem?

I also see the warnings for conflicting server names. How do I fix that? I have only used the default templates and only edited them where necessary.  

Certificate exists; parameters unchanged; starting nginx
[cont-init.d] 50-config: exited 0.
[cont-init.d] 99-custom-files: executing...
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-files: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
nginx: [alert] detected a LuaJIT version which is not OpenResty's; many optimizations will be disabled and performance will be compromised (see https://github.com/openresty/luajit2 for OpenResty's LuaJIT or, even better, consider using the OpenResty releases from https://openresty.org/en/download.html)

nginx: [error] lua_load_resty_core failed to load the resty.core module from https://github.com/openresty/lua-resty-core; ensure you are using an OpenResty release from https://openresty.org/en/download.html (rc: 2, reason: module 'resty.core' not found:

no field package.preload['resty.core']
no file './resty/core.lua'
no file '/usr/share/luajit-2.1.0-beta3/resty/core.lua'
no file '/usr/local/share/lua/5.1/resty/core.lua'
no file '/usr/local/share/lua/5.1/resty/core/init.lua'
no file '/usr/share/lua/5.1/resty/core.lua'
no file '/usr/share/lua/5.1/resty/core/init.lua'
no file '/usr/share/lua/common/resty/core.lua'
no file '/usr/share/lua/common/resty/core/init.lua'
no file './resty/core.so'
no file '/usr/local/lib/lua/5.1/resty/core.so'
no file '/usr/lib/lua/5.1/resty/core.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
no file './resty.so'
no file '/usr/local/lib/lua/5.1/resty.so'
no file '/usr/lib/lua/5.1/resty.so'
no file '/usr/local/lib/lua/5.1/loadall.so')
nginx: [warn] conflicting server name "ombi.*" on 0.0.0.0:443, ignored
nginx: [warn] conflicting server name "sl.*" on 0.0.0.0:443, ignored
nginx: [warn] conflicting server name "tautulli.*" on 0.0.0.0:443, ignored
nginx: [warn] conflicting server name "ombi.*" on [::]:443, ignored
nginx: [warn] conflicting server name "sl.*" on [::]:443, ignored
nginx: [warn] conflicting server name "tautulli.*" on [::]:443, ignored
Server ready

 

In my default site config I enable the http redirect to https

 

server {
	listen 80;
	listen [::]:80;
	server_name _;
	return 301 https://$host$request_uri;
}

This works for all dockers except for Tautulli. When I go to the http for that I get page not found, any ideas? 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.