[Support] Nginx Proxy Manager (NPM) Official

[mgutt]

2 minutes ago, blaine07 said:

and instead using all IP:PORT I don’t think I even need the syntax

Correct. It's only needed if you manually created a custom network AND some containers like Nextcloud are using this custom network AND if NPM runs in host AND if NPM needs to be able to reach containers in the custom network. So it's a method to join a container in two different networks.

[blaine07]

2 minutes ago, mgutt said:

Correct. It's only needed if you manually created a custom network AND some containers like Nextcloud are using this custom network AND if NPM runs in host AND if NPM needs to be able to reach containers in the custom network. So it's a method to join a container in two different networks.

My Nextcloud and other stuff IS on a custom network BUT NPM isn’t running on HOST; it’s a static BRO IP on network. Yeah I don’t think it’s something I even need; likely my problem, too lol. I’ll remove and report back soon. 

[tinynja98]

Hey mgutt thanks a lot for your quick response!

 

9 hours ago, mgutt said:

Absolutely overcomplicated explanation. You don't need to touch any config file if you use NPM in unRAID. All steps are done through the GUI.

 

My bad... I forgot to mention I only followed the NPM GUI part of the explanation, didn't touch any config file

 

9 hours ago, mgutt said:

As you are using a hostname and not the IP of the nextcloud containers, it seems you created a custom network and npm and nextcloud are part of it?! Are you using docker compose?!

 

Indeed! I created a custom bridge network so I can use hostname instead of the IP because I noticed it changes everytime I start it up. However I have no idea if I am using docker compose, I don't know what this is, but I don't remember taking specific actions in order to use docker compose.

 

9 hours ago, mgutt said:

And I suggest to start without their proxy and realize the direct access including let's encrypt certificate, first.

 

So I followed your suggestion and I am now using another subdomain I got from freedns.afraid.org just to remove cloudflare completely from the equation. Let's call it "test.subdomain.org". Also I'm setting this up for another docker container (deluge) since I feel like it is a "simpler" webserver, just for testing things out. And I also reinstalled NPM to restart with a blank slate.

 

So when I setup a host in NPM as shown in the attached picture, with a newly generated Let's Encrypt SSL certificate, I am able to access deluge from the internet when typing "http://test.subdomain.org", but I still get an ERR_CONNECTION_TIMED_OUT when I try to access it with "https://test.subdomain.org". Again I verified if ports 80 (TCP) and 443 (TCP) are open with canyouseeme.org, and both are indeed open. Just to see if another webserver was using those ports, I stopped the NPM container and sure enough both ports become closed. If this can be of any help, I also attached the various nginx config files associated with this host (I didn't modify any of these manually).

 

What do you think could be causing this timeout issue? Thanks again for your help!

 

EDIT: If I use a local subdomain (say deluge.lan) instead of using "test.subdomain.org", I can access deluge with HTTP, and I also can access it with HTTPS, but I get the "Your connection is not private" message from Chrome, so I have to click on "Show advanced settings" and "Proceed to deluge.lan (unsafe)".

 

proxy.conf block-exploits.conf ssl-ciphers.conf letsencrypt-acme-challenge.conf 1.conf

Edited December 11, 2022 by tinynja98

[mgutt]

1 hour ago, tinynja98 said:

So when I setup a host in NPM as shown in the attached picture, with a newly generated Let's Encrypt SSL certificate, I am able to access deluge from the internet when typing "http://test.subdomain.org"

Ok that's a good start. This means your domain uses your correct Public IP and Port 80 is working properly, too, as you were able to verify the new SSL certificate.

 

1 hour ago, tinynja98 said:

If I use a local subdomain (say deluge.lan) instead of using "test.subdomain.org", I can access deluge with HTTP, and I also can access it with HTTPS, but I get the "Your connection is not private" message from Chrome, so I have to click on "Show advanced settings" and "Proceed to deluge.lan (unsafe)".

This part is not important for us. NPM forwards the traffic through HTTP to Port 8112. Maybe the deluge container although supports HTTPS on a different port.. but it's not interesting for us as we only need HTTP for local communication.

 

1 hour ago, tinynja98 said:

but I still get an ERR_CONNECTION_TIMED_OUT when I try to access it with "https://test.subdomain.org"

That is the problem.

 

1 hour ago, tinynja98 said:

I stopped the NPM container and sure enough both ports become closed

Interesting. So it seems NPM is the only container, which listens to Port 443, but it still fails... 🤔

 

Regarding 1.conf your NPM container listens to the ports 8080 and 4443. So you router forwards public traffic from port 80 to 8080 and 443 to 4443, right? What about IPv6? It seems your NPM has enabled it. Are you using IPv6? Does your domain have an AAAA (IPv6) DNS entry?

 

 

[jeffrey.el]

On 12/10/2022 at 9:07 PM, blaine07 said:

Anyone have any ideas how to stop this? Rebooting container stops it for a bit but it always comes back sooner or later. I thought “98” pointed to a config possibly but no dice. 
 

Any ideas?

 

See attached. 

 

Edit: seems to always continue to work fine when it does this; maybe slows down slightly. 

I came here looking for a solution to this as well, me and a friend of mine both use this container and both have this issue.

Everything ran fine for months and just a few days ago when I tried to edit an proxy entry, I got an internal error in the web interface and this issue started to show up. A restart of the container indeed fixes the issue temporarily, but it's not a solid fix.

I've already been looking into this when that friend of mine first started having this issue, I checked everything..
We both have an unique IP configured for the container, no double ports or anything.

I checked everything I could possible think of what might cause this, but since this issue started showing up out of nowhere without me making any changes network wise to the Unraid host or to the docker container itself it kind of confuses me.. 

The container itself sometimes runs fine with the error happening, sometimes it stops working, but as soon as you try to make a change in the web gui it ceases to function..

Where you able to find anything which might point to a possible cause? 

[blaine07]

6 minutes ago, jeffrey.el said:

I came here looking for a solution to this as well, me and a friend of mine both use this container and both have this issue.

Everything ran fine for months and just a few days ago when I tried to edit an proxy entry, I got an internal error in the web interface and this issue started to show up. A restart of the container indeed fixes the issue temporarily, but it's not a solid fix.

I've already been looking into this when that friend of mine first started having this issue, I checked everything..
We both have an unique IP configured for the container, no double ports or anything.

I checked everything I could possible think of what might cause this, but since this issue started showing up out of nowhere without me making any changes network wise to the Unraid host or to the docker container itself it kind of confuses me.. 

The container itself sometimes runs fine with the error happening, sometimes it stops working, but as soon as you try to make a change in the web gui it ceases to function..

Where you able to find anything which might point to a possible cause? 

I can’t confirm yet I’ve resolved my issue but it’s likely my own idiocracy. See my post above about post arguments. 

[mgutt]

8 minutes ago, jeffrey.el said:

We both have an unique IP configured for the container, no double ports or anything.

Are you sure? Stop NPM when the error is logged and then execute this to check which ports are open in unraid:

 

ss -tulpn

 

Or filter it directly for port 443

 

ss -tulpn | grep 443

[jeffrey.el]

9 minutes ago, blaine07 said:

I can’t confirm yet I’ve resolved my issue but it’s likely my own idiocracy. See my post above about post arguments. 

Well I hope it solves the issue for you, as for me, I'm not using any post Arguments..

 

 

8 minutes ago, mgutt said:

Are you sure? Stop NPM when the error is logged and then execute this to check which ports are open in unraid:

 

ss -tulpn

 

Or filter it directly for port 443

 

 

ss -tulpn | grep 443

 

I'm getting the error on all ports which I'm using in different proxy entries;

 

But the output of the command you send is the following;
 

tcp   LISTEN 0      4096                   0.0.0.0:9443       0.0.0.0:*    users:(("docker-proxy",pid=31783,fd=4))                
tcp   LISTEN 0      4096                      [::]:9443          [::]:*    users:(("docker-proxy",pid=31790,fd=4))

 

[mgutt]

4 minutes ago, jeffrey.el said:

I'm getting the error on all ports which I'm using in different proxy entries;

 

 

I'm confused why nginx should try to bind the port 8090 or 8091. Did you add any stream hosts or could there be an other reason why NPM tries to listen to these ports?!

 

[jeffrey.el]

2 minutes ago, mgutt said:

 

I'm confused why nginx should try to bind the port 8090 or 8091. Did you add any stream hosts or could there be an other reason why NPM tries to listen to these ports?!

 

Yes I added those ports myself for specifc proxy hosts, but that's exactly why I'm so confused. All ports are failing all of a sudden.

 

I have a few specific instances running which use a certificate for a API access, which listens on different ports.

Edited December 11, 2022 by jeffrey.el

[mgutt]

1 minute ago, jeffrey.el said:

Yes I added those ports myself for specifc proxy hosts

Which should not cause a "bind". A "bind" should only be done if NPM listens itself to these ports and not if it forwards the traffic. Or did you add these ports in the container config as well?!

[jeffrey.el]

Just now, mgutt said:

Which should not cause a "bind". A "bind" should only be done if NPM listens itself to these ports and not if it forwards the traffic. Or did you add these ports in the container config as well?!

Yes I added the ports to the container config as well, because the incoming connections are on these ports as well, this way I can use NPM to take care of the certificate renewals.

[mgutt]

Just now, jeffrey.el said:

Yes I added the ports to the container config as well

Does not make sense to me. NPM listens only to Port 80, 81 and 443.

 

9 minutes ago, jeffrey.el said:

this way I can use NPM to take care of the certificate renewals.

Certificate renewals are done through Port 80.

 

Or is your NPM running in bridge network, you changed port 80 to 9090 and you forward your router port 80 to port 8090?! This would be correct. But you said "added the ports".

 

 

[jeffrey.el]

1 minute ago, mgutt said:

Does not make sense to me. NPM listens only to Port 80, 81 and 443.

 

Certificate renewals are done through Port 80.

 

Or is your NPM running in bridge network, you changed port 80 to 9090 and you forward your router port 80 to port 8090?! This would be correct. But you said "added the ports".

 

 

I understand your confusion, certificate renewal is done through port 80, and indeed NPM listens to ports 80, 81 and 443. 

I however have an instance running in my network which listens to port 8090 and 8091, so to make NPM redirect these hosts with the certificate it requested and manages, I had to add those ports to the container and in the Advanced configuration section I added;

listen 8090 ssl;
listen 8091 ssl;

 

But like I said for this to work, NPM has to be able to listen to those ports as well, so in Unraid I added them to the container;

 

And those ports are port forwarded to the NPM container as well, so all should be good on that part!

[mgutt]

9 minutes ago, jeffrey.el said:

redirect these hosts

And the target hosts are using other ports than 8090/8091?

[jeffrey.el]

12 minutes ago, mgutt said:

And the target hosts are using other ports than 8090/8091?

No, they are listening to those ports as well;

 

But like I said everything has worked fine for months, and now all of a sudden this happens.. Is there anything that might cause the bind issues? Is there maybe a limit on the amount of proxy hosts that NPM can handle?

[LorcanT]

Hi mgutt,

 

Having a bit of an odd issue.

 

after a reboot recently I have had all of my proxies return a 502 error and I have no idea whats causing it.

 

My proxy hosts are all saying they are online and I can access them directly over a local network but I am genuinely stumped as to why anything like this is happening.

 

Upon examination of the log error files of each of the proxies they all show the same error - 'peer closed connection in SSL handshake while SSL handshaking to upstream' I haven't changed anything drastic with my setup on unraid or anything but I notices in my docker log thats its saying that the 'logrotated' command is failing for all the logs... could that have anything to do with it?

 

For context im using duckdns proxied through cloudflare with my own domain. I have been using it perfectly fine for the last week and its only after the reboot tonight its deciding to fail.

[blaine07]

@mgutt I haven’t had time to edit my Post Arguments yet but I just want to say thank you. Thank you for all the help you provide all of us in this thread. I can see it’s daunting. Good work mate; thank you!

[tinynja98]

Hey heyyy I just got my first *glimmer* of hope! If I understood correctly, it seems the problem is coming from my Fizz router. Let me explain how I came up to this conclusion.

 

My ISP is Fizz, so I have a Fizz router/modem (in a single unit). It sucks, so I'm using a TP-Link router "on top" of that Fizz router, and I have OpenWRT on that TP-Link router to give me access to a whole bunch of features I don't have with the Fizz router/modem. When I forward ports for my applications, I do make sure to do it on both routers (as you saw my NPM host worked with HTTP).

 

So I decided as much as possible to test each router individually (even though I cannot completely remove the Fizz router/modem because I need it for its "modem" component). Here are the tests I performed (please follow along with the attached diagram). Please note that I did make sure to forward the ports to the correct local IP every time, and I am still using the subdomain from freedns.afraid.org.

1. Scenario A is for reference, this was the setup I have been using up to this point, which yields successful access when using HTTP but ERR_CONNECTION_TIMED_OUT when using HTTPS.
2. Scenario B, my intent was to removed the TP-Link router from the equation, by placing the NPM server right behind the Fizz router. This resulted in exactly the same outcome as scenario A.
3. Scenario C, this is when it gets interesting. My intent was to remove the Fizz router from the equation. I put my computer "outside" the TP-Link router, and I added a line in my /etc/hosts which redirects "test.subdomain.org" to the TP-Link router.This way (if I'm understanding things correctly) the request never goes through the Fizz router, and the domain name still matches the one in the certificate. And it finally worked! Successful access via HTTP and HTTPS, and the connection is secured.

All of this leads me to believe there is something wrong in the configuration of the Fizz router (please confirm my conclusions). If this is right, the question that remains is what kind of setting should I be looking for in my Fizz router that would result in this sort of behavior?

 

Since I think most people have no experience with Fizz routers, I've attached a PDF file with screenshots of every page of the admin panel.

 

Thanks a lot!!

 

Fizz Admin Panel.pdf

Edited December 12, 2022 by tinynja98

[mgutt]

5 hours ago, tinynja98 said:

If this is right, the question that remains is what kind of setting should I be looking for in my Fizz router that would result in this sort of behavior?

As A and C are technically the same, I would say one of these routers do not support hair pinning / NAT Loopback properly.

 

PS You should use a smartphone to be sure that your problem is only present in your local network.

[mgutt]

6 hours ago, LorcanT said:

after a reboot recently I have had all of my proxies return a 502 error and I have no idea whats causing it.

Check debug 5xx errors on the first page of this thread. As it happened after a reboot I would assume you created a custom network, which is now missing?! There is a docker setting in unRAID to keep custom networks.

[mgutt]

7 hours ago, jeffrey.el said:

they are listening to those ports as well;

Which ips / networks are used by the containers (target and npm)?

[jeffrey.el]

57 minutes ago, mgutt said:

Which ips / networks are used by the containers (target and npm)?

NPM is 10.0.50.10, the target containers have different IP's from different subnets, but inter vlan routing is enabled for those subnets.

I've never had any issues with targets or anything, it always worked, the only issue I'm having is the bind error issue and the IP 10.0.50.10 is not used by anything else, so there is no IP conflict or anything..

[tinynja98]

1 hour ago, mgutt said:

As A and C are technically the same, I would say one of these routers do not support hair pinning / NAT Loopback properly.

 

Alright I finally got it to work!!! It only took 2 full days of messing around and a corrupt bsmodules file in the flash drive but I got there haha. After much digging I found a way to completely disable the routing function of my Fizz router/modem, and I am only using TP-Link with OpenWRT now.

 

I didn't know about "NAT Loopback" concept, it would make sense that this could be the problem in the Fizz router. I'm also thinking it could have been a firewall issue in that router as I did encounter connection problems with my work-related setup, and the solution ended up being something regarding its firewall settings (i don't know what they use but its a very closed up environment, for security reasons i suppose).

 

Anyway.... I thank you again for the tips you've provided me, it was very nice to have at least someone with whom to brainstorm about this issue.

 

Wish you all the best!

Edited December 12, 2022 by tinynja98

[mgutt]

1 hour ago, jeffrey.el said:

NPM is 10.0.50.10

And network? You need to be precisely.