[Support] Nginx Proxy Manager (NPM) Official

[illegal-deference8464]

maybe not update but since last restart of it atleast. i moved it to the top of my list of containers to start as well so it would kick off first

[Batman]

After updating to Unraid 6.12.10, it appears that "host access to custom networks" is no longer working.

 

I have a number of Docker containers on my server, nearly all of which are using the host or bridge network. The two that aren't are Pi Hole and my internal NPM instance--both of these are using br0 since I need a dedicated IP address for them to work.

 

On Unraid 6.12.5 and prior, I had "host access to custom networks" enabled with (I believe) ipvlan as the custom network type. With this setup, I defined a DNS record for my internal NPM (proxy.local.mydomain.com) and a CNAME record for each container (i.e. home.local.mydomain.com). NPM would then add SSL and route the traffic to http://HOSTNAME:PORT. This then allowed me to access the Home Assistant docker instance with home.local.mydomain.com.

 

Now on Unraid 6.12.10, I'm getting 502 errors for all of my containers except for NPM itself (nginx-internal.local.mydomain.com).

 

I've tried restarting Unraid, disabling and reenabling "host access to custom networks", both ipvlan and macvlan, and running NPM and Pi Hole as privileged. I recall I've had some issues in the past getting NPM on br0 to talk to containers on host and bridge, but everything was working before the update.

 

Running the test command in the first post showed that my containers are reachable from NPM:

# curl -sS http://10.0.0.100:8123/ >/dev/null && echo "Container is reachable"
Container is reachable

 

Anything else I can try? Any other solutions that avoid the br0 troubles? My motherboard does have a second NIC, but I haven't tried anything with it yet.

 

It appears "host access to custom networks" is not the issue, but rather than NPM is failing to resolve my server name. If I replace the name with the IP address, it works as expected.

 

I rebooted my router, then the server, and I still get 502 errors if NPM is using the server's hostname. I think that's fixed similar issues in the past. I prefer to use names like that where possible (in case I change the server's IP), but I may just revert to using the IP to get everything working again.

 

Last edit: I also have an external instance of NPM that uses the bridge network instead, and it looks like I was already using the server IP there (for the same reason), so I guess the two instances are consistent now.

Edited May 11 by Batman

[relink]

I have this up and running fine, but Im noticeing that my cache ssd is being absolutely hammered during a large transfer thats going through the proxy. 

 

I moved the appdata for npm to a seperate drive thinking it would solve the issue but It didn't. I am transferring several TB of data from my NAS to my Seafile server running on Unraid. You have to use the domain name when connecting to seafile otherwise youll get a CSRF error and no writes will commit. But I am noticeing that my cache drive is being written to non stop the entire transfer, but it's not filling up. 

Nothing I mentioned is using the cache drive. npm's app data is on a different SSD. Seafile and MariaDB are both on a ZFS pool. The cache drive shouldn't be involved at all other than thats where the "system" share is located. 

What can I do to get this to stop? That SSD can't handle this. 

[mgutt]

6 hours ago, relink said:

. I am transferring several TB of data from my NAS to my Seafile server running on Unraid. You have to use the domain name when connecting to seafile otherwise youll get a CSRF error and no writes will commit. But I am noticeing that my cache drive is being written to non stop the entire transfer, but it's not filling up. 

Did you enable caching for this host?

 

Another reason could be proxy_request_buffering, which is by default on:

https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_buffering

 

That's why I'm using this advanced setting for my nextcloud (which even solved timeouts while moving huge files and timeouts while uploading huge files):

 

location / {
  add_header Strict-Transport-Security "max-age=15552000; includeSubdomains; preload;";
  client_body_buffer_size 512k;
  client_max_body_size 32G;
  proxy_request_buffering off;
  proxy_read_timeout 600s;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection $http_connection;
  proxy_http_version 1.1;
  # Proxy!
  include conf.d/include/proxy.conf;
}

[illegal-deference8464]

On 5/7/2024 at 4:43 PM, mgutt said:

Only after update? Any other containers or unRAID itself listening to Port 80?

 

Check output of:

ss -tulpn | grep ":80"

so i rebuilt the server and started over and then updated again and it started again. it seems something in the update system keeps breaking it. im not sure where to go from here when i run the "ss -tulpn" it says "bash: ss: command not found"

[buckethead]

Whenever I try and make the TLS certs, it fails. The test button says that it should work when I try that, but the timeout still happens during the actual verification. If I go to domain.com I get the nginx congratulations page and if I go to the acme pathway I get the expected error mentioned by the documentation. I think this is caused by an unraid update in the last year or so, cause it started happening after I updated unraid.

 

Does anyone know if some unraid configuration might've changed that would break this verification?

[mgutt]

3 hours ago, buckethead said:

If I go to domain.com I get the nginx congratulations page

From outside like mobile connection

 

If yes: Check your logs.

 

[buckethead]

16 hours ago, mgutt said:

From outside like mobile connection

 

If yes: Check your logs.

 

Yes is like that from outside and I have checked the logs. It just says it failed to download the challenge files. I'm not sure why though, since they're accessible. Is there a way to check it's not some kind of permissions issue?

 

Detail: During secondary validation: xx.xx.xx.xx: Fetching http://domain.com/.well-known/acme-challenge/cMkVI9mOwEO60J1OjIM3hTb91C4r0yC7_tF1WU-lzHE: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

2024-05-13 14:29:38,730:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

 

[relink]

On 5/12/2024 at 12:41 AM, mgutt said:

Did you enable caching for this host?

 

Another reason could be proxy_request_buffering, which is by default on:

https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_buffering

 

That's why I'm using this advanced setting for my nextcloud (which even solved timeouts while moving huge files and timeouts while uploading huge files):

 

location / {
  add_header Strict-Transport-Security "max-age=15552000; includeSubdomains; preload;";
  client_body_buffer_size 512k;
  client_max_body_size 32G;
  proxy_request_buffering off;
  proxy_read_timeout 600s;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection $http_connection;
  proxy_http_version 1.1;
  # Proxy!
  include conf.d/include/proxy.conf;
}

sorry it took so long to respond, had 2 drive failures the next day. but i'm back up and running now and my cache drive is still getting hammered.

 

I did have caching on, i have since turned it off.

 

 and this is my current advanced config section:

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
client_max_body_size 0;
proxy_request_buffering off;

 

[mgutt]

7 hours ago, buckethead said:

Is there a way to check it's not some kind of permissions issue?

I never heard about something similar.

 

Would you pm me the domain, so I can try it out from my location?

 

6 hours ago, relink said:

did have caching on, i have since turned it off

And did it solve your issue?

[buckethead]

On 5/17/2024 at 1:23 AM, mgutt said:

I never heard about something similar.

 

Would you pm me the domain, so I can try it out from my location?

 

And did it solve your issue?

The domain is my actual name, so I'd rather not send it out. My brother did try it from his house and got the normal nginx setup message. 

 

I did some more testing, and I can go to the url that it's testing. While it's testing, I can open the file on my browser with the full url. So I can go to http://emby.mydomain.com/.well-known/acme-challenge/Vvdxy3NeUIJsbP5VLQnMLN1xcAdXGiQo11Y09bObDSU and it shows the the contents in the browser. So it's weird that it's timing out.

 

I also tried running the certbot command manually with -v flag and it didn't give any new information.

Edited May 20 by buckethead

[buckethead]

On 5/20/2024 at 5:50 PM, buckethead said:

The domain is my actual name, so I'd rather not send it out. My brother did try it from his house and got the normal nginx setup message. 

 

I did some more testing, and I can go to the url that it's testing. While it's testing, I can open the file on my browser with the full url. So I can go to http://emby.mydomain.com/.well-known/acme-challenge/Vvdxy3NeUIJsbP5VLQnMLN1xcAdXGiQo11Y09bObDSU and it shows the the contents in the browser. So it's weird that it's timing out.

 

I also tried running the certbot command manually with -v flag and it didn't give any new information.

 

I've figured this out. The reason it works whenever I try it and not whenever the certbot tool tried it is because in March 2024 Lets Encrypt updated their certification verification policies. They now verify from multiple regions. I had blocked all incoming traffic from anywhere not US and Canada. It just happened to coincide with when I updated unraid

[cscoppa]

I got bit by this too, took me a LONG time to stumble on the solution.    (Unifi gear)

[Duckers]

What's the proper steps to add a docker properly to a subdomain? I've tried to add even hastebin the same way inn npm as i did my image host, but it keeps redirecting to my xbackbone image subdomain.

Edited June 7 by Duckers

[AlexB_BN]

Guys I need your help. I'm about a week deep into my Unraid experience and can't setup NPM to access File Browser via HTTPS. HTTP is working fine for some reason.

 

As recommended here I tried using NPM as host and DuckDNS and File Browser as bridge. But nothing works so far.

 

Any suggestions?

 

Any help is appreciated. Thanks!

 

 

[Kilrah]

Normal to use HTTP between NPM and the service, it's on the outside that it matters.

[AlexB_BN]

1 hour ago, Kilrah said:

Normal to use HTTP between NPM and the service, it's on the outside that it matters.

Thanks for the answer but now I'm confused lol

 

I should be able to access  mydomain.duckdns.org via HTTPS right? That's what's not working for me.

 

Sorry in advance for the confusion.

[mgutt]

9 hours ago, AlexB_BN said:

can't setup NPM to access File Browser via HTTPS. HTTP is working fine for some reason

It seems you messed up the port settings. For example you are forwarding all requests on your unraid server on port 18443 to the container port 18443. But the container (nginx) listens only to 80, 81, 443 and 3000. You can't change the ports of nginx itself. So edit your container settings and change the container ports back to the default. So 18443 (host) forwards to 443 (container). 

 

 

Why did you even change the container ports?

[AlexB_BN]

3 hours ago, mgutt said:

It seems you messed up the port settings. For example you are forwarding all requests on your unraid server on port 18443 to the container port 18443. But the container (nginx) listens only to 80, 81, 443 and 3000. You can't change the ports of nginx itself. So edit your container settings and change the container ports back to the default. So 18443 (host) forwards to 443 (container). 

 

Ok. So ist should like this now right? Yet I get a 502 Bad Gateway openresty error...

 

Quote

Why did you even change the container ports?

 

I was trying to replicate what you did in your Nextcloud, Duck, NPM YouTube Tutorial lol

 

[mgutt]

44 minutes ago, AlexB_BN said:

Ok. So ist should like this now right? Yet I get a 502 Bad Gateway openresty error...

Bad Gateway usually means that NPM is not able to reach the target. What happens if you open your public IP in your browser? Note: most browser change it to https:// so edit it, so it really loads http://yourIP

 

It should show the NPM welcome page (if not try with your mobile phone and mobile connection).

 

Regarding your screenshot: The port setting looks correct now. And you are sure you can use port 80 on your unraid server (usually this is used by unraid itself if you don't changed it).

 

[AlexB_BN]

6 hours ago, mgutt said:

Bad Gateway usually means that NPM is not able to reach the target. What happens if you open your public IP in your browser? Note: most browser change it to https:// so edit it, so it really loads http://yourIP

 

It should show the NPM welcome page (if not try with your mobile phone and mobile connection).

 

- When I try to open my IP I get the "ERR_CONNECTION_REFUSED" error. But the NPM welcome page worked initially after installation!

 

- 502 error within my network and "err_ssl_unrecognized_name_alert" error on my phone. A quick SSL Server Test scan retuned with a "Inconsistent server configuration". Yikes...

 

Yes I changes Port 80 for Unraid when I set it up.

 

Thank you for your help @mgutt. Much appreciated!

[mgutt]

18 minutes ago, AlexB_BN said:

"err_ssl_unrecognized_name_alert" error on my phone.

It seems you did not use http://yourIP

This error is usually only returned by opening https://yourIP, which is pretty normal as your public IP can't have a valid SSL certificate.

 

Your router forwards 80 to unraid:80?

 

 

[AlexB_BN]

2 hours ago, mgutt said:

It seems you did not use http://yourIP

This error is usually only returned by opening https://yourIP, which is pretty normal as your public IP can't have a valid SSL certificate.

 

I did. The error I received was ERR_CONNECTION_REFUSED

 

Quote

Your router forwards 80 to unraid:80?

 

I think so... See below

 

Thank you again!

[mgutt]

The router forwarding is wrong. 

 

You are forwarding from port 80, which is http, to port 1880 on your unraid server. In your last screenshot of your containers, you showed, that you forward port 80 of your host (unraid) to port 80 of the container (npm). 

 

As nothing is listening on port 1880, this explains why the connection can't be established.

 

Two options:

- change the router forwarding from 80 to 80

- or change the host port of the container to 1880

 

 

[AlexB_BN]

9 hours ago, mgutt said:

The router forwarding is wrong. 

 

You are forwarding from port 80, which is http, to port 1880 on your unraid server. In your last screenshot of your containers, you showed, that you forward port 80 of your host (unraid) to port 80 of the container (npm). 

 

As nothing is listening on port 1880, this explains why the connection can't be established.

 

Two options:

- change the router forwarding from 80 to 80

- or change the host port of the container to 1880

 

Ok. Thanks. I did it. See below

 

We made progress: My IP now leads to the NMP welcome page.

 

http://192.168.178.99:8666/ now opens File Browser

BUT https://192.168.178.99:8666/ as well as my duckdns domain leads to 502 openresty error page.

 

Thanks for your patience with this Rookie sir! Much appreciated as always!