mgutt Posted August 2, 2021 Author Share Posted August 2, 2021 5 hours ago, jackwan1 said: when I tried to do the same for others, I do not get the steams. Did you enable cache assets? (you should not) Your said RTSP. Doesn't it use Port 554? NPM listens only to port 80 and 443. If NPM should listen to 554 you need to: - open the port 554 on your router with NPM as your target - open the advanced config tab of the proxy host and add the following rule: listen 554; If this does not work we need to check the nginx.conf. This the nginx conf, created by NPM if no option has been enabled: # ------------------------------------------------------------ # example.com # ------------------------------------------------------------ server { set $forward_scheme http; set $server "127.0.0.1"; set $port 80; listen 80; listen [::]:80; server_name example.com; access_log /data/logs/proxy-host-2_access.log proxy; error_log /data/logs/proxy-host-2_error.log warn; location / { # Proxy! include conf.d/include/proxy.conf; } # Custom include /data/nginx/custom/server_proxy[.]conf; } As you can see it includes "/data/nginx/custom/server_proxy.conf", but this file needs to be created by the user. So it adds nothing. And it includes "conf.d/include/proxy.conf" which contains the following rules: add_header X-Served-By $host; proxy_set_header Host $host; proxy_set_header X-Forwarded-Scheme $scheme; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Real-IP $remote_addr; proxy_pass $forward_scheme://$server:$port; Why I'm posting this: Maybe it includes a rule or misses a rule which breaks video streaming. Try to search in the internet for an nginx.conf which works for security webcams and then we compare the rules. Quote Link to comment
DrLucasMendes Posted August 2, 2021 Share Posted August 2, 2021 Dear @mgutt I think I found a typo in your template because I don't remember in setting this folders. It seems that you set Data and Certificates to /mnt/cache/appdata instead of /mnt/user/appdata. My UNRAID has different cache drivers and I found the NPMO appdata in the wrong drive. LOL I don't know if it was something I did before and my UNRAID kept the template saved. However, I would suggest for you to double check there. Thank you again. All the best, Lucas Quote Link to comment
DrLucasMendes Posted August 2, 2021 Share Posted August 2, 2021 (edited) 17 hours ago, mgutt said: This is something which will be removed in future Unraid versions. My suggestion: Run NPM as host and run all other containers as bridge. Ohh, thank you!. I will redo the settings with your suggestions. Should I set "Privileged" on in the NPM? I forgot HOST network provides the UNRAID server IP to the docker. I will follow your Plex suggestion @mgutt making a new bridge network and add it to the NPM. I think it is the safe way to go. Thank you very much for your time and dedication. Your work is awesome. Lucas Edited August 2, 2021 by DrLucasMendes Quote Link to comment
Candle Posted August 2, 2021 Share Posted August 2, 2021 14 hours ago, mgutt said: There is no easy fix possible as this is a limitation of docker itself: https://stackoverflow.com/a/51973512/318765 I guess I am confused. Don't your instructions show to use a custom network? How do we use "host" if that is the right way to do it? Quote Link to comment
mgutt Posted August 2, 2021 Author Share Posted August 2, 2021 1 hour ago, Candle said: How do we use "host" if that is the right way to do it? Both methods work, but with host it's more stable in an ipv6 network. I tried to use the custom network solution with ipv6, but it fails if my router gets a new ipv6 prefix. Sadly it's not possible to create custom networks with "dynamic" ipv6 prefixes or automatically update the fixed ipv6 of a container. I will update my post and show the "host" method. 1 Quote Link to comment
jackwan1 Posted August 3, 2021 Share Posted August 3, 2021 (edited) On 8/2/2021 at 12:05 AM, mgutt said: Did you enable cache assets? (you should not) Your said RTSP. Doesn't it use Port 554? NPM listens only to port 80 and 443. If NPM should listen to 554 you need to: - open the port 554 on your router with NPM as your target - open the advanced config tab of the proxy host and add the following rule: listen 554; If this does not work we need to check the nginx.conf. This the nginx conf, created by NPM if no option has been enabled: # ------------------------------------------------------------ # example.com # ------------------------------------------------------------ server { set $forward_scheme http; set $server "127.0.0.1"; set $port 80; listen 80; listen [::]:80; server_name example.com; access_log /data/logs/proxy-host-2_access.log proxy; error_log /data/logs/proxy-host-2_error.log warn; location / { # Proxy! include conf.d/include/proxy.conf; } # Custom include /data/nginx/custom/server_proxy[.]conf; } As you can see it includes "/data/nginx/custom/server_proxy.conf", but this file needs to be created by the user. So it adds nothing. And it includes "conf.d/include/proxy.conf" which contains the following rules: add_header X-Served-By $host; proxy_set_header Host $host; proxy_set_header X-Forwarded-Scheme $scheme; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Real-IP $remote_addr; proxy_pass $forward_scheme://$server:$port; Why I'm posting this: Maybe it includes a rule or misses a rule which breaks video streaming. Try to search in the internet for an nginx.conf which works for security webcams and then we compare the rules. Dear @mgutt Thank you for your help. Did a little research on the net and found NGINX RTMP Streaming Server Installation Guide (bartsimons.me) perhaps that is the way to go. because right now nginx proxy manager (as I know and my host setup) can handle http(s) very well, but it is lacking of ability to handle streaming service in h.264 or h.265 format and the newer cameras and nvrs are no longer use RTSP streaming in their app. If I go with the installation of RTMP streaming server, what should I do with the nginx config in the proxy manager? do I add the "rtmp" set of the config in the "advanced" section? Edited August 3, 2021 by jackwan1 Quote Link to comment
mgutt Posted August 4, 2021 Author Share Posted August 4, 2021 On 8/3/2021 at 8:02 PM, jackwan1 said: but it is lacking of ability to handle streaming service in h.264 or h.265 format NPM has no "ability". Its only a GUI for nginx. On 8/3/2021 at 8:02 PM, jackwan1 said: NGINX RTMP Streaming Server Installation Guide (bartsimons.me) This guide explains how to use nginx as a media streaming server. Is this your target? NPM is mainly to setup reverse proxy rules. On 8/3/2021 at 8:02 PM, jackwan1 said: If I go with the installation of RTMP streaming server, what should I do with the nginx config in the proxy manager? Not possible, as you can't modify the nginx installation inside of the NPM docker. I mean you could, but it will be lost after the next update. I would use a different container, if really needed: https://hub.docker.com/r/jasonrivers/nginx-rtmp And you find this part in the manual: Quote OBS Configuration Under broadcast settigns, set the follwing parameters: Streaming Service: Custom Server: rtmp://<your server ip>/live Play Path/Stream Key: mystream This means OBS pushes the stream to this RTMP server and multiple people could use the server address to watch the stream. If your camera is able to push its stream, then this could be a solution. But I think it should even work with a "stream" rule as mentioned here: https://stackoverflow.com/a/66621298/318765 NPM supports stream rules through the stream tab, which I'm using for my minecraft server (but you can't add multiple streams for the same port depening on different domains): Quote Link to comment
jackwan1 Posted August 6, 2021 Share Posted August 6, 2021 thanks @mgutt I see what you are saying and the RTMP server concept is too complicated for a home setting. I also see the nginx streaming port setting but that is the same as a simple port fowarding in my router which is the current setup and its been working for years. I guess for constant streaming video, we can only use one port for each host. I also tried rtsp on port 554, the vlc player will work only for one camera and will not work with other brands/model. Lots to learn and explore. Quote Link to comment
jackwan1 Posted August 9, 2021 Share Posted August 9, 2021 (edited) Hi @mgutt I have a few elementary questions about setting up https in proxy-manager. 1. Does every dns require a separate ssl certificate? 2. Does the server has to listen on port 443 for https? I am trying to setup webmin access via proxy-manager, as you know webmin default port is 10000 and the default access scheme is https. so here is my setting in proxy-mgr, I also ran a ssl certificate in proxy-mgr for that. my config.json autoindex_localtime on; autoindex on; server { listen 80 default_server; server_name _; return 301 https://$host$request_uri; } location / { try_files $uri /index.html; } when I enter in the browser webmin4.dns name It has no problem to connect to the login page of [email protected],x (see photo) However, after I login I got a page like this. Note the url has port 10000 defined and it returns a bad connection error. If I remove the port designation(10000) on that error page, I will be directed to the webmin dash board, no problem defind https://webmin4.dns name wont help What did I do wrong? Edited August 12, 2021 by jackwan1 Quote Link to comment
mgutt Posted August 9, 2021 Author Share Posted August 9, 2021 1 hour ago, jackwan1 said: 1. Does every dns require a separate ssl certificate? If you want to add Lets Encrypt Certificates, then yes, but if you want to use a wildcard certificate, then no: https://www.the-digital-life.com/nginx-proxy-manager-ssl/ 1 hour ago, jackwan1 said: 2. Does the server has to listen on port 443 for https? It depends. If you want to use IPv6, then yes, as IPv6 has no port forwarding. If you only want to use IPv4, then no, as you can forward any port. 1 hour ago, jackwan1 said: my config.json Where do you found this file?! 1 hour ago, jackwan1 said: However, after I login I got a page like this. Note the url has port 10000 defined and it returns a bad connection error. So you open https://webmin4.dns/ and you see the login page. Then you login and you are forwarded to https://webmin4.dns:10000/ which fails? Then this is a "bug" of webmin. Ironically it is present since a decade: https://serverfault.com/questions/98987/webmin-doesnt-work-fine-behind-reverse-proxy I think you have the following solutions to solve this: a) add this to the advanced config tab of your proxy host: proxy_redirect http://192.168.x.x:10000 https://webmin4.dns; proxy_redirect is an nginx option to overwrite the redirect. It's explained here: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect You need to test it. Maybe webmin4 does not forward to an IP. Then use this instead: proxy_redirect http://webmin4.dns:10000 https://webmin4.dns; b) allow NPM and your proxy host, to listen to port 10000 by adding this to the advanced config: listen 10000; Now, NPM does not listen only to the ports 80 and 443. It listens to 10000 as well. But beware, this does not work if your webmin container and your nging proxy manager container both use the same network like bridge (as only one can listen to the same port). So it could be necessary to change the webmin container host port to for example 10001 while its still listening internally to 10000: Of course you need to change the NPM proxy host accordingly, so it forwards the traffic from 10000 (and 443) to 10001: And if NPM is running as bridge, you need to add this port in the containers config: c) Change webmin to port 443 and change your NPM proxy host accordingly: I'm not sure if this works as 443 is usually only for https and not http traffic. So test first if you can reach webmin by http://192.168.x.x:443 (http not https!) before going further. If it works, c) should be the easiest option to solve this. Quote Link to comment
jackwan1 Posted August 9, 2021 Share Posted August 9, 2021 OK @mgutt Your solution a. works, however I have to modify it as follows: proxy_redirect https://webmin4.dns:10000 https://webmin4.dns; Notice both urls uses https not one http and the other https. Once it passed the webmin login, the url became internal to the webmin, so it was https://dns:10000 all we have to tell nginx is to redirect it to https://dns, problem solved. And my external http to https Json conversion also works. so now all I have to do is type in a web browser the dns without http or https and it will reach the webmin login page and when I login, it goes to the dash board. 1 Quote Link to comment
sdballer Posted August 10, 2021 Share Posted August 10, 2021 Does this docker work if my isp blocks port 80? I tried the other guys repo and it was a no go so I stuck with swag. Hoping to finally use npm… Quote Link to comment
jackwan1 Posted August 10, 2021 Share Posted August 10, 2021 (edited) 32 minutes ago, sdballer said: Does this docker work if my isp blocks port 80? I tried the other guys repo and it was a no go so I stuck with swag. Hoping to finally use npm… First u need to find out the http error code when they block your port 80, then you can add port 80 redirection in config. Json with some thing like this server { listen 80 default_server; server_name _; return 301 https://$host$request_uri; } Edited August 10, 2021 by jackwan1 Quote Link to comment
mgutt Posted August 10, 2021 Author Share Posted August 10, 2021 24 minutes ago, jackwan1 said: then you can add port 80 redirection in config You can't forward something which never reaches the proxy. 51 minutes ago, sdballer said: Does this docker work if my isp blocks port 80? I don't see a reason why it shouldn't work, but you can't automatically authorize ssl certificates as their check is fixed on Port 80. So you would need to install and authorize them on your own. For example through your own domain's dns entry. But this should be also valid for swag. Quote Link to comment
jackwan1 Posted August 12, 2021 Share Posted August 12, 2021 (edited) @mgutt Here is another intersting problem, this time involves Edgerouter empty GUI behind nginx reverse proxcy. The problem manifested just as described by the op in the following thread. Basically I CAN Login to the edgerouter, but when I get there the WebGUI is empty. There were many discussions but I can't get any out of it. There is a suggestion on a websocket fix on ubiquiti community forum. Access Edgemax gui via nginx reverse proxy - websocket problem | Ubiquiti Community by gainfulshrimp "server { listen 80; server_name ubnt.mydomain.com; return 301 https://$host$request_uri; } upstream erl { server 192.168.1.1:443; keepalive 32; } server { listen 443 ssl http2; include /etc/nginx/snippets/letsencryptcerts.conf; server_name ubnt.mydomain.com; include /etc/nginx/snippets/letsencryptauth.conf; client_max_body_size 512m; location / { include /etc/nginx/snippets/localonly.conf; proxy_pass https://erl; proxy_http_version 1.1; proxy_buffering off; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forward-For $proxy_add_x_forwarded_for; } } " But I do not know enough of it to implement in the NPM. There was also a post indicated that haproxy will fix the problem, why there is a difference. I added the folowing in the advanced section but it did not work proxy_http_version 1.1; proxy_buffering off; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header Host $Host(or the domain-name); proxy_set_header X-Real-IP $remote_addr(or the ip address:443); proxy_set_header X-Forward-For $proxy_add_x_forwarded_for; Please help. Edited August 12, 2021 by jackwan1 Quote Link to comment
mgutt Posted August 12, 2021 Author Share Posted August 12, 2021 9 hours ago, jackwan1 said: There is a suggestion on a websocket fix on ubiquiti community forum. I tried to reverse proxy the Unifi Controller Container. As long I did not enable Websockets I received this error message: But after enabling "Websockets Support" in the Proxy Host settings, the error was gone (but I never had an empty UI): Another important part is the "https" Scheme as unifi does not allow communication through "http". Note: Enabling Websocket Support adds these rules to the nginx config file: proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $http_connection; proxy_http_version 1.1; This would be the complete config if you enable Websocket Support, add SSL, force SSL and adding something through the Advanced Tab: # ------------------------------------------------------------ # unifi.example.com # ------------------------------------------------------------ server { set $forward_scheme https; set $server "192.168.178.8"; set $port 8443; listen 80; listen [::]:80; listen 443 ssl http2; listen [::]:443; server_name unifi.example.com; # Let's Encrypt SSL include conf.d/include/letsencrypt-acme-challenge.conf; include conf.d/include/ssl-ciphers.conf; ssl_certificate /etc/letsencrypt/live/npm-2/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/npm-2/privkey.pem; # Force SSL include conf.d/include/force-ssl.conf; # Websockets Support proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $http_connection; proxy_http_version 1.1; # Logs access_log /data/logs/proxy-host-3_access.log proxy; error_log /data/logs/proxy-host-3_error.log warn; # Rules added through the Advanced Tab #listen 8080; location / { # Websockets Support proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $http_connection; proxy_http_version 1.1; # Proxy add_header X-Served-By $host; proxy_set_header Host $host; proxy_set_header X-Forwarded-Scheme $scheme; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Real-IP $remote_addr; proxy_pass $forward_scheme://$server:$port; } } So you don't need to add those results through the Advanced Config again. If you need to add rules, add only those that aren't already part of the config. 1 Quote Link to comment
jackwan1 Posted August 12, 2021 Share Posted August 12, 2021 (edited) @mgutt I forwarded the https to 443 on my router, no conflict of ports, its not running in a container. what I failed to do is turn-on websocket support in NGM, once I did that, everything worked fine. Incidentally when I put this into the advanced section "proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $http_connection; proxy_http_version 1.1;" The NPM entry for the edgerouter went off line, so I took it out and its all good. Edited August 12, 2021 by jackwan1 Quote Link to comment
mgutt Posted August 12, 2021 Author Share Posted August 12, 2021 2 minutes ago, jackwan1 said: Incidentally when I put this into the advanced section ... The NPM entry for the edgerouter went off line That's what I meant. By that you added a rule which already exists. I think this isn't allowed an breaks the nginx configuration at all. Quote Link to comment
jackwan1 Posted August 12, 2021 Share Posted August 12, 2021 (edited) side effect on opening external access to edgerouter ui using reverse proxy. As stated before, I have no problem to access the router from external, however, I found the following side affect which is described by op in this thread. Edgerouter Lite Logs Show Someone Trying to SSH into UniFi AC : Ubiquiti (reddit.com) I thought I had set up the firewall rules on my edgerouter correctly to drop the packets to prevent someone from accessing my unifi AP AC Lite via ssh from outside of the network. Looking at the logs on my router today it seems as if that isn't the case. My question is, what do I have to change? I have setup a static DNS (via duckdns) service to remotely access the edgerouter and have the AP connected to my UniFi account. That is all I need. SSH can be from inside the network only. Port forwarding is disabled. Snippet of my logs: Jan 19 19:13:17 ubnt sshd[9296]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=153.99.182.4 user=root Jan 19 19:13:12 ubnt sshd[9263]: PAM service(sshd) ignoring max retries; 6 > 3 Jan 19 19:13:12 ubnt sshd[9263]: PAM 5 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=153.99.182.4 user=root Jan 19 19:13:05 ubnt sshd[9267]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=58.218.204.181 user=root Jan 19 19:12:58 ubnt sshd[9267]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=58.218.204.181 user=root Jan 19 19:12:55 ubnt sshd[9263]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=153.99.182.4 user=root Jan 19 19:12:51 ubnt sshd[9232]: PAM service(sshd) ignoring max retries; 6 > 3 I can disable ssh in my Router UI, but based on the post below, it did not really solve the problem That disables ssh access to your ERL. Your AP isn't getting any SSH attempts, your ERL is. If you've set it up properly then this shouldn't be a concern. The way it should be done is that WAN_LOCAL only allows established and related in and SSH. SSH should be configured to only allow public key authentication, no password authentication. The webui will always only authenticate with a password so make it listen only on some management IP and restrict access as you see fit, maybe even only allow access from the ERL itself and use SSH tunneling to log into the webui. You certainly don't have to set it up this way but you need to secure any outside access such that anything on the internet can't connect without an authorized public key. From what you've told us you left the webui totally open to the world so blocking just ssh doesn't help anything, you need to block the webui from remote attackers as well. Is there any thing Nginx can do on this, or I have to configure ssh public key? What if I forwarded to a non standard port locally. In the same token, am I also subject to these attempts by opening webmin? Edited August 12, 2021 by jackwan1 Quote Link to comment
mgutt Posted August 12, 2021 Author Share Posted August 12, 2021 Is the Edge Router your main router and firewall? Of course you should not allow access to it's web panel through the internet. Who knows if it's software has a security hole. 1 hour ago, jackwan1 said: am I also subject to these attempts by opening webmin? Of course. If it can be reached through the internet, then it will be attacked. But there is a little difference between your edge router and webmin. Webmin is running isolated in a container. If it has a security hole and an attacker takes it over, then he will be locked inside of the container. But depending on the used network he could try to attack other local clients in your network. So the safest variant is a VPN tunnel. Quote Link to comment
jackwan1 Posted August 13, 2021 Share Posted August 13, 2021 @mgutt thanks so much for your help. Yes the edge router is my only router which also is my firewall. The webmin, if it is hacked, could create havocs to my ubuntu server. After all these days of work, i am going to close down the services. Its a bad idea to begin with, not knowing the consequences of doing so. Quote Link to comment
Juise99 Posted August 19, 2021 Share Posted August 19, 2021 OK, not sure whats going on here. I've been using Djoss's container for about 6 months no problems. I played around with a wildcard cert and it blew up. Couldn't change entries, errors deleting existing entries, just all bad. I went the lazy route and setup your container and everything worked for about 2 days. Now I haven't touched anything and I'm having the same issues as the other container. What logs should I be looking at? basic setup: home.mydomain.com is an A Record that is updated with an app if my IP changes. allmyotherstuff.mydomain.com are C names pointing to home.mydomain.com pfsense router with rules for 80 and 443 to go the reverse proxy I have entries for both public and local access everything worked! Now if I try to regenerate a new ssl cert for an existing entry I get "ENOENT: no such file or directory, open '/data/nginx/proxy_host/7.conf'" Quote Link to comment
mgutt Posted August 19, 2021 Author Share Posted August 19, 2021 2 hours ago, Juise99 said: ENOENT: no such file or directory, open '/data/nginx/proxy_host/7.conf'" /data points to mnt/user/appdata/Nginx-Proxy-Manager-Official/data So please check the content of this folder. Quote Link to comment
jackwan1 Posted August 19, 2021 Share Posted August 19, 2021 The ssh login attempts described in my previous post is recorded in my edge router logs. Edge router from Ubiquiti is an industrial router, it has lot more functions than those of commercial grade. My old netgear routers for example, will never record such so you will never know. Quote Link to comment
Kopernikus Posted August 20, 2021 Share Posted August 20, 2021 Hi, I'm new to Unraid, but already have some experience with nginx on my Synology NAS. So I have a few questions about this docker container: - Whats the difference between this docker container and the one from "jlesage" (where most tutorials refer to)? - In your docker the network is set to custom br0 but in the "jelesage" docker it's set to default bridge network, what's the reason for this? - I have my own wildcard certificate so I won't be using Letsencrypt, so I have no plans of using http, only https, will the docker work if I only forwared port 443 to this docker container? Thx and keep up the good work 😀 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.