External Access Guidelines


TexasDave

Recommended Posts

 

One of the things I love about unRAID is the fact it forces me (in a good way) to learn new things. I have been running it for a few years and really enjoy it. 

 

I have opened up my Plex server to several friends and they love it. And as far as I can see, this is "secure". 

 

I now want to access both my apps (like Sonarr, CP, Mylar, ...) outside my LAN. I do not have to do this but want the capability and want to learn how. 

 

Ideally, I would also like to be able to manage my unRAID system outside my LAN. Again, I do not have to but want to learn. 

 

In reading posts, it seems most people do the following:

 

  • Setup the OpenVPN-AS server if they want to access the unRAID console.
  • Then they setup "reverse proxy" to access their apps. Looks like Apache or Nginx are prime candidates?
  • For the reverse proxy, people then seem to leverage "Lets Encrypt"?
  • Looks like many folks also leverage a DDNS (I have DuckDNS installed and working well). I use it for ombi 

 

Is that generally correct? Why do folks not use the VPN for the apps as well? Is it the fact that you need a VPN client to access your apps? 

 

I am also wondering why folks do not use VPN features in their router? Again, is this because you would need a VPN client?

 

I am just trying to get a handle on the various ways I could: 

 

  • Continue to provide easy access to my Plex Server to friends
  • Access my dockers (Sonarr, CP, Mylar, ...) outside my LAN
  • Access my unRAID console outside my LAN
  • Take some of the grunt work out of the above using things like DuckDNS and Lets Encrypt

 

Any high level guidelines would be appreciated. I just want to understand the various pieces before i jump in.

 

I think I (mostly) understand what I need to do.....but am happy to be corrected and educated!

 

Thanks!

 

 

Link to comment

Hey Dave, 

 

You're spot on with all your conclusions.

 

Personally I reverse proxy my apps and use a username and password at the reverse proxy layer to secure them, all communication between proxy and app (which are both on Unraid) are via https or http with no password protection.

 

I use the OpenVPN-AS docker as my VPN to connect to my Unraid webui, as I couldn't implement it at my router level.  However I've recently upgraded to pfsense so at some point I will implement it on that.

 

First step is get a domain name.  I'd recommend namecheap personally, they have been great and removes the hassle of dealing with duckdns etc.  I have a dynamic IP and update my IP address either with ddclient, or directly from my pfsense router.

 

Once you've got your domain name and the DNS records setup correctly, letsencrypt setup is straightforward.  Then the next step is to configure nginx (which is included in the letsencrypt container) to reverse proxy your apps.  That's the point you may need some help.  Post in the thread and we can sort you out no problem.

 

Hope that helps and that you're well mate.

Edited by CHBMB
Link to comment

Many thank for this.

 

So I should really do this in two parts....

 

unRAID Console Access

  • Do it either with VPN on unRAID (OpenVPN-AS) or on router....
  • My router (ASUS RT-AC68U) does support VPN (so I can look at that)
  • I did have OpenVPN running fine and fully working last week.
  • Then I screwed it up and have not been able to get it working again. When will I learn not to mss around so much...
  • Will dig around and go with one of the above - not sure which. 
  • Just being a bit lazy and a bit frustrated as after getting OpenVPN up have spent many hours since trying to fix it...but I like a challenge too. 

Access to docker consoles / apps

  • Reverse proxy is the way to go...
  • Get a domain name (does namecheap offer DDNS?). I think so given your message...
  • You mention ddclient and I see there is a docker for that and that namecheap support this.
  • This is what you do - yes? Which docker? 
  • Then setup LetsEncrypt. Then configure Nginx (but this is built into the LetsEncrypt).

I guess it is time to get started and waste an afternoon... :-)

 

PS: Hope things are good on the home front? Drop me a PM and let me know status of the "little monster" (your words!) :-)

Link to comment

I used to have an AC68U so the OpenVPN can be done on that, I forget exactly how.

 

Yep, namecheap has DDNS.  I used the linuxserver.io ddclient predictably enough.  

 

Here's the config:

##NAMECHEAP##
daemon=600
use=web, web=dynamicdns.park-your-domain.com/getip
protocol=namecheap			
server=dynamicdns.park-your-domain.com
login=server.com
password=DDNS-PASSWORD
@, www, music, photos, nextcloud,

Setting up the letsencrypt bit is easy, just pull the docker and fill the boxes in.  You do need to have your DDNS setup right for it to work though. Here's my docker run command:

docker run -d --name="nginx" --net="bridge" --privileged="true" -e TZ="Europe/London" -e HOST_OS="unRAID" -e "EMAIL"="[email protected]" -e "URL"="server.com" -e "SUBDOMAINS"="www,nextcloud,photos,music" -e "ONLY_SUBDOMAINS"="false" -e "DHLEVEL"="4096" -e "PUID"="99" -e "PGID"="100" -p 81:80/tcp -p 443:443/tcp -v "/mnt/cache/appdata/nginx":"/config":rw linuxserver/letsencrypt

Then you just have to setup Nginx, which is the same as any other Nginx setup, so easy if you know how, not so easy if you don't! 

  • Upvote 1
Link to comment
2 minutes ago, TexasDave said:

I do not see a linux server ddclient docker @ linuxserver.com or in the listing in the unRAID apps tab?

 

Just see one from captainsano and mace? I am probably being dense. Thanks!

Hmm, well it's on dockerhub, perhaps we never made a template for it.  Either of the others will work just fine though.

Link to comment

On your config file for ddclient: 

##NAMECHEAP##
daemon=600
use=web, web=dynamicdns.park-your-domain.com/getip
protocol=namecheap			
server=dynamicdns.park-your-domain.com
login=<MYSERVERNAME>
password=<MYPASSWORD>
@, www, music, photos, nextcloud,

One should substitute their server name and password in the conf file...basically your domain name and key from namecheap. 

 

What is the @ sign and the items following the @ sign? 

 

Thanks!

Link to comment
6 hours ago, TexasDave said:

 

One of the things I love about unRAID is the fact it forces me (in a good way) to learn new things. I have been running it for a few years and really enjoy it. 

 

I have opened up my Plex server to several friends and they love it. And as far as I can see, this is "secure". 

 

I now want to access both my apps (like Sonarr, CP, Mylar, ...) outside my LAN. I do not have to do this but want the capability and want to learn how. 

 

Ideally, I would also like to be able to manage my unRAID system outside my LAN. Again, I do not have to but want to learn. 

 

In reading posts, it seems most people do the following:

 

  • Setup the OpenVPN-AS server if they want to access the unRAID console.
  • Then they setup "reverse proxy" to access their apps. Looks like Apache or Nginx are prime candidates?
  • For the reverse proxy, people then seem to leverage "Lets Encrypt"?
  • Looks like many folks also leverage a DDNS (I have DuckDNS installed and working well). I use it for ombi 

 

Is that generally correct? Why do folks not use the VPN for the apps as well? Is it the fact that you need a VPN client to access your apps? 

 

I am also wondering why folks do not use VPN features in their router? Again, is this because you would need a VPN client?

 

I am just trying to get a handle on the various ways I could: 

 

  • Continue to provide easy access to my Plex Server to friends
  • Access my dockers (Sonarr, CP, Mylar, ...) outside my LAN
  • Access my unRAID console outside my LAN
  • Take some of the grunt work out of the above using things like DuckDNS and Lets Encrypt

 

Any high level guidelines would be appreciated. I just want to understand the various pieces before i jump in.

 

I think I (mostly) understand what I need to do.....but am happy to be corrected and educated!

 

Thanks!

 

 

 

Great questions... !

 

I myself am using the vpn function in my router. That way I do not have to do -anything- within Unraid.. and every app I add still functions when using the vpn.. I much prefer it this way.

 

It also makes sure that SHOULD my unraid system crash, I can still VPN and use IPMI to reboot the system, so I even think it is more functional as well as it is easier.

 

Link to comment
8 hours ago, CHBMB said:

 

 

I use the OpenVPN-AS docker as my VPN to connect to my Unraid webui, as I couldn't implement it at my router level.  However I've recently upgraded to pfsense so at some point I will implement it on that

I just setup a VPN yesterday on pfsense yesterday (i started using about a month ago as well) and it was easy, although i can only access my network and not the wan so i need to do some tweaking.

 

Re ddns have a look at doing in pfsense as it's easier to control if your real IP or VPN IP is shared-one less thing to debug.

Link to comment
1 minute ago, DZMM said:

I just setup a VPN yesterday on pfsense yesterday (i started using about a month ago as well) and it was easy, although i can only access my network and not the wan so i need to do some tweaking.

 

Re ddns have a look at doing in pfsense as it's easier to control if your real IP or VPN IP is shared-one less thing to debug.

 

I will setup OpenVPN on my pfsense box at some point, it's already handling my DDNS perfectly.

Link to comment

@CHBMB - Thanks! I saw samples with www and @  but was unsure what the @ was. I saw Nextcloud is another service. I saw you have photos and music. Are these subdomains? 

 

For LetsEncrypt, I see in the Docker setup it asks you to fill in ports for http and https. Did you just leave map these one to one (443-->443, 80-->80) and then forward them in your router? 

 

Thanks again!

 

 

Link to comment

They are indeed subdomains.

 

You can leave 443 as 443 but for 80 change host port to 81 or something as Unraid uses port 80.  

 

In your router forward 443 to 443 on Unraid and redirect 80 to 81 on  Unraid.

Edited by CHBMB
Link to comment

Thanks to @CHBMB, I managed to get ddClient and LetsEncrypt working. I also have my VPN working fine to get in to all my systems. 

 

I think I will go with the VPN as I feel a bit more comfortable with this and I do not have to have several dockers running to get the same thing. I also not like having port 80 and 443 exposed. 

 

I assume with a strong password that I change and another strong password on my dockers and unRAID console I should be set?

 

I am going to keep LetsEncrypt up and running as that enables me to run Muximux (which is very nice). 

 

Now to get Hydra working. :-)

 

Thanks for everyones help!

Link to comment

CHBMB and others have a great setup, I use OPENVPN server on my pfsense box and it works great, I also use it as client to connect to Airvpn,org. However i also use the reverse proxy dockers, because i run nextcloud and it was much more simple to do this than configure pfsense (to me). Yes i had to expose some ports but pfsense has done such a good job at keeping bad things out that im happy. 

 

  • Like 1
Link to comment

 

I really appreciate the input. 

 

One other question - do most folks have some type of authorization (login) on apps such as NZBGet, Sonarr, Radarr, CouchPotato, etc?

 

If you are not exposing any of the above, I guess no password is needed? What about if you go the reverse proxy route? 

 

Or do you just have passwords on unRAID and VPN?

 

I think I am going to get a pfSense device. Is that what most folks use in the unRAID community? Which one? Thanks!

Edited by TexasDave
Link to comment
5 hours ago, TexasDave said:

 

I really appreciate the input. 

 

One other question - do most folks have some type of authorization (login) on apps such as NZBGet, Sonarr, Radarr, CouchPotato, etc?

 

If you are not exposing any of the above, I guess no password is needed? What about if you go the reverse proxy route? 

 

Or do you just have passwords on unRAID and VPN?

 

I think I am going to get a pfSense device. Is that what most folks use in the unRAID community? Which one? Thanks!

 

As the communication between the app and the reverse proxy is all on your internal network I don't user a username/password.  Instead I implement the username/password on the reverse proxy.  I trust a well maintained nginx instance way more than I do any of those apps.  This does however mean you need to be sure of the integrity of your LAN.

 

Pfsense is awesome, it's an open source firewall project so can install it on any device with two NICs.  There are reviews of a couple of different devices on our website.

Link to comment
9 hours ago, TexasDave said:

 

I really appreciate the input. 

 

One other question - do most folks have some type of authorization (login) on apps such as NZBGet, Sonarr, Radarr, CouchPotato, etc?

 

If you are not exposing any of the above, I guess no password is needed? What about if you go the reverse proxy route? 

 

Or do you just have passwords on unRAID and VPN?

 

I think I am going to get a pfSense device. Is that what most folks use in the unRAID community? Which one? Thanks!

I've been running pfSense in a VM for about a month and it's very impressive - i bought a dual nic on eBay for £60 - very cost effective!  I finally got my traffic shaping working last night, which has made a massive difference as i only have a 18/1 connection, so being able to prioritise traffic automatically is brilliant.

Link to comment
1 minute ago, DZMM said:

I've been running pfSense in a VM for about a month and it's very impressive - i bought a dual nic on eBay for £60 - very cost effective!  I finally got my traffic shaping working last night, which has made a massive difference as i only have a 18/1 connection, so being able to prioritise traffic automatically is brilliant.

I haven't even looked at traffic shaping yet.  Fancy sticking up a guide? 

Link to comment
13 hours ago, CHBMB said:

I haven't even looked at traffic shaping yet.  Fancy sticking up a guide? 

Edit: working on full guide now as I've cracked it

 

Happy to share when I've got time to put something together that hopefully someone else can follow and understand what I did!  

 

The traffic shaper in pfSense needs work I think, although I think the fact that my WAN traffic is all going via AirVPN made it harder to setup.  I'll run through what I finally settled upon (or stumbled upon....) last night as an acceptable solution after literally dozens of attempts over several days to shape my traffic, and I'll do a proper guide later.

 

In brief,  pfSense offers three ways of shaping.  The most basic "PRIQ" assigns a priority of 0-7 with seven being the highest where traffic with priority 7 gets bandwidth first and priority 6 doesn't until 7 has taken all it wants, then priority 5 and so on.  The problem here is that certain services can hog all the bandwidth rather than ensuring everyone gets a 'little' e.g. setting POP3 at priority 3 might mean that it NEVER gets any bandwidth i.e. emails would never get sent if 4-7 are using up all the bandwidth.

 

The most advanced method "HFSC" fixes this problem by creating queues and allowing you to set that even if the WAN or LAN is maxed out, that each of the queues gets a min amount of bandwidth (can be set as a % of the parent queue or an absolute amount e.g. 1Mb) e.g. the default queue for qP2P is given 5% of a backlogged queue, qVoip is given a guaranteed configurable min to ensure call quality remains acceptable.

 

My solution (will do in more detail in a proper post):

  1. ran the wizard and choose dedicated links as I have 1 LAN and 1 WAN connection
  2. set my line speeds based on my broadband connection - 18Mbps and 1Mbps
  3. Chose the types of traffic I wanted to prioritise - VoiP, P2P, NNTP, SMTP, Hangouts, individual games etc etc

Ended up with default rules for LAN (won't talk about WAN and qACK for now - just want to show potential and how easy it was for me to eventually create a solution that worked for me):

 

LAN - PARENT initially set at 18Mbps max via wizard

---qLink - child of LAN.  I think this queue is for internal LAN traffic and personally is why I think the shaper is so hard to figure out.  More below

---qInternet - child of LAN.  pfSense gave this 80% of LAN bandwidth, and created the children below:

--------qACK

--------qP2P - set at 5% of parent qInternet bandwidth if backlogged

--------qVoiP

--------qGames - set at 20% I think of parent qInternet bandwidth if backlogged.  Selected games added here by pfSense through automatic rules

--------qOthersHigh - set at 20% of parent qInternet bandwidth if backlogged.  Selected protocols added here that I asked to prioritise in the wizard e.g. Hangouts

--------qOthersLow - set at 5% of parent qInternet bandwidth if backlogged.  i added things like email here

  

I then created another queue qSabnzbd under Parent qInternet, and created a floating rule to put all traffic on port 563 here (pfSense only created a rule for port 119) - did this so I could see if it worked by watching the impact in Status/Queues

 

Now the problem I had, was that some of my non-unRAID internet traffic was getting thrown into qLink (think the VMs were the problem here as I don't think pfSense could work out where the traffic was coming from and going to properly), so impossible to shape (or I couldn't work out how to) -  I can't limit qLink because it would start impacting internal traffic e.g. streaming a film to a TV.... P2P traffic was a big culprit.

I've double-checked and my internal LAN traffic isn't going into qLink, just my P2P and some other random internet traffic that is trickling through.  I solved this by setting the 'bandwidth share of a backlogged queue' for P2P at 20%, so that if my line is maxed out from qInternet, then P2P via qLink would be capped at 20% of bandwidth.  I could set this lower, but until I'm certain what other internet traffic is going into qLink, I don't want to starve this queue.

 

I fixed this by:

I then made the following tweaks:

  1. Increasing LAN to 1Gbps to match my actual network
  2. Setting child qLink at 986Mbps
  3. Setting child qInternet at 14Mbps
  4. scheduling my qBittorrent client to not exceed 2Mbps during the day and unlimited overnight

Why this works for me?

  1. Setting qLink at 20% max share 986Mbps means internal traffic internet traffic that's allocated to qLink always gets through as planned by pfSense without shaping
  2. During the day, my P2P traffic that's unfortunately in qLink is locked at a max of 2Mbps - if I didn't it would take all my bandwidth.  Any other internet traffic that leaks through to qLink will get priority - again ok with me as this will be user-initiated and not background traffic like P2P or Sabnzbd.  I've assumed that this will be at most a couple of Mbps
  3. I set qInternet at 14Mbps to ensure that the bulk of my internet traffic gets shaped e.g. if I'm watching a youtube video (in the qOthersHigh), then it will take bandwidth away from qSabnzbd (qSabnzbd queue is allocated only 5% of qInternet in a backlogged queue).  qOthersHigh is allocated a guaranteed 20% of a backlogged queue i.e. 3.6Mbps, so if the youtube video needs 3Mbps then qSabnzbd (and all over traffic) will use a maximum of 15.4Mbps 11Mbps, and jump back up to 184Mbps when the video is finished
  4. overnight when we're asleep it's ok for P2P via qLink to nab all the traffic with qInternet taking whatever's left
  5.  

It's working brilliantly, because as soon as we start browsing or want to watch a video or stream something, pfSense immediately takes traffic away from lower priority tasks e.g. qSabnzbd and gives it back as soon as we've finished.  It means my pitiful 18Mbps (they are digging fibre in my village so not for much longer) is working so much more efficiently, prioritising foreground traffic and giving it to background traffic immediately when not needed.

 

The only drawback is if more than 4Mbps of internet traffic leaks into qLink then it will mean that qInternet won't get shaped because less than 14Mbps will be going through that queue, so pfSense won't shape.  This should hopefully be rare, but hopefully in the future I'll work out how to get all my internet traffic through the qInternet queue.

 

That was a lot longer than I planned for a brief teaser!

Edited by DZMM
updates
  • Upvote 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.