Jump to content
DZMM

GUIDE: How To Traffic Shape With pfSense

12 posts in this topic Last Reply

Recommended Posts

I've noticed that more and more users are creating pfSense VMs like me, or are buying dedicated boxes.  After using pfSense for a month I can wholeheartedly recommend running it in a VM as the outlay is very low, £60 in my case for a dual Intel nic, and the improvement in security, performance and network control are immense over an ISP provided modem/router.

 

I've spent a long time trying to get the traffic shaper working, and once I figured it out it was very easy to do.  I'm sharing this with the community to help others who are new to pfSense and to encourage others to do so.

 

Background

I recommend reading this guide https://calomel.org/pf_hfsc.html for a very clear explanation of how Hierarchical Fair Service Curve (HFSC) traffic management works.

 

In summary, without traffic shaping your internet connection your internet traffic or packets are processed on a first in/first out basis, which means it can be easy for one type of service to hog bandwidth and hard for other services to get enough bandwidth e.g. VoIP.  pfSense offers 3 ways of shaping traffic - PRIQ, CBQ and HFSC.  

 

PRIQ is the most basic and assigns a priority of 0-7 with seven being the highest where traffic with priority 7 gets bandwidth first and priority 6 doesn't until 7 has taken all it wants, then priority 5 and so on.  The problem here is that certain services can hog all the bandwidth rather than ensuring everyone gets a 'little' e.g. setting POP3 at priority 3 might mean that it NEVER gets any bandwidth i.e. emails would never get sent if 4-7 are using up all the bandwidth.

 

The most advanced method HFSC, fixes this problem by creating queues and allowing you to set, on a queue by queue basis, when your LAN or WAN is maxed out:

  • how much bandwidth should be guaranteed to each queue e.g. a min of 1Mbps for VoIP when the line is maxed out
  • the maximum bandwidth a queue should get out of the total available if the line is maxed out (m2)
  • You can even set how much bandwidth a queue gets initially (m1) and for how long (d) before reverting to its final max limit (m2).  An application of this could be if you want to give a lot of bandwidth initially to http traffic so short interactive page loads are fast (m1 high), but if someone is doing a big download then the lower m2 speed kicks in to stop bandwidth being hogged

 

The pfSense wizard takes care of setting up the majority of HFSC rules necessary and a few tweaks are needed to personalise the rules e.g :

  • I've created a rule that makes sure any traffic from my wife's smartphone or laptop goes into the high priority queue so I don't get any 'why is the internet so slow?' complaints
  • the default rules only created a rule for port 119 NTTP traffic, so I added one for port 563 so my Sabnzbd traffic was shapped

 

The key to shaping is to make sure you add limits that match your line speeds as pfSense only shapes once the max is hit.  E.g. I'm on a 24/1 connection and get around 19/1 on a good day, so I put 18/0.95 into the shaper wizard to make sure that the pfSense shaper always kicks in.

 

Running the wizard:

 

1. Go to Firewall/Traffic and choose Wizards.  I have 1 LAN (AirVPN_LAN in the screenshots) and 1 WAN (AirVPN_WAN) so I choose Dedicated Links:

 

9212dbc857

 

2. Then choose HFSC, set your speeds and interfaces.  Remember to at a min reduce your line speed for your WAN to make sure pfSense shapes:

2.thumb.png.08eb9a63db1a68bc162b12e309ca23db.png

 

3. Choose if you want to prioritise VoIP and how the min traffic you want to guarantee - this has to be less than 30% of the max line speed:

3.thumb.png.71bc7fa8e742849982a1fd8805a2b85e.png

 

4. Penalty Box:  is where all traffic from a given IP or Alias can be capped - I don't use this

 

5. Next up, decide if you want to shape P2P traffic and tick which protocols.  I only use BitTorrent so I only ticked that one:

4.thumb.png.4944fabc3aa881c53a4e05c7ee6d822d.png

 

6. Then Gaming - lots of choices here:

5.thumb.png.5a0fe32e9c5d3b9b5996658fb3196857.png

 

7. Other protocols can be raised or lowered.  Any that aren't listed can be added through custom floating rules later e.g. SSL NTTP on port 563 isn't included:

6.thumb.png.96da7429bcfbf34fcce661041e502640.png

 

Then click finish and wait for pfSense to automatically create all the rules.  Once pfSense has finished go to Firewall/Traffic Shaper and you'll see the queues that have been created:

7.thumb.png.b0af638b7a9dfbb2ef42c89ea1b8477d.png

 

  • What you can see is that AirVPN_WAN and AirVPN_LAN have both been setup as Parent queues, where AirVPN has two Children qLink and qInternet
  • qInternet has further Children which have been created because of the choices I made in the wizard e.g. qP2P to shape P2P traffic, qOthersHigh for the protocols I chose as high and so on.
  • qLink is the default queue, so if pfSense can't match any internet traffic it goes in this queue along with any internal LAN traffic.  Unfortunately,  my P2P traffic went in here so I had to create an additional rule to match this traffic to a queue below qInterent


If you click on any of the queues you can control its behaviour.  AirVPN_LAN is the top-level queue so it has fewer options - just make sure it's showing the right speed for your downstream speed - in my case 18Mbps

 

Here's qLink with default settings:

8.thumb.png.3f5d3d2a99f6ca69f0bcaaf9f225e510.png

 

Key:

  • Priority: This only applies if you chose PRIQ i.e. on a scale of 0-7.  It doesn't apply with HFSC
  • Queue Limit: how many packets to queue when the bandwidth has been exceeded on a FIFO basis for dropping.  Do not be tempted to set this too high as you'll suffer from "buffer bloat" - more here https://calomel.org/pf_hfsc.html.  I left alone
  • Scheduler options: I left alone, but there's some caution against using ECN here https://calomel.org/pf_hfsc.html
  • Bandwidth: This says how much this queue should use as a percent of the amount available to its Parent or an absolute amount.  If anything is entered in Link Share m2 this number is overridden.  I've found that putting numbers in both Link Share m2 and Bandwidth (really just a shortcut for m2) can cause problems, so I'd recommend just using Bandwidth and leaving linkshare m2 blank unless you want to use m1 (see below)
  • Max Bandwidth for Queue / Upper Limit: This sets an Upper Limit for how much bandwidth a queue can have E.g. even if you've set Bandwidth at 100% above, putting 50% here will limit the queue to 50% (daft, I know).  m1 sets the initial max bandwidth, d the duration in milliseconds for the m1 limit and then m2 for the limit after d has expired e.g. you can say set a m1 of 10Mbps for HTTP for 10000 (d) milliseconds and then drop to 1Mbps (m2) so that large transfers don't hog bandwidth, but small ones get through quickly e.g. HTTP pages
  • Min Bandwidth / Real Time: guarantees bandwidth regardless of what other queues are doing - the sum of all child Real Time allocations cannot exceed 80%
  • B/W Share / Link Share: sets how much bandwidth a queue gets when the Parent queue has hit its max capacity e.g. for qP2P the default is a 5% allocation of qInternet's capacity if the line is maxed out:

11.thumb.png.0f510cf3d0704b4f1c066e7159371245.png

 

pfSense runs through the following questions when it traffic shapes each packet:

 

  1. For the given queue, does it have a Real Time allocation and is this enough or does it need queuing?
  2. If Real Time allocation isn't enough, does the packet have enough Link Share to be sent or does it need queuing?
  3. If using Link Share, is there an Upper Limit set that needs to be obeyed, potentially overriding the Link Share?

 

Once I got my head around the three points above and that children queues share the bandwidth allocated to their parent, editing the rules became quite easy.

 

As you can see, the default wizard also caps qP2P to a max of 5% the total line capacity at all times in the Upper Limit row.

 

I changed the default rules to get the result I wanted by:

  1. Changing the speed for my LAN to match the speed of my network - 1Gbps
  2. Changing the speed of qInternet to match my internet speed - 18Mbps.  I also set Upper Limit and Link Share to 18Mbps
  3. Using percentages for all my Child rules under LAN qInternet rather than Mbps so they automatically adjust if I change the top number
  4. Not using any Upper Limits.  If I use Link Share m2 entries, they always match the Bandwidth number
  5. Children: these share whatever is available for parents.  For each of these enter in Bandwidth what % of the parent you want to allocate if the line was maxed out via Link Share - e.g. for P2P I set 5% Bandwidth i.e. if the line is maxed out P2P gets only 5% of 18Mb (from qInternet allocation) i.e. it's not starved of traffic, but doesn't get a lot.  Other qInternet Children got different Bandwidths - qOthersHigh (20%), qOthersLow (10%), qGames (20%) - tweak to your personal preferences
  6. Ensuring that all my Real Time allocations added up to under 80% e.g. I currently have qACK 20%, qVOIP 1% (i.e. disabled, but left rule in for the future), qGames 1% (disabled again), qOthersHigh 55%, QOthers Low 3% = grand total 80%.  

 

Hopefully that all made sense.  I now have a happier household as no-one service hogs the internet, with foreground traffic getting priority and background traffic allowed to only grab bandwidth when foreground services don't need it.

 

I'll do another couple of posts to show how I created a custom queue for Sabnzbd and to explain how qACK works in WAN.

 

Edited by DZMM
  • Upvote 1

Share this post


Link to post

Haven't got around to this yet, but awesome guide mate...

Sent from my LG-H815 using Tapatalk

Share this post


Link to post


Creating a custom queue is easy.  I do this to isolate special types of traffic or to be able to see if any rules I've created for traffic that isn't covered in the Wizard has worked.  For example, this is how I've created a new child queue qSabnzbd for my port 563 NNTP traffic as the wizard only covers port 119
 

  • clink on qInternet and at the bottom click on 'Add new Queue' to add a child queue
  • in the new window give your new queue a name and then set the desired Service Queue Options

 

17.thumb.png.392699c0739ae49ffcda04a065866031.png
My options above have set qSabnzbd to use 100% of qInternet's allocation if available, but to only use 5% if other services have maxed out qInternet.

 

To create the firewall rule go to Firewall/Rules and then click on Flouting.  Floating rules are typically used for traffic shaping as they don't block or reject - just apply an action.  They work differently to normal firewall rules as they don't block i.e. rather than stopping at the first rule that matches they are all active on all traffic.  So, be careful you don't have conflicting rules or make sure you have your most important at the bottom not top.

 

To create my Sabnzbd/Port 563 rule, I clicked on Add selected TCP as my protocol, Source Any, and changed the destination port to 563.  I then scrolled to the bottom and selected qACK/qSabnzbd for the queue (for TCP rules you have to select qACK, for UDP you don't).

 

I also found that I had to tick 'Quick' at the top of the rule creation page, which forces the rule to act immediately, to get this rule to work.  

 

Once complete your rule should look like this:

16.thumb.png.d58540ddd727218cce08953beea01e44.png

 

Edited by DZMM

Share this post


Link to post

The final change I made was to my WAN qACK settings.  Again, https://calomel.org/pf_hfsc.html has a good explanation of what qACK is:

 

Quote

The queue is for ack (acknowledge) packets without payload. Ack packets are the method your system tells the remote servers you have received the payload they sent and to send the next one. By prioritizing these packets you can keep your transfer rates high even on a highly saturated link. For example, if you are downloading a file and you receive a chunk of data the remote system will not send you the next chunk of data until you send them an OK. The OK is the ack packet. When you send the ack packet the remote system knows you got the packet and it has checked out, thus it will send the next one. If on the other hand you delay ack packets, the transfer rate will diminish quickly because the remote system wont send anything new until you respond.

 

Even if you have a super-fast connection, optimising your qACK is one reason I believe every user should traffic shape with pfSense - if your ack packets are not getting priority, then your download speeds won't be optimised.

 

Through the wizard, pfSense allocates 20% of upstream bandwidth to qACK.  For most users this won't be enough and to check go to Status/Queues and you'll see how many qACK packets have been lost.  Here's my results after leaving my machine running at 20% overnight:

18.thumb.png.688b70a98868863dafc25a81bbbc09db.png

 

As you can see in AIRVPN_WAN/qInternet/qACK I've dropped 140 packets which isn't that bad - I was expecting worse as before it went into the thousands within minutes, but I think because I've artificially capped my downstream by a meg or two and because I've shaped my connection, it's reducing how many packets are getting lost. 

 

To increase the bandwidth available to qACK, click on the queue name on the left and increase Bandwidth to a higher number.  Keep monitoring your status, and if you're still seeing packet loss keep adding in increments of 5-10% until the packet loss stops.  Don't be afraid to use a high number like 60-70% if necessary, unless you have other upload traffic that's important that you want to make sure gets a minimum upstream allocation as well.

 

Edited by DZMM
  • Upvote 1

Share this post


Link to post
8 hours ago, CHBMB said:

Haven't got around to this yet, but awesome guide mate...

Sent from my LG-H815 using Tapatalk
 

thanks - I've added a couple more posts

 

 

 

Edited by DZMM

Share this post


Link to post

Hey, I have a couple of questions 

 

Can you explain what you meant by "Changing the speed for my LAN to match the speed of my network - 1Gbps". I don't see where you would of set the Lan speed?

 

I can see that everything is getting dumped into High at the moment, I will need to work on some rules but currently looking at getting DCSP working from windows. Not much progress though.

 

Based on this screenshot does that show that the Q is full and this believes its capping out my internet connection? The limit is set at 14Mbps.

 

I'll do some more reading later, as some values I don't understand yet.

Worth adding that even in it's current state, I was able to set a download going and then start up streams on other laptops without issues and seeing the download slow down. It worked better than the limiters.

pfsense.thumb.jpeg.98961487e49014707866a94708d6c02c.jpeg

Edited by Tuftuf

Share this post


Link to post
1 hour ago, Tuftuf said:

Hey, I have a couple of questions 

 

Can you explain what you meant by "Changing the speed for my LAN to match the speed of my network - 1Gbps". I don't see where you would of set the Lan speed?

 

 

If you click on Root queue under Interface NOVA you can set this to your LAN speed.  If pfsense was working perfectly all internet traffic would then go under qInternet and any 'on-net' traffic e.g. transfers between PCs should go into qLink.  Unfortunately some of my internet traffic goes into qLink, but not a lot.  qInternet is where you want to set your line speed

 

1 hour ago, Tuftuf said:

 

I can see that everything is getting dumped into High at the moment, I will need to work on some rules but currently looking at getting DCSP working from windows. Not much progress though.

 

What port is DCSP (??) on?  What I did is the following:

  1. created a new queue say qDCSP (qNNTP in my example) so I could track my rule worked
  2. then created a firewall rule so that any traffic on destination port xx got tagged with that tag - in my example I created a new alias qNNTP_Ports for all NNTP ports (119, 563)
  3. In the advanced options set the queue for TCP traffic to qACK/qNNTP, qACK/qDCSP etc or none/qNNTP none/qDCSP for UDP traffic
  4. 1 hour ago, Tuftuf said:

     

    Based on this screenshot does that show that the Q is full and this believes its capping out my internet connection? The limit is set at 14Mbps.

     

    Hmm I'm not really sure what the bars mean.  To be honest I don't think the stats here are 100% accurate e.g. sometimes it will show traffic way in excess of my linespeed

     

    1 hour ago, Tuftuf said:

    Worth adding that even in it's current state, I was able to set a download going and then start up streams on other laptops without issues and seeing the download slow down. It worked better than the limiters.

     

    That's the beauty of it for me.  It's not 100% perfect, but once you've got the rules setup you'll really see the benefits as everything works without some services getting starved of bandwidth

1.png

2.png

Edited by DZMM

Share this post


Link to post

Replying when I should be working, I do have more research to do before I understand But here it goes.

 

DSCP is a traffic shaping QOS option, It can be set in windows and applied to an application in my case Overwatch.exe and then traffic leaving overwatch will be QOS tagged.

 

I then needed to create a rule on the firewall and under advanced Diffserv select EF which will then catch this traffic and shape it as required.

It didn't work but that's part of the research!

 

Just set Nova to 1Gbit and qInternet to 14Mbit.  I'll get some screenshots later.

Under Nova > qLink this is set to 20% -- I'm assuming this should be much higher to cover the LAN speed.

Share this post


Link to post
28 minutes ago, Tuftuf said:

Replying when I should be working, I do have more research to do before I understand But here it goes.

 

DSCP is a traffic shaping QOS option, It can be set in windows and applied to an application in my case Overwatch.exe and then traffic leaving overwatch will be QOS tagged.

 

I then needed to create a rule on the firewall and under advanced Diffserv select EF which will then catch this traffic and shape it as required.

It didn't work but that's part of the research!

 

Just set Nova to 1Gbit and qInternet to 14Mbit.  I'll get some screenshots later.

Under Nova > qLink this is set to 20% -- I'm assuming this should be much higher to cover the LAN speed.

I'd tackle it differently and do it all in pfSense and forget using windows:

 

- create a new queue e.g. qMyGames or qMyHigh

- get the ports for overwatch https://us.battle.net/support/en/article/300479 and create a new alias GamingPorts or something similar (alias is better for adding more ports in the future)

- then create a firewall rule and say that any traffic on the GamingPorts alias is assigned qACK/qMyGames for TCP or none/qMyGames for UDP

- then for the qMyGames queue give it a high bandwidth allocation/priority

 

For qLink - it doesn't really matter what you set as it only kicks in when it needs to shape Nova.  Given that Nova is set at 1000Mbps and qInterent at 14, there's always 1000-14=986Mbps available for qLink.  You only have to worry about qLink if you are lucky enough to have a gigabit internet connection or a very fast internet connection where you need to start worrying about starving your local traffic of bandwidth

 

Edit: don't forget to make qMyGames a child of qInternet

Edited by DZMM

Share this post


Link to post

This looks like a great guide for me; after I get pfsense actually setup and running on a VM under unRAID.  Do you happen to know of a good guide for the initial setup of pfsense on unRAID, to get me started?

 

My regular wifi router (ASUS RT-N66W) runs Tomato (shibby 1.40), but Tomato seems to not be updated very frequently, and the CPU in that router simply cannot handle and VPN duties, as it just slows my (already terrible) satellite internet to a crawl.

 

I'm hoping I can just use the wifi part of the router to allow devices to connect, but put all the 'processing' onto pfsense.

 

thanks.

Share this post


Link to post
On 7/23/2017 at 7:21 PM, JustinChase said:

This looks like a great guide for me; after I get pfsense actually setup and running on a VM under unRAID.  Do you happen to know of a good guide for the initial setup of pfsense on unRAID, to get me started?

 

My regular wifi router (ASUS RT-N66W) runs Tomato (shibby 1.40), but Tomato seems to not be updated very frequently, and the CPU in that router simply cannot handle and VPN duties, as it just slows my (already terrible) satellite internet to a crawl.

 

I'm hoping I can just use the wifi part of the router to allow devices to connect, but put all the 'processing' onto pfsense.

 

thanks.

The only unRAID specific issues are to make sure you have a nic you can passthrough - some useful info in this thread 

Other than that, create a freebsd VM with seabios not OVMF.

Then just follow the standard pfsense setup instructions.

Share this post


Link to post

I'm sharing a neat way I've just simplified my traffic shaping.

 

My pfSense setup is based on the guides I've posted here https://forums.lime-technology.com/topic/61401-useful-pfsense-links/ which has allowed me to setup VLANs, and then I've assigned devices to VLANs eg.

  • VL20_VPN: I've put my deluge, jackett, sonarr, radarr etc dockers here
  • VL30_CLRNET: I've put nzbget docker here, as I don't see the point of wasting 'power' sendign through the VPN when it's already encrypted
  • VL50_Priority: my STBs, PCs, Laptops, Phones etc are on here 

For each of my VLANs I have a set of firewall rules that look like this:

5a0f1657b5f62_FireShotCapture11-woody.local.lan-Firewall_Rules_V_-https___172_30_12.1_firewall_rules_php.thumb.png.4b02716eea58d4955eba2923495b26a4.png

As you can see I have a rule allowing traffic out to the WAN.  To shape this connection (remember you have to shape as traffic leaves not arrives) I've added a tag VL20_queue to the rule in advanced options:

5a0f166713bf7_FireShotCapture12-woody.local.lan-Firewall_Rul_-https___172_30_12.1_firewall_rules_edit_php.thumb.png.cb72024feaad7a9a3816273b5061b687.png

Then I've created two floating rules to assign TCP and UDP traffic to the relevant queue using the tags:

 

5a0f1684e0d8d_FireShotCapture13-woody.local.lan-Firewall_Rul_-https___172_30_12.1_firewall_rules_edit_php.thumb.png.c50aae56fd71ed63aaf3afde307f40ea.png

 

 

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now