Stop Hogging All The Bandwidth!
It’s like a scene from an American western: you have a gigabit connection (The Good), a couple of heavy users (The Bad), and a monthly bandwidth limit (The Ugly.) In this strange new world of working from home you need a an Internet connection that has good bandwidth (speed) and good latency (response time) and it needs to work with multiple devices simultaneously. There are several ways we could solve this ranging from paying for top tier connections at a premium price or with some fancy do-it-yourself routing. We will be focusing on the latter in this article because, well, simply put we don’t have deep pockets!
First, let us take a moment to go over a few things. What exactly is bandwidth and latency in relation to network connections? Whenever I get asked to explain network bandwidth and network latency I like to use a water pipe analogy. If you can imagine a big water pipe with a shutoff valve on each end and two workers, also one on each end, that have to communicate when it is time to send more water. If you think of bandwidth as representing the diameter of the water pipe and latency as the speed the two workers are communicating at when it is time to send water through the pipe, you should be able to imagine how they work together to affect the volume of water moving through a pipe over time. The bigger the water pipe (or bandwidth) we have, the more water (data) can be transferred at a given time. Now latency is a bit tricker and that’s why I use workers in my analogy. The workers need to be able to communicate quickly so that water is sent when it needs to be stopped when it needs to stop.
So in terms of latency, if the workers were using a chalk board to write “start” or “stop” and holding the board up so the other working could see it, then we’d have a functional communication system but it would be very slow. Now instead, let us upgrade the workers to walkie-talkies and suddenly the communication is much quicker. If we can communicate quicker (low latency) we are able to get the water (data) flow control to be quicker resulting in more responsiveness.
Bandwidth and latency need to work hand-in-hand to provide “fast” internet. Latency affects response times and can be very noticeable in real-time communication and online video games. A connection with low bandwidth is going to transfer data slowly and is why your Netflix and video conferencing will reduce in quality or even stutter. Alright, enough with that, let’s get on with bandwidth throttling!
It is not too difficult to set up a Linux router and have it throttle all the bandwidth by rate limiting. The downside to that is that every user and every device will get throttled. While the kids do not need to watch Netflix at 4k at all hours of the day (and night), some of us do need to have a good Internet connection for work. So how can we work from home and use the full pipe available to select devices (i.e. work laptop) but still limiting everything else? On the other side of things if we let the kids watch 4k videos 24/7 we will hit our monthly bandwidth cap. We can handle both of these problems by dividing our users and devices into two groups: those of us on the lease that pay the bills (the parents) and those that are freeloaders (the kids.) For some it might be more important to slowing down Netflix, Disney+, Hulu, etc. while still allowing our computer to have full speed access.
If you take a moment to Google various related keywords you will find there are several guides out there on how to do basic rate limiting/bandwidth throttling, but here we are interested in allowing some devices to be in an “unlimited” group while others get rate limited. What we will be doing is controlling devices by their assigned IP addresses. This will allow us to decide who gets full speed and who gets rate limited without having to list out every device individually.
I feel the need to take a moment to describe my home network so that you can understand the lay of the land. On my network, IP addresses are assigned both statically and dynamically by my custom Linux router/server. My Linux router provides DHCPv4, DHCPv6 (my ISP gives a /64 block), DNS, firewall, and some other services. If you do not have a machine that can be dedicated as a router a good alternative is to buy a router (https://openwrt.org/supported_devices) that can run OpenWRT (https://openwrt.org/).
The part that really matters here is that my router runs Linux and that IP addresses are consistent in my network. This is important because the way we will be handling the rate limiting is by IP address. If we have short-lived IP addresses we will have to update the config often for both the IPs that we want in the “unlimited” group and those in the rate limited group. That just sounds like a lot of busy work and I don’t have time for such things.
The behind the scenes magic is handled in the Linux kernel. The way we tell it what we want to happen is by interacting through the traffic control (tc) tool. I wrote a small GNU Bash script that sets up two network groups, one unlimited and one limited, and to handle setting the traffic control options up.
Here is the entire script for your enjoyment:
Let’s go over each line so you can understand what is happening. The first line that does not start with a hash/pound “#” symbol has “NETDEV=eno1” This line contains the network interface card (NIC) name of the NIC that is connected internally to our local area network (LAN.) Then we have “tc qdisc del dev $NETDEV root handle 1” which sets up the root device. This is followed by “tc qdisc add dev $NETDEV root handle 1: htb default 1” sets up the root device to use hierarchy token bucket (HTB) which allows us to control bandwidth through classes. The following 2 lines set the speed (rate) of the root device (1:1) which will be used by the “unlimited” to 1 gigabit per second (that’s the maximum our download speed from my ISP provides) and then we set the speed (rate) of the “limited” group (1:10) to have a base speed (rate) of 250 megabits per second with a burstable ceiling of 500 megabits per second. The last section is a scripted way of setting the IP addresses between 192.168.1.100 to 192.168.1.199 to be in the “limited” group. These are the addresses my router will assign dynamically. The line with “src” represents the source address and the “dst” line is the destination address. This way we can control the speed (rate) for both incoming and outcoming connections on a given IP address. Any IP address that is not between 192.168.1.100 and 192.168.1.199 automatically falls into the “unlimited” group . I have my DHCP server set up to assign 192.168.1.10 to 192.168.1.99 statically to the devices and machines that I do not want to be rate limited. This group includes my work laptop, my home servers, and some of my projects.
The above script can be run directly or you could drop it in somewhere to be executed at system boot. Another option, for systemd users, is to put it in /etc/networkd-dispatcher/routable.d/09-my-traffic-controller.sh which will allow it to run as soon as your network devices reach a routable state.
As you can see by the example script, bandwidth limiting is fairly easy and does not require a specialized degree from your local technical college. It allows everyone to have internet access but keeps the kids from sucking down the whole internet watching Disney+. We can even expand the script further and add additional groups. Maybe we want to add a new group for the Xbox that is neither in the “unlimited” group nor the “limited” group because we don’t want game updates to take hours. We can accomplish that by adding a new group, say 1:11, that is defined with more bandwidth. It is also worth noting that we could increase the ceiling for this new group so that if nobody is using the internet except the Xbox it can have a larger burst. This can come in handy if the kids have gone off to bed and you are on the couch with your significant other and the two of you want to play a quick match in Fortnite, but darn it, there’s an update you have to install first — burstable ceiling to the rescue!
The options that tc provides is quite flexible and it is worth the time to take a quick peek at the man page. It certainly has made the quality of my work-at-home life better and kept me from pulling out all my hair — only some! I hope this article finds you well and has provided some useful information on what traffic control looks like in Linux and how to get started with it. I enjoy receiving feedback; be it suggestions, corrections, or questions. Feel free to drop some love, be safe, and hack away!
First, let us take a moment to go over a few things. What exactly is bandwidth and latency in relation to network connections? Whenever I get asked to explain network bandwidth and network latency I like to use a water pipe analogy. If you can imagine a big water pipe with a shutoff valve on each end and two workers, also one on each end, that have to communicate when it is time to send more water. If you think of bandwidth as representing the diameter of the water pipe and latency as the speed the two workers are communicating at when it is time to send water through the pipe, you should be able to imagine how they work together to affect the volume of water moving through a pipe over time. The bigger the water pipe (or bandwidth) we have, the more water (data) can be transferred at a given time. Now latency is a bit tricker and that’s why I use workers in my analogy. The workers need to be able to communicate quickly so that water is sent when it needs to be stopped when it needs to stop.
So in terms of latency, if the workers were using a chalk board to write “start” or “stop” and holding the board up so the other working could see it, then we’d have a functional communication system but it would be very slow. Now instead, let us upgrade the workers to walkie-talkies and suddenly the communication is much quicker. If we can communicate quicker (low latency) we are able to get the water (data) flow control to be quicker resulting in more responsiveness.
Bandwidth and latency need to work hand-in-hand to provide “fast” internet. Latency affects response times and can be very noticeable in real-time communication and online video games. A connection with low bandwidth is going to transfer data slowly and is why your Netflix and video conferencing will reduce in quality or even stutter. Alright, enough with that, let’s get on with bandwidth throttling!
It is not too difficult to set up a Linux router and have it throttle all the bandwidth by rate limiting. The downside to that is that every user and every device will get throttled. While the kids do not need to watch Netflix at 4k at all hours of the day (and night), some of us do need to have a good Internet connection for work. So how can we work from home and use the full pipe available to select devices (i.e. work laptop) but still limiting everything else? On the other side of things if we let the kids watch 4k videos 24/7 we will hit our monthly bandwidth cap. We can handle both of these problems by dividing our users and devices into two groups: those of us on the lease that pay the bills (the parents) and those that are freeloaders (the kids.) For some it might be more important to slowing down Netflix, Disney+, Hulu, etc. while still allowing our computer to have full speed access.
If you take a moment to Google various related keywords you will find there are several guides out there on how to do basic rate limiting/bandwidth throttling, but here we are interested in allowing some devices to be in an “unlimited” group while others get rate limited. What we will be doing is controlling devices by their assigned IP addresses. This will allow us to decide who gets full speed and who gets rate limited without having to list out every device individually.
I feel the need to take a moment to describe my home network so that you can understand the lay of the land. On my network, IP addresses are assigned both statically and dynamically by my custom Linux router/server. My Linux router provides DHCPv4, DHCPv6 (my ISP gives a /64 block), DNS, firewall, and some other services. If you do not have a machine that can be dedicated as a router a good alternative is to buy a router (https://openwrt.org/supported_devices) that can run OpenWRT (https://openwrt.org/).
The part that really matters here is that my router runs Linux and that IP addresses are consistent in my network. This is important because the way we will be handling the rate limiting is by IP address. If we have short-lived IP addresses we will have to update the config often for both the IPs that we want in the “unlimited” group and those in the rate limited group. That just sounds like a lot of busy work and I don’t have time for such things.
The behind the scenes magic is handled in the Linux kernel. The way we tell it what we want to happen is by interacting through the traffic control (tc) tool. I wrote a small GNU Bash script that sets up two network groups, one unlimited and one limited, and to handle setting the traffic control options up.
Here is the entire script for your enjoyment:
#! /bin/bash
# To check the status try something like: tc class show dev $NETDEV
# The network device we are throttling
NETDEV=eno1
# reinit
tc qdisc del dev $NETDEV root handle 1
# create the default class
tc qdisc add dev $NETDEV root handle 1: htb default 1
tc class add dev $NETDEV parent 1: classid 1:1 htb rate 1000mbps
tc class add dev $NETDEV parent 1: classid 1:10 htb rate 250mbps ceil 500mbps
for IP in {100..199}
do
tc filter add dev $NETDEV protocol ip parent 1:0 prio 1 u32 match ip src 192.168.1.$IP flowid 1:10
tc filter add dev $NETDEV protocol ip parent 1:0 prio 1 u32 match ip dst 192.168.1.$IP flowid 1:10
done
Let’s go over each line so you can understand what is happening. The first line that does not start with a hash/pound “#” symbol has “NETDEV=eno1” This line contains the network interface card (NIC) name of the NIC that is connected internally to our local area network (LAN.) Then we have “tc qdisc del dev $NETDEV root handle 1” which sets up the root device. This is followed by “tc qdisc add dev $NETDEV root handle 1: htb default 1” sets up the root device to use hierarchy token bucket (HTB) which allows us to control bandwidth through classes. The following 2 lines set the speed (rate) of the root device (1:1) which will be used by the “unlimited” to 1 gigabit per second (that’s the maximum our download speed from my ISP provides) and then we set the speed (rate) of the “limited” group (1:10) to have a base speed (rate) of 250 megabits per second with a burstable ceiling of 500 megabits per second. The last section is a scripted way of setting the IP addresses between 192.168.1.100 to 192.168.1.199 to be in the “limited” group. These are the addresses my router will assign dynamically. The line with “src” represents the source address and the “dst” line is the destination address. This way we can control the speed (rate) for both incoming and outcoming connections on a given IP address. Any IP address that is not between 192.168.1.100 and 192.168.1.199 automatically falls into the “unlimited” group . I have my DHCP server set up to assign 192.168.1.10 to 192.168.1.99 statically to the devices and machines that I do not want to be rate limited. This group includes my work laptop, my home servers, and some of my projects.
The above script can be run directly or you could drop it in somewhere to be executed at system boot. Another option, for systemd users, is to put it in /etc/networkd-dispatcher/routable.d/09-my-traffic-controller.sh which will allow it to run as soon as your network devices reach a routable state.
As you can see by the example script, bandwidth limiting is fairly easy and does not require a specialized degree from your local technical college. It allows everyone to have internet access but keeps the kids from sucking down the whole internet watching Disney+. We can even expand the script further and add additional groups. Maybe we want to add a new group for the Xbox that is neither in the “unlimited” group nor the “limited” group because we don’t want game updates to take hours. We can accomplish that by adding a new group, say 1:11, that is defined with more bandwidth. It is also worth noting that we could increase the ceiling for this new group so that if nobody is using the internet except the Xbox it can have a larger burst. This can come in handy if the kids have gone off to bed and you are on the couch with your significant other and the two of you want to play a quick match in Fortnite, but darn it, there’s an update you have to install first — burstable ceiling to the rescue!
The options that tc provides is quite flexible and it is worth the time to take a quick peek at the man page. It certainly has made the quality of my work-at-home life better and kept me from pulling out all my hair — only some! I hope this article finds you well and has provided some useful information on what traffic control looks like in Linux and how to get started with it. I enjoy receiving feedback; be it suggestions, corrections, or questions. Feel free to drop some love, be safe, and hack away!
Trackbacks
No Trackbacks
Comments
Display comments as Linear | Threaded