Skip to content

Stop Hogging All The Bandwidth!

It’s like a scene from an American western: you have a gigabit connection (The Good), a couple of heavy users (The Bad), and a monthly bandwidth limit (The Ugly.) In this strange new world of working from home you need a an Internet connection that has good bandwidth (speed) and good latency (response time) and it needs to work with multiple devices simultaneously. There are several ways we could solve this ranging from paying for top tier connections at a premium price or with some fancy do-it-yourself routing. We will be focusing on the latter in this article because, well, simply put we don’t have deep pockets!

First, let us take a moment to go over a few things. What exactly is bandwidth and latency in relation to network connections? Whenever I get asked to explain network bandwidth and network latency I like to use a water pipe analogy. If you can imagine a big water pipe with a shutoff valve on each end and two workers, also one on each end, that have to communicate when it is time to send more water. If you think of bandwidth as representing the diameter of the water pipe and latency as the speed the two workers are communicating at when it is time to send water through the pipe, you should be able to imagine how they work together to affect the volume of water moving through a pipe over time. The bigger the water pipe (or bandwidth) we have, the more water (data) can be transferred at a given time. Now latency is a bit tricker and that’s why I use workers in my analogy. The workers need to be able to communicate quickly so that water is sent when it needs to be stopped when it needs to stop.

So in terms of latency, if the workers were using a chalk board to write “start” or “stop” and holding the board up so the other working could see it, then we’d have a functional communication system but it would be very slow. Now instead, let us upgrade the workers to walkie-talkies and suddenly the communication is much quicker. If we can communicate quicker (low latency) we are able to get the water (data) flow control to be quicker resulting in more responsiveness.

Bandwidth and latency need to work hand-in-hand to provide “fast” internet. Latency affects response times and can be very noticeable in real-time communication and online video games. A connection with low bandwidth is going to transfer data slowly and is why your Netflix and video conferencing will reduce in quality or even stutter. Alright, enough with that, let’s get on with bandwidth throttling!

It is not too difficult to set up a Linux router and have it throttle all the bandwidth by rate limiting. The downside to that is that every user and every device will get throttled. While the kids do not need to watch Netflix at 4k at all hours of the day (and night), some of us do need to have a good Internet connection for work. So how can we work from home and use the full pipe available to select devices (i.e. work laptop) but still limiting everything else? On the other side of things if we let the kids watch 4k videos 24/7 we will hit our monthly bandwidth cap. We can handle both of these problems by dividing our users and devices into two groups: those of us on the lease that pay the bills (the parents) and those that are freeloaders (the kids.) For some it might be more important to slowing down Netflix, Disney+, Hulu, etc. while still allowing our computer to have full speed access.

If you take a moment to Google various related keywords you will find there are several guides out there on how to do basic rate limiting/bandwidth throttling, but here we are interested in allowing some devices to be in an “unlimited” group while others get rate limited. What we will be doing is controlling devices by their assigned IP addresses. This will allow us to decide who gets full speed and who gets rate limited without having to list out every device individually.

I feel the need to take a moment to describe my home network so that you can understand the lay of the land. On my network, IP addresses are assigned both statically and dynamically by my custom Linux router/server. My Linux router provides DHCPv4, DHCPv6 (my ISP gives a /64 block), DNS, firewall, and some other services. If you do not have a machine that can be dedicated as a router a good alternative is to buy a router (https://openwrt.org/supported_devices) that can run OpenWRT (https://openwrt.org/).

The part that really matters here is that my router runs Linux and that IP addresses are consistent in my network. This is important because the way we will be handling the rate limiting is by IP address. If we have short-lived IP addresses we will have to update the config often for both the IPs that we want in the “unlimited” group and those in the rate limited group. That just sounds like a lot of busy work and I don’t have time for such things.

The behind the scenes magic is handled in the Linux kernel. The way we tell it what we want to happen is by interacting through the traffic control (tc) tool. I wrote a small GNU Bash script that sets up two network groups, one unlimited and one limited, and to handle setting the traffic control options up.
Here is the entire script for your enjoyment:
#! /bin/bash
# To check the status try something like: tc class show dev $NETDEV
# The network device we are throttling
NETDEV=eno1
# reinit
tc qdisc del dev $NETDEV root handle 1
# create the default class
tc qdisc add dev $NETDEV root handle 1: htb default 1
tc class add dev $NETDEV parent 1: classid 1:1 htb rate 1000mbps
tc class add dev $NETDEV parent 1: classid 1:10 htb rate 250mbps ceil 500mbps
for IP in {100..199}
do
tc filter add dev $NETDEV protocol ip parent 1:0 prio 1 u32 match ip src 192.168.1.$IP flowid 1:10
tc filter add dev $NETDEV protocol ip parent 1:0 prio 1 u32 match ip dst 192.168.1.$IP flowid 1:10
done


Let’s go over each line so you can understand what is happening. The first line that does not start with a hash/pound “#” symbol has “NETDEV=eno1” This line contains the network interface card (NIC) name of the NIC that is connected internally to our local area network (LAN.) Then we have “tc qdisc del dev $NETDEV root handle 1” which sets up the root device. This is followed by “tc qdisc add dev $NETDEV root handle 1: htb default 1” sets up the root device to use hierarchy token bucket (HTB) which allows us to control bandwidth through classes. The following 2 lines set the speed (rate) of the root device (1:1) which will be used by the “unlimited” to 1 gigabit per second (that’s the maximum our download speed from my ISP provides) and then we set the speed (rate) of the “limited” group (1:10) to have a base speed (rate) of 250 megabits per second with a burstable ceiling of 500 megabits per second. The last section is a scripted way of setting the IP addresses between 192.168.1.100 to 192.168.1.199 to be in the “limited” group. These are the addresses my router will assign dynamically. The line with “src” represents the source address and the “dst” line is the destination address. This way we can control the speed (rate) for both incoming and outcoming connections on a given IP address. Any IP address that is not between 192.168.1.100 and 192.168.1.199 automatically falls into the “unlimited” group . I have my DHCP server set up to assign 192.168.1.10 to 192.168.1.99 statically to the devices and machines that I do not want to be rate limited. This group includes my work laptop, my home servers, and some of my projects.

The above script can be run directly or you could drop it in somewhere to be executed at system boot. Another option, for systemd users, is to put it in /etc/networkd-dispatcher/routable.d/09-my-traffic-controller.sh which will allow it to run as soon as your network devices reach a routable state.

As you can see by the example script, bandwidth limiting is fairly easy and does not require a specialized degree from your local technical college. It allows everyone to have internet access but keeps the kids from sucking down the whole internet watching Disney+. We can even expand the script further and add additional groups. Maybe we want to add a new group for the Xbox that is neither in the “unlimited” group nor the “limited” group because we don’t want game updates to take hours. We can accomplish that by adding a new group, say 1:11, that is defined with more bandwidth. It is also worth noting that we could increase the ceiling for this new group so that if nobody is using the internet except the Xbox it can have a larger burst. This can come in handy if the kids have gone off to bed and you are on the couch with your significant other and the two of you want to play a quick match in Fortnite, but darn it, there’s an update you have to install first — burstable ceiling to the rescue!

The options that tc provides is quite flexible and it is worth the time to take a quick peek at the man page. It certainly has made the quality of my work-at-home life better and kept me from pulling out all my hair — only some! I hope this article finds you well and has provided some useful information on what traffic control looks like in Linux and how to get started with it. I enjoy receiving feedback; be it suggestions, corrections, or questions. Feel free to drop some love, be safe, and hack away!

ping (ICMP) blocked? No problem, enter hping!

When traveling while working we often find ourselves using networks that have certain restrictions. For example, many free WiFi hotspots will block things like ICMP echo requests (ping) which makes things a bit difficult when you are trying to figure out problems with connectivity, where packet loss is occurring, and what is reachable and what is not. Enter hping, a TCP/IP utility that can do far more than the name suggests.

First an example using standard ping (ICMP):

ping -c1 http://www.google.com
PING http://www.google.com (172.217.14.228) 56(84) bytes of data.

— http://www.google.com ping statistics —
1 packets transmitted, 0 received, 100% packet loss, time 0ms


As we can see in the above output, we never got a reply. Is the network down in between? Is there a firewall that’s blocking our requests? We don’t know.. So let’s take a look at using hping to send a TCP packet, instead of ICMP, to port 80 on a well known website:

sudo hping -c 1 -S -p 80 http://www.google.com
Password:
HPING http://www.google.com (wlp2s0 172.217.14.228): S set, 40 headers + 0 data bytes
len=44 ip=172.217.14.228 ttl=127 id=13972 sport=80 flags=SA seq=0 win=11680 rtt=51.0 ms

— http://www.google.com hping statistic —
1 packets tramitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 51.0/51.0/51.0 ms


Ah-ha! We got a reply! So TCP connections to port 80 at least are not blocked. We can work with that (hint this is where having your own custom, configurable OpenVPN server comes in handy: configure it to answer on port 80 and you are back to work).

Breakdown of the options used: as with the standard ping tool, the “-c” flag tells it we only want to send N count requests. In this example we are only sending one (1) request. Without this command line flag, hping (and ping) will ping non-stop until you stop it with ctrl-c. The next flag is “-S” which sets the SYN flag as by default it will not be set by hping. The “-p” flag tells hping we want to send our request to a specific port, in the above example we use port 80 which is the standard non-SSL webserver port. The last option is the hostname or IP address we want to interact with (www.google.com in our example.)

I am going to try something new and use a terminal utility named asciinema to record the examples above. Please let me know what you think: Watch a video of the above example made with asciinema!

Review: GL.iNet GL-AR750S-Ext Travel Router and EasyTether

I, along with my family, like to travel several times a year. The type of traveling varies – sometimes it is somewhere within a day or two driving distance and other times we take the a flight to somewhere more distant and stay longer. This is possible because I am a mostly (99% of the time) remote worker and can work from most locations so long as I have power for my laptop and accessories and high speed Internet. When I am stationary I try to use the local WiFi if possible, but if I am traveling or at a location that does not have suitable WiFi then I will use my Android smart phone (Verizon service) to fill the gap. Using my phone is fine for one device, but what if I need to connect more than one? When I’m at a hotel, do I really want to trust their WiFi? Well it turns out the answer to both of these questions is to pair my phone with a specialized router.

First some background: most phone carriers (at least in the US) allow hot-spots on the phone, but they come with a catch: you are limited, per month, in how much bandwidth you are allowed to consume. In Verizon-land it is somehow acceptable to cap this at 15GB per month which is a bit unrealistic in this day and age. The phone itself has unlimited data, but anything you wish to connect to the built-in hot-spot or otherwise tether to the phone is limited to 15GB per month. One day I got to thinking “so how can I tap into that unlimited side of the phone?” A bit of searching turned up a few options. Out of those options, I would say the top two are: PDANet+, which as been around for many years and EasyTether, a more recent addition. On the phone-side they are both paid apps on the Google Play Store, and while I own both, I have found EasyTether to work with more devices and it also seems to use closer to the phones full network speed.

What this looks like then is on my Android phone I use a paid app named EasyTether and on my laptop I use a free piece of software also named EasyTether. The creators of EasyTether provide client-side support for Linux, Windows, Mac, and some specialized devices. I personally have used it with Linux, Windows, and a few specialized routers. EasyTether can connect to the phone via Bluetooth or USB. The USB option is faster and will also trickle charge your phone. The Bluetooth option is good if you are unable to plug your phone in (i.e. you are also using it and having a cord is annoying) but are within 25 or so feet of your client-end (i.e. laptop), but Bluetooth 3 and 4 have a ~25 megabit/second limit. This means that you will run into a big bottleneck with Bluetooth while with USB you will probably top out your cell network bandwidth before you hit the 5gigabit/second limit of USB 3. These limits are of course on paper and other factors come into play.

My initial thought was to use a Raspberry PI and tether my phone to it, but that meant the access point was software which has never been very good in my experience. Then I got to thinking about specialized travel routers. After some searching and reading reviews I found the GL.iNet GL-AR750S-Ext Travel Router, hereafter referred to as the ar750s to keep things easy for me. The ar750s is a dual-band (2.4GHz and 5GHz) router, with external antennas, running OpenWRT (Linux-based) firmware and it has 3 gigabit Ethernet ports. The OpenWRT firmware can be customized in just about any way you could imagine. The ar750s has built-in OpenVPN client and server, dual flash, ac750, external storage (micro-SD), and has a bunch of included utilities. One of its features is the ability to connect to an existing WiFi as an extender, act as a broadband router, and the list goes on.

After I ordered the router from Amazon I expected to have to do a bit of hacking to make their pairing work, but it turns out both GL.iNet and EasyTether work together already. GL.iNet has a nice EasyTether guide. The only difference for me was the EasyTether version needed to be a newer (latest) version instead of the quite old one reference in the guide. If for some reason that guide is no longer available, what you need to do is extract the EasyTether file on your computer, find the ipk under “*\ar71xx\generic” and scp it to the router, ssh to the router as root (using whatever password you set in the web interface), run “opkg update” and “opkg install . Once you have the IPK installed, you need to run “easytether-usb” to set it up. Then edit /etc/config/network and add “config interface ‘wan’ \ option ifname ‘tap-easytether’ \ option proto ‘dhcp'” (where the “\” is, put a newline). Oh, and you will need to have USB debugging enabled on your phone.

This setup works pretty darn good, but requires ssh’ing into the router each time you want to bring the connection up or if it dies. So I wrote a simple SH script, available from my github. In case that dies, here it is in all its glory:

#!/bin/sh

# Version
version=0.04

# A simple script that checks for connectivity (including working DNS resolution)
# If no connectivity, reset tethering
#
# Requires easytether-usb to be installed and already setup/working
# Manual (one time) setting of the USB device ID of the tether device (TETHERDEV).

# Find the tethering device with lsusb. Example (Samsung/Verizon kids tablet):
# Bus 001 Device 013: ID 04e8:6860 Samsung Electronics Co., Ltd Galaxy (MTP)
TETHERDEV=04e8:6860
# 04e8:6860 is for Samsung’s USB identification (both Note 10 and kids tablet)

# Some highly available website to check (www.google.com is backed by lots of servers)
CHECKWWW=www.google.com

# How long before we check again?
SLEEPITOFF=60

while true; do
curl –connect-timeout 10 $CHECKWWW > /dev/null 2>&1

if [ $? != 0 ]; then
# No internet
echo “Network down, Houston we have a problem!” # FIXME: For debugging
# Reset the USB device
usbreset $TETHERDEV
# Wait a few seconds for device to be ready again
sleep 5s
# (re)start easytether-usb to make a connection
easytether-usb
else
# Internet working
echo “Network up, all good in the hood!” # FIXME: For debugging
fi
sleep $SLEEPITOFF
done


I put this file /usr/local/sbin and make sure it is executable. Then we need to edit /etc/rc.local (just before the ‘exit’) and add “/usr/local/sbin/fixnet.sh&” so it will start at boot. Be sure to change the TETHERDEV to match the USB ID of your phone (found with “lsusb”) in order for it to work. I use curl instead of ping, because ICMP packets are filtered/blocked in my testing.

Once you have everything up and running you only need to enable EasyTether USB on your phone via the app and plug it into the USB port on the router. The router is very easy to use and once configured I rarely need to do anything other than power it on. Speaking of power, the ar750s runs great from a battery pack. The battery pack that I use to charge my phone on the go will also power the ar750s for several hours.

So that covers mobile data (traveling) and places that have no Internet, but what about when I’m at a hotel or using some other network (be it WiFi or wired) that cannot be trusted? Well, for those incidents, I prefer to have my devices behind another layer of isolation via the ar750s. The ar750s supports being a router via Ethernet as well as acting as a WiFi repeater. If the hotel WiFi is not encrypted (often the case) I like to use OpenVPN whenever possible to close the gap. When I am using the router without my phone, the above script will keep trying to reset USB and connect EasyTether to a non-existent device. This has not caused any problems for me. However, it might be a problem if you try to use the USB port for something else – so be warned!