Building a network monitor

As so many of us I’m currently working from home. My internet at home is most of the time pretty ok, but it can get very flaky, as in, speed drops, it can’t connect to certain hosts etc.

However, every time I report this to my ISP, I get the same routine: restart, unplug, hold for 30 seconds, pray to Ubuntu and other ridiculous demands that make no sense because I didn’t change anything in the hardware (I didn’t even move it).

So to prove my point, I decided to refurbish one of my Raspberry Pis as a network monitor using InfluxDB to collect the results and Grafana to spice up the charts

So to do this you obviously need

  • A raspberry PI
  • An internet connection (doh!)
  • A time series database InfluxDB
  • Some python3 scripts to collect the data.
  • A graphing tool Grafana

Setting up the PI

Following one of the many tutorials, I managed to setup a Pi in headless mode. Aka, it’s plugged into power and ethernet and using my router it’s configured to always be at a certain IP address. It has SSH running so I can connect to it from my laptop. This is the basic setup and requires the minimal amount of wires.

I didn’t set it up wireless as I want to be as close as possible to the router so it literally sits next to the router plugged in in one of the available ethernet ports of the router. This should give me the optimal network so they can’t blame my wifi.

First thing you do obviously is to change the password for the pi user to something other than raspberry. The thing will be hooked to the internet and you can never be safe enough.

Once I log into it using ssh, I first upgrade it to the latest versions, so I need to run a few commands: 

sudo apt update
sudp apt upgrade

Go grab a cup of coffee as this might take a while if you haven’t been updating that Pi regularly. You have to be on at least Stretch or Buster for InfluxDB. You can’t go from Jesse to Buster in one go, so if you want to do that, you have some time to waste.

If you want to be quick about this and you don’t care about your existing image, it’s faster to just grab the latest image using the imager from https://www.raspberrypi.org/downloads/

If you want a graphical UI, you can enable VNC simply by typing sudo raspi-config and select the Interfacing Options, VNC, enable. This should install the vncserver and configure it. When it’s done, it should be running on port 5900. Obviously, I have ssh enabled as that’s the main way to connect to it.

Installing InfluxDB

InfluxDB is not a standard install so we will need to get it from another source than the default apt repositories. I took my cue from https://pimylifeup.com/raspberry-pi-influxdb/

wget -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -

All on one line. This will download the key and pass it into apt-key for further use. As I am on buster, I can use:

echo "deb https://repos.influxdata.com/debian buster stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

Obviously, replace buster with stretch for stretch. Now we’ve added a new repo, we can again do ‘sudo apt-get update’ and then ‘sudo apt install influxdb

And to make sure influxdb runs on boot:

sudo systemctl unmask influxdb
sudo systemctl enable influxdb
sudo systemctl start influxdb

our next step here is to setup a database in influx, so we must be able to login, for that, we need the client.

sudo apt install influxdb-client

And then you can just type `influx`. First we’ll be setting up authentication (replace your password with what you want)

CREATE USER admin WITH PASSWORD '<password>' WITH ALL PRIVILEGES

And then we’re going to make this user the only admin by setting up authentication by editing the influxdb.conf file:

so exit out of the client and back on console, type ‘sudo nano /etc/influxdb/influxdb.conf‘ and Find the [http] section and under there add:

auth-enabled = true
pprof-enabled = true
pprof-auth-enabled = true
ping-auth-enabled = true

Then we need to restart the database by typing ‘sudo systemctl restart influxdb

Now, if we want to connect we need to type:

Influx -username admin -password ‘’

By passing in the empty string influx will prompt for the password. This keeps it out of your history file. Though, I doubt that your pi will be setup to host many users so you might just not give much for this.

Ok, so now we’re going to setup another user for it’s own database, I called mine netmonitor and the database the same.

create database netmonitor
create user “netmonitor” with password ‘<again your choice>’
grant all on “netmonitor” to “netmonitor”

Take note that the password requires single quotes!

Great, InfluxDB is installed. Now it’s time to collect the data

 

The Python bits

At first I started out with two scripts, one that used speedtest-cli (I based this one on prior art from https://pimylifeup.com/raspberry-pi-internet-speed-monitor/) and another one that simply connects to Google’s DNS server at 8.8.8.8 with a 3 seconds timeout to monitor if the internet is up or down. Later I added a 3rd that captures the output of pings to various sites I was interested in (at some point I will probably make that configurable)

Obviously it’s fun to paste a lot of Python code here, but you can also just check out the repository here: https://github.com/siriousje/speedtest because over time I will probably add some more scripts.

So, the easiest way to get the code is to login to your Pi, and if for some reason you don’t want to type sudo before each command, do some root magic like 'sudo su -‘ and become root.

Clone this repository under /opt so that it creates /opt/speedtest Then make sure to install all requirements: sudo pip3 install -r requirements.txt\

Copy config.yml.sample to config.yml and update the values for your installation, then add the scripts to crontab using sudo crontab -e

it should look something like

*/5 * * * * python3 /opt/speedtest/speedtest-influx.py
*/1 * * * * python3 /opt/speedtest/internet-influx.py
*/1 * * * * python3 /opt/speedtest/latency-influx.py

Change the numbers to what you want, I run the speedtest only once per 5 minutes.

You can also use environment variables to override your config, but I wouldn’t really bother 🙂

Anyway, this is it, it should now be adding data to your InfluxDB. You can check this by logging into Influx as the netmonitor user by typing

influx -database "netmonitor" -username "netmonitor" -password "what you did"

and then as a normal database, `select * from internet_latency` to see the data flowing in.

Setting up Grafana

Congratulations! You made it to the sexy bits, graphs! This is what we came here for! Let’s go!

Depending on your PI, you may need to find the correct version of Grafana for your architecture. There is a difference between the Pi v1 and Pi Zero v.s. the Pi 2 and up. I use a pi3, so I go with 

wget https://dl.grafana.com/oss/release/grafana_6.6.1_armhf.deb
sudo dpkg -i grafana_6.6.1_armhf.deb

If you were using a pi-zero you should use this:

wget https://dl.grafana.com/oss/release/grafana-rpi_6.6.1_armhf.deb
sudo dpkg -i grafana-rpi_6.6.1_armhf.deb

The small difference there is the -rpi. And again we want this up and running after boot, so here we go:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable grafana-server
sudo /bin/systemctl start grafana-server

And then we go to the webinterface (as my pi is fixed to 192.168.1.199, the service is running on port 3000) so for me it is at http://192.168.1.199:3000/login and change the password to something you fancy

Next step is to configure the datasources!

Grafana already suggest the defaults for InfluxDB, but you still have to type it in the field (it’s a placeholder…) You want to keep localhost because it’s all on that same Pi.

Then obviously, fill out the login details for the datasource. So you need to setup the basic auth and the InfluxDB details:

And then you can test and save your datasource. If it’s green it’s good to go. Great, now we’re going to create a dashboard and add our statistics. So click on the plus, then add dashboard and then we choose add Query

Then we can basically select from (I started with internet_speed), selected field pin, changed mean() to distinct() by clicking on the plus behind it and selecting Aggregate/distinct() and then we can give it an alias whatever you want:

Now if you are using tags, like for the internet_latency stat where we use the tags to collect the hosts, you can add tag(host) to the group by setting and it will plot all hosts over each other

Anyway, configure your dashboard anyway you want. Mine ended up looking like this:

Which gives me a clear view that my internet is capped on the top, but also has a lot of drops in speed and even regular timeouts getting connected to the nameserver.

Anyway, with this data at least I have something to tell my ISP that it’s not just me 🙂