Monitoring your Infrastructure with Prometheus

Prometheus is an open source, next-generation monitoring and time series system that collects metrics from agents running on target hosts, storing the collected data onto the Prometheus server to be analysed for problem diagnosis.

In this series of posts, we will first install and configure Prometheus, then we look at the different exporters and visualising Prometheus data in Grafana.

Prometheus Installation

There are a couple of different methods to install Prometheus. For the purpose of this tutorial, we will install Prometheus in a Docker container running on an Ubuntu 18.04 server.

First, install Docker in your system and enable it.


sudo apt-get install docker.io
sudo systemctl start docker
sudo systemctl enable docker


Then, create this file tree in /srv/docker:


root@demo:/srv/docker# find prometheus
prometheus
rometheus/prometheus.yml
rometheus/data


Configuring Prometheus

Note that /srv/docker/prometheus/data needs to have permissions that will allow the container to write, so you need to ensure write permissions are enabled for that directory. The Prometheus configuration file can be found here: srv/docker/prometheus/prometheus.yml

We will use asimple shell script to start Prometheus. Once it has been started you won’t need to start it again because its setup as a daemon. This means Docker will restart it on boot.

Here is the shell script:


root@demo:/srv/docker# cat prometheus.sh
#!/bin/bash
docker run \
-d \
--restart always \
--name prometheus \
-p 9090:9090 \
-h prometheus \
-v /srv/docker/prometheus:/prometheus-data \
prom/prometheus --config.file=/prometheus-data/prometheus.yml --storage.tsdb.path=/prometheus-data/data --storage.tsdb.retention=3650d --web.enable-lifecycle


Verify that the Prometheus container is running by using Docker ps -a command.


root@demo:/srv/docker# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
be6ba6bddffe prom/prometheus "/bin/prometheus --c…" 2 minutes ago Up 2 minutes 0.0.0.0:9090->9090/tcp prometheus


It should be now available on port 9090. Open this port if it’s blocked by your firewall.

Prometheus expects to monitor things over HTTP and expects the exporters to be embedded into the application being monitored. The Prometheus community manages a list of well-known port numbers on GitHub, but here’s a list of the most useful ones:

Prometheus itself has a dashboarding and alerting language, but Grafana is much nicer and it has prebuilt dashboards so we’re going to use that instead.

In the next posts, we will configure an exporter and view the metrics of different nodes within Grafana.

Want to learn more about Monitoring with Prometheus and Grafana? Check out our 2 day Introduction to Monitoring course.