From a conceptual point of view, there is no difference between container monitoring and the traditional method. You still need metrics, logs, health checks, and service discovery. These aren’t things that are unknown or haven’t been explored before. The problem with containers is the abstraction that they bring with them; let’s look at some of the issues:
- Containers behave like mini virtual machines; however, in reality, they are processes running on a server. However, they still have everything to monitor that we would in a virtual machine.
A container process will have many metrics, very similar to virtual machines, to be treated as separate entities altogether. When dealing with containers, most people make this mistake when they map containers to a particular virtual machine.
- Containers are temporary, and most people don’t realize that. When you have a container and it is recreated, it has a new IP. This can confuse traditional monitoring systems.
- Containers running on clusters can move from one node (server) to another. This adds another layer of complexity as your monitoring tool needs to know where your containers are to scrape metrics from them. This should not matter with the moremodern, container-optimized tools.
Prometheus helps us address these challenges as it is built from a distributed application’s point of view. To understand this, we’ll look at a hands-on example. However, before that, let’s install Prometheus on a separate Ubuntu 22.04 Linux machine.
Installing Prometheus
Installing Prometheus consists of several steps, and for simplicity, I’ve created a Bash script for installing and setting up Prometheus on an Ubuntu machine.
Use the following commands on a separate machine where you want to set up Prometheus:
$ git clone https://github.com/PacktPublishing/Modern-DevOps-Practices-2e.git \ modern-devops
$ cd modern-devops/ch3/prometheus/
$ sudo bash prometheus_setup.sh
To check whether Prometheus is installed and running, check the status of the Prometheus service using the following command:
$ sudo systemctl status prometheus
prometheus.service – Prometheus
Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; vendor preset:
enabled)
Active: active (running) since Tue 2023-06-01 09:26:57 UTC; 1min 22s ago
As the service is Active , we can conclude that Prometheus has been installed and is running successfully. The next step is configuring the Docker server to enable Prometheus to collect logs from it.
Configuring cAdvisor and the node exporter to expose metrics
Now, we’ll launch a cAdvisor container on the machine running Docker to expose the metrics of the Docker containers. cAdvisor is a metrics collector that scrapes metrics from containers. To launch the container, use the following command:
$ docker run -d –restart always –name cadvisor -p 8080:8080 \ -v “/:/rootfs:ro” -v “/var/run:/var/run:rw” -v “/sys:/sys:ro” \ -v “/var/lib/docker/:/var/lib/docker:ro” google/cadvisor:latest
Now that cAdvisor is running, we need to configure the node exporter to export node metrics on the Docker machine. To do so, run the following commands:
$ cd ~/modern-devops/ch3/prometheus/
$ sudo bash node_exporter_setup.sh
Now that the node exporter is running, let’s configure Prometheus to connect to cAdvisor and the node exporter and scrape metrics from there.