Let’s start by finding the current logging driver:
$ docker info | grep “Logging Driver”
Logging Driver: json-file
Currently, the default logging driver is set to json-file. If we want to use journald or Splunk as the default logging driver, we must configure the default logging driver in the daemon.json file.
Edit the /etc/docker/daemon.json file using an editor of your choice. If you’re using vim, run the following command:
$ sudo vim /etc/docker/daemon.json
Add the log-driver entry to the daemon.json configuration file:
{
“log-driver”: “journald”
}
Then, restart the Docker service:
$ sudo systemctl restart docker
Check the status of the Docker service:
$ sudo systemctl status docker
Now, rerun docker info to see what we get:
$ docker info | grep “Logging Driver”
Logging Driver: journald
Now that journald is the default logging driver, let’s launch a new NGINX container and visualize the logs:
$ docker run –name nginx-journald -d nginx 66d50cc11178b0dcdb66b114ccf4aa2186b510eb1fdb1e19d563566d2e96140c
Now, let’s look at the journald logs to see what we get:
$ sudo journalctl CONTAINER_NAME=nginx-journald
…
Jun 01 06:11:13 99374c32101c fb8294aece02[10826]: 10-listen-on-ipv6-by-default.sh: info:
Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
…
Jun 01 06:11:13 99374c32101c fb8294aece02[10826]: 2023/06/01 06:11:13 [notice] 1#1: start
worker process 30
…
We can see the logs in the journal.
Similarly, we can configure the Splunk logging driver to send data to Splunk for analytics and visualization. Let’s have a look.
Edit the /etc/docker/daemon.json file using an editor of your choice. If you’re using vim, run the following command:
$ vim /etc/docker/daemon.json
Add the log-driver entry to the daemon.json configuration file:
{
“log-driver”: “splunk”,
“log-opts”: {
“splunk-token”: “<Splunk HTTP Event Collector token>”,
“splunk-url”: “<Splunk HTTP(S) url>”
}
}
Then, restart the Docker service:
$ sudo systemctl restart docker
Check the status of the Docker service:
$ sudo systemctl status docker
Now, rerun docker info to see what we get:
$ docker info | grep “Logging Driver”
Logging Driver: splunk
Since Splunk is now the default logging driver, let’s launch a new NGINX container and visualize the logs:
$ docker run –name nginx-splunk -d nginx dedde062feba33f64efd89ef9102c7c93afa854473cda3033745d35d9065c9e5
Log in to your Splunk instance; you will see the Docker logs streaming. You can then analyze the logs and create visualizations out of them.
You can also have different logging drivers for different containers, and you can do so by overriding the defaults by passing the log-driver and log-opts flags from the command line. As our current configuration is Splunk, and we want to export data to a JSON file, we can specify log-driver as json-file while running the container. Let’s have a look:
$ docker run –name nginx-json-file –log-driver json-file -d nginx 379eb8d0162d98614d53ae1c81ea1ad154745f9edbd2f64cffc2279772198bb2
To visualize JSON logs, we need to look into the JSON log directory – that is, /var/lib/docker/ containers/<container_id>/<container_id>-json.log.
For the nginx-json-file container, we can do the following:
$ cat /var/lib/docker/containers\
/379eb8d0162d98614d53ae1c81ea1ad154745f9edbd2f64cffc2279772198bb2\
/379eb8d0162d98614d53ae1c81ea1ad154745f9edbd2f64cffc2279772198bb2-json.log
{“log”:”/docker-entrypoint.sh: /docker-entrypoint.d/ is not
empty, will attempt to perform configuration\n”,”stream”:”
stdout”,”time”:”2022-06-01T06:27:05.922950436Z”}
…
{“log”:”/docker-entrypoint.sh: Configuration complete; ready
for start up\n”,”stream”:”stdout”,”time”:”2023-06-01T06:27:
05.937629749Z”}
We can see that the logs are now streaming to the JSON file instead of Splunk. That is how we override the default log driver.
Tip
In most cases, it is best to stick with one default logging driver so that you have one place to analyze and visualize your logs.
Now, let’s understand some of the challenges and best practices associated with Docker logging.