Docker log management - definition & overview

Docker log management includes logging drivers in the platform to give you access to performance data. It also enables logging driver plugins for those who would like to integrate another logging tool. Using Docker's logging capabilities, you can view log data at both the daemon process and container levels.

Log management is important for making development decisions, conducting root cause analysis when something goes wrong, demonstrating security compliance, and much more.

When working with containerized apps in Docker, you need more data (compared to hardware-centric apps) to get to the heart of an issue. Here are some fundamental terms and definitions to contextualize Docker log management.

Docker Compose: Docker Compose is a tool for defining and running multi-container Docker applications. Docker Compose defines the services that make up your app in docker-compose. Docker Compose applies many rules declared within a single docker-compose.yml configuration file.

Docker Swarm: A Docker Swarm is a container orchestration tool running the Docker application. It's configured to join together in a cluster.

Docker Engine: Docker Engine is the underlying client-server technology that builds and runs containers using Docker's components and services. It comprises the Docker daemon, a REST API and the CLI that talks to the Docker daemon through the Docker API, a RESTful API accessed by an HTTP client such as wget or curl, or the HTTP library, which is part of most modern programming languages.

Docker daemon logs: Docker daemon logs are the logs Docker generates. You will need those to debug errors in the Docker engine.

For all of the advantages a Docker container offers DevOps teams, they also change how to complete log management. In Docker, effective logging includes logging events from the host OS, the application, and the Docker platform.

There are several methods for capturing logs in a Docker environment, each with advantages. Which works best for you depends on your application environment and unique needs.

Application-based logging

An application inside a Docker container can use a logging framework to manage the logging process. This allows a user to bypass Docker and the host OS, logging events to a remote server and offering more control over the event.

However, it also adds significant overhead to the app processes. This might be an effective method if you can benefit from using the app's logging framework and want to avoid adding log functionality to the host.

Data volumes for log retention

When a container shuts down, it loses its data. To retain log events, they need to be sent elsewhere. This can be an external logging service, or it can be a data volume, a directory within the container linked to the host machine.

Saving log events to a data volume helps ensure that log data doesn't get lost when the container shuts down. It also makes the log data available for sharing with other containers.

Docker logging driver

Docker includes logging drivers on their platform. These drivers offer performance advantages over other methods because they bypass the need to read or write from a log file. Instead, the logging driver reads events straight from the container's output and forwards them to the host. This may not, however, be a reliable method for many users, as it does not allow for log parsing.

Detailed logging container

The Docker logs command shows information logged by a running container. In Docker, primarily, there are two types of log files. The Docker daemon logs provide insight into the Docker service's overall status. The Docker container logs cover all the logs related to a particular container. You can deploy a dedicated container to manage log events within the Docker environment. This eliminates dependency on a host for log events and scales automatically without requiring additional configuration. The Docker containers generate a Docker container log. They need to be collected directly from the containers. A Docker image is a file used to execute code in a Docker container. Docker images act as a set of instructions to build a Docker container. The docker logs command shows information logged by a running container.

Suppose you are using Docker for containerized app development. In that case, you may want to start exploring your options for logging from the platform and using log data to enhance your apps and the development process. Many Sumo Logic customers have adopted the Docker platform. Some have offered insights into how they manage their logs, using Sumo Logic to automate log analysis and make the most of their data. Sumo Logic provides Docker monitoring.

Check out these methods we've compiled for collecting Docker logs and pushing them to the Sumo Logic platform. These techniques help you get ideas about how to work with logs in your Docker environment.