Kubernetes vs Mesos vs Swarm

If you're reading this blog, you might ask yourself what container orchestration engines are, what problems they solve, and how the different engines distinguish themselves. Read on for a high-level overview of Kubernetes, Docker Swarm, and Apache Mesos, as well as a few of their notable similarities and differences.

Container orchestration engines

Cloud orchestration is a relatively new category of software tools designed to help IT organizations manage interconnections and interactions between disparate systems in increasingly complex cloud environments. While definitions vary, Kubernetes, Docker Swarm, and Apache Mesos are DevOps tools known as Container Orchestration Engines (COEs). COEs are software platforms for managing containers and automating the deployment, scaling, and operations of containers across a cluster of nodes. COEs provide a way to deploy, manage, and scale applications quickly and easily, and provide an abstraction layer between pools of resources, and the application containers that run on those resources.

Along with containers, the major problem COEs solve is how to take multiple discrete resources in the cloud or data center and combine them into a single pool, onto which various applications can be deployed. These applications can range from simple three-tier web architectures to large-scale data ingestion and processing, and everything in between.

Each of these tools provides different feature sets, and vary in their maturity, learning curves, and ease of use. Some high-level features they share are:

• Container scheduling, which consists of performing such functions as starting and stopping containers; distributing containers among the pooled resources; recovery of failed containers; rebalancing containers from failed hosts to healthy ones, and scaling applications via containers, whether manually or automatically.

• High availability of either the application and containers, or the container orchestration tool itself.

• Health checks to determine container or containerized application health.

• Service discovery, which is used to determine where various services are located on a network in a distributed computing architecture.

• Load Balancing requests, whether internally generated in a cluster, or externally from outside clients.

• Attaching various types (network, local) of storage to containers in a cluster.

Note this list is by no means exhaustive, and is meant to be representative of some of the high-level services provided by COEs. It's also worth mentioning that while each of the tools discussed here will perform these functions to a certain degree, the implementations can vary quite a bit.

Kubernetes capabilities

Kubernetes (also known as "K8s") was first released in June 2014, and is written in Go. Translated from Ancient Greek, the word Kubernetes means “Helmsman.” Kubernetes is an open source system for automating the deployment, scaling, and management of container instances.

Docker is the most ubiquitous container orchestration tool currently supported by Kubernetes, but CoreOS rkt (pronounced "rocket") is also supported.

You can read more about Docker and Kubernetes.

In terms of features, Kubernetes probably has the most natively integrated of the three options examined in this blog. Very widely used, Kubernetes has a large community behind it. Google Cloud Platform (GCP) uses Kubernetes for its own Container as a Service (CaaS) offering, called Google Container Engine (GKE). There are various other platforms that support Kubernetes, including Amazon EKS, Amazon ECS, Red Hat OpenShift, and Microsoft Azure.

Kubernetes uses a YAML-based deployment model. In addition to scheduling containers on hosts, Kubernetes provides many other features. Major features include built-in auto scaling, load balancing, volume management, and secrets management. In addition, there is a web UI to manage and troubleshoot the cluster. With these features included, Kubernetes often requires less third-party software than Swarm or Mesos.

Also differentiating Kubernetes from Docker Swarm and Mesos is the concept of a Kubernetes pod, a group of containers scheduled together to make up a "service," in Kubernetes terminology.

It is possible to configure the master as a high-availability Kubernetes cluster, but this is considered an advanced use case, and this is not as well supported as single node master installations.

Kubernetes has a somewhat steeper learning curve, and can take more effort to configure than Docker Swarm. Due in part to its tighter integration of features, Kubernetes is sometimes considered more opinionated than the other two engines discussed here.

Docker Swarm capabilities

Docker Swarm is Docker's native container orchestration engine. Originally released in November 2015, it is also written in Go. Swarmkit is the Docker native version of Swarm, included as of version 1.12, which is the recommended version of Docker if you want to use Swarm.

Swarm is tightly integrated with the Docker API, making it well-suited for use with Docker. The same primitives that apply to a single host Docker cluster are used with Swarm. This can simplify managing Docker container infrastructures, as there is no need to configure a separate orchestration engine, or relearn Docker concepts to use Swarm.

Like Kubernetes, Swarm has a YAML-based deployment model using Docker Compose. Other noticeable features include auto-healing of clusters, overlay networks with DNS, high availability through multiple masters, and network security using TLS with a Certificate Authority.

Swarm does not support native auto scaling or external load balancing. Scaling must be done manually or through third-party solutions. Along the same vein, Swarm includes ingress load balancing, but external load balancing is done through a third-party load balancer, such as AWS ELB. Also notable is a lack of a web interface for Swarm.

Mesos capabilities

Apache Mesos version 1.0 was released in July 2016, but it has roots back to 2009, when PhD students initially developed it at UC Berkeley. Unlike Swarm and Kubernetes, Mesos is written in C++.

Mesos is somewhat different than the first two mentioned here, in that it takes more of a distributed approach to managing data center and cloud resources. Mesos can have multiple masters, which use Zookeeper to keep track of the cluster state among the masters and form a high-availability cluster.

Other container management frameworks can be run on top of Mesos, including Kubernetes, Apache Aurora, Chronos, and Mesosphere Marathon. In addition, Mesosphere DC/OS, a distributed data center operating system, is based on Apache Mesos.

This means Mesos takes a more modular approach to container management, allowing users to have more flexibility in the types of applications, and the scale on which they can run.

Mesos can scale to tens of thousands of nodes, and has been used by the likes of Twitter, Airbnb, Yelp, and eBay. Apple even has its own proprietary framework based on Mesos called Jarvis, which is used to power Siri.

Some features available in Mesos worth mentioning are support for multiple types of container engines, including Docker and its own "Containerizer," as well as a web UI, and the ability to run on multiple OSes, including Linux, OS X, and even Windows.

Due to its complexity and flexibility, Mesos has a steeper learning curve than Docker Swarm. But, that same flexibility and complexity are also strengths that allow companies like Yelp and eBay to use Mesos to manage large-scale applications.

Logging and container orchestration tools

Container orchestration tools typically generate logs related to their internal operations and the status of containerized applications. These logs usually cover deployment, scaling, maintenance tasks, performance metrics like resource usage or latency, and error reports from failed deployments/tasks.

Depending on the size and activity of the container cluster, a container orchestration tool can generate hundreds to thousands of log entries per hour. You can check this by using monitoring and logging tools available for the platform. For example, Kubernetes clusters have the Container Management Interface (CMI) to capture and store logs from within the pod or node level. Similarly, you can use Graphite/InfluxDB with Apache Mesos to collect resource usage metrics across your applications throughout its cluster setup.

Because a container orchestration tool generates so many log entries, it's essential to use a centralized logging system with proper aggregation to manage these logs better and quickly identify any issues within your containers. Log analytics is critical in managing the orchestration of containers. Logs provide insight into what happens within a container cluster, such as which processes are running, resource utilization metrics, and errors from failed deployments. Advanced log analytics tools can integrate these insights with alerting systems, allowing developers to quickly identify and address issues within their container clusters for improved performance and stability.

Apart from log analytics, other critical factors for container management include automated deployments through CI/CD pipelines, scalability, resource optimization strategies to manage hosting costs, consistent monitoring and alerting of application performance metrics, and best practices around container security. All these elements help ensure optimal performance when running applications in a distributed environment, which is the focus of most container orchestration tools.

Choosing the right COE for your needs

As you can see, cluster management and the associated tools can deepen quickly. We've only just touched upon some features and use cases for the different container orchestration tools presented here. Each has its own strengths and weaknesses, and a solid understanding of your own use case will dictate which is most suitable for your application implementation.

With that said, if you're just looking to get up and running and test out using an orchestration engine, then Docker Swarm is probably a good choice. When you're ready to delve further into the subject, or possibly deploy something leaning toward industrial grade, look to Kubernetes. If flexibility and massive scale are your goals, consider Apache Mesos.

How can Sumo Logic help?

Learn how Sumo Logic can help you manage your container orchestration engines with a modern log management and analytics solution to improve your monitoring and troubleshooting, increase your security posture, and gain key business insights.

Additional resources

• Read our blog on How to Design a Microservices Architecture with Docker Containers

• Read our blog on Logging and Monitoring Kubernetes

• Read our blog on Container Orchestration with Mesos Marathon

• Learn more on the Docker App for Sumo Logic

• Learn more on the Kubernetes App for Sumo Logic