Containers—largely popularized by Docker—have profoundly changed the way development and operations teams test and deploy modern software. Containers help companies modernize by making it easier to scale and deploy applications, but containers have also introduced new challenges and more complexity by creating an entirely new infrastructure ecosystem.

Early adopters are now deploying thousands of container instances daily, and that’s a complexity of scale they have to manage. So how do they do it?

Enter the age of Kubernetes.

Originally developed by Google as an offshoot of its Borg project, Kubernetes is an open-source container orchestration platform designed to automate the deployment, scaling, and management of containerized applications. In fact, Kubernetes has established itself as the de facto standard for container orchestration and is the flagship project of the Cloud Native Computing Foundation, backed by key players like Google, AWS, Microsoft, IBM, Intel, Cisco, and Red Hat.

Sometimes referred to as “k8s,” Kubernetes makes it easy to deploy and operate applications based on a microservice architecture. It does so by creating an abstraction layer on top of a group of hosts, so that development teams can deploy their applications and let Kubernetes manage:

  • Controlling resource consumption by application or team
  • Evenly spreading application load across a host infrastructure
  • Automatically load balancing requests across the different instances of an application
  • Monitoring resource consumption and resource limits to automatically stop applications from consuming too many resources and restarting the applications again
  • Moving an application instance from one host to another if there is a shortage of resources in a host, or if the host dies
  • Automatically leveraging additional resources made available when a new host is added to the cluster
  • Easily performing canary deployments and rollbacks

See also: Docker vs. Kubernetes: It’s Not About One or the Other

OK, but why all the buzz? Why is Kubernetes so popular?

As more and more organizations move to microservice and container architectures, they’re looking for strong, proven platforms. DevOps practitioners are moving to Kubernetes for four main reasons:

1. Kubernetes helps you move faster. Indeed, Kubernetes allows you to deliver a self-service Platform-as-a-Service (PaaS) that creates a hardware layer abstraction for development teams. Your development teams can quickly and efficiently request the resources they need. If they need more resources to handle additional load, they can get those just as quickly, since resources all come from an infrastructure shared across all your teams.

kubernetes cloud diagramNo more filling out forms to request new machines to run your application. Just provision and go, and take advantage of the tooling developed around Kubernetes for automating packaging, deployment, and testing, such as Codefresh or Helm.

2. Kubernetes is cost efficient. Kubernetes and containers allow for much better resource utilization than hypervisors and VMs do. Because containers are so lightweight, they require less CPU and memory resources to run.

3. Kubernetes is portable. Kubernetes runs on Amazon Web Services (AWS), Microsoft Azure, and the Google Cloud Platform (GCP), and you can also run it on-premise. You can move workloads without having to redesign your applications or completely rethink your infrastructure, which lets you standardize on a platform and avoid vendor lock-in.

4. Cloud providers will even manage Kubernetes for you. As noted earlier, Kubernetes is currently the clear standard for container orchestration tools. It should come as no surprise, then, that major cloud providers now offer Kubernetes-as-a-Service platforms. Amazon EKS, Google Cloud Kubernetes Engine, Azure Kubernetes Service (AKS), Red Hat Openshift, and IBM Cloud Kubernetes Service all provide a full Kubernetes platform management, so you can focus on what matters most to you—shipping applications that delight your users.

So, how does Kubernetes work?

The central component of Kubernetes is the cluster. A cluster is made up of many virtual or physical machines that each serve a specialized function either as a master or as a node. Each node hosts groups of one or more containers (which contain your applications), and the master communicates with nodes about when to create or destroy containers. At the same time, it tells nodes how to re-route traffic based on new container alignments.

The following diagram depicts a general outline of a Kubernetes cluster:

[click to enlarge]

The Kubernetes master

The Kubernetes master is the access point (or the control plane) from which administrators and other users interact with the cluster to manage the scheduling and deployment of containers. A cluster will always have at least one master, but may have more depending on the cluster’s replication pattern.

The master stores the state and configuration data for the entire cluster in ectd, a persistent and distributed key-value data store. Each node has access to ectd and through it, nodes learn how to maintain the configurations of the containers they’re running. You can run etcd on the Kubernetes master or in standalone configurations.

Masters communicate with the rest of the cluster through the kube-apiserver, the main access point to the control plane. For example, the kube-apiserver makes sure that configurations in etcd match with configurations of containers deployed in the cluster.

The kube-controller-manager handles control loops that manage the state of the cluster via the Kubernetes API server. Deployments, replicas, and nodes have controls handled by this service. For example, the node controller is responsible for registering a node and monitoring its health throughout its lifecycle.

Node workloads in the cluster are tracked and managed by the kube-scheduler. This service keeps track of the capacity and resources of nodes and assigns work to nodes based on their availability.

The cloud-controller-manager is a service running in Kubernetes that helps keep it “cloud-agnostic.” The cloud-controller-manager serves as an abstraction layer between the APIs and tools of a cloud provider (for example, storage volumes or load balancers) and their representational counterparts in Kubernetes.


All nodes in a Kubernetes cluster must be configured with a container runtime, which is typically Docker. The container runtime starts and manages the containers as they’re deployed to nodes in the cluster by Kubernetes. Your applications (web servers, databases, API servers, etc.) run inside the containers.

Each Kubernetes node runs an agent process called a kubelet that is responsible for managing the state of the node: starting, stopping, and maintaining application containers based on instructions from the control plane. The kubelet collects performance and health information from the node, pods, and containers it runs and shares that information with the control plane to help it make scheduling decisions.

The kube-proxy is a network proxy that runs on nodes in the cluster. It also works as a load balancer for services running on a node.

The basic scheduling unit is a pod, which consists of one or more containers guaranteed to be co-located on the host machine and that can share resources. Each pod is assigned a unique IP address within the cluster, allowing the application to use ports without conflict.

You describe the desired state of the containers in a pod through a YAML or JSON object called a Pod Spec. These objects are passed to the kubelet through the API server.

A pod can define one or more volumes, such as a local disk or network disk, and expose them to the containers in the pod, which allows different containers to share storage space. For example, volumes can be used when one container downloads content and another container uploads that content somewhere else.

Since containers inside pods are often ephemeral, Kubernetes offers a type of load balancer, called a service, to simplify sending requests to a group of pods. A service targets a logical set of pods selected based on labels (explained below). By default, services can be accessed only from within the cluster, but you can enable public access to them as well if you want them to receive requests from outside the cluster.

Deployments and replicas

A deployment is a YAML object that defines the pods and the number of container instances, called replicas, for each pod. You define the number of replicas you want to have running in the cluster via a ReplicaSet, which is part of the deployment object. So, for example, if a node running a pod dies, the replica set will ensure that another pod is scheduled on another available node.

A DaemonSet deploys and runs a specific daemon (in a pod) on nodes you specify. They’re most often used to provide services or maintenance to pods. A daemon set, for example, is how New Relic Infrastructure gets the Infrastructure agent deployed across all nodes in a cluster.


Namespaces allow you to create virtual clusters on top of a physical cluster. Namespaces are intended for use in environments with many users spread across multiple teams or projects. They assign resource quotas and logically isolate cluster resources.


Labels are key/value pairs that you can assign to pods and other objects in Kubernetes. Labels allow Kubernetes operators to organize and select subsets of objects. For example, when monitoring Kubernetes objects, labels let you quickly drill down to the information you’re most interested in.

Stateful sets and persistent storage volumes

StatefulSets give you the ability to assign unique IDs to pods in case you need to move pods to other nodes, maintain networking between pods, or persist data between them. Similarly, persistent storage volumes provide storage resources for a cluster to which pods can request access as they’re deployed.

Other useful components

These Kubernetes components are useful but not required for regular Kubernetes functionality:

Kubernetes DNS

Kubernetes provides this mechanism for DNS-based service discovery between pods. This DNS server works in addition to any other DNS servers you may use in your infrastructure.

Cluster-level logs

If you have a logging tool, you can integrate it with Kubernetes to extract and store application and system logs from within a cluster, written to standard output and standard error. If you want to use cluster-level logs, it’s important to note that Kubernetes does not provide native log storing; you must provide your own log storage solution.

New Relic can support your Kubernetes journey

To fully understand what’s going on in your environment, you need to see into all its layers, including inside your containers. That means holistic, application-centric, and infrastructure-centric monitoring. Monitoring application performance in Kubernetes is important, but you also need visibility into your Docker and Kubernetes infrastructure.

The New Relic Infrastructure on-host integration for Kubernetes provides deep monitoring of your Kubernetes layer. This integration collects metrics that monitor data and metadata for nodes, Namespaces, Deployments, ReplicaSets, Pods, and containers, so you can fully monitor the frontend and backend applications and hosts running in your clusters. Expect total visibility, alerting, and dashboards for all Kubernetes entities that live in between your applications.

Using New Relic to take advantage of the power of Kubernetes is essential for modern software companies looking to move faster with confidence. Learn how to get started monitoring containers in your Kubernetes environment in the New Relic documentation.

Ali Gerrard is a product marketing manager at New Relic. View posts by .

Interested in writing for New Relic Blog? Send us a pitch!