Gain Full Visibility into Microservices Architectures Using Kubernetes with Sumo Logic and Amazon EKS

The following blog is a collaborative piece from Sumo Logic and AWS. Special thanks to co-authors Anoop Sunke, AWS partner solutions architect and Frank Reno, solutions architect at Sumo Logic for their joint contributions and expert technical insight.

What is EKS

Amazon Elastic Container Service for Kubernetes (Amazon EKS) is a managed service that makes it easy for you to run Kubernetes on AWS without needing to install and operate your own Kubernetes clusters.

With Amazon EKS you get a highly-available, and secure Kubernetes control plane without needing to worry about provisioning, upgrades, or patching. Amazon EKS is certified Kubernetes conformant so you can use all existing plugins and tooling from the Kubernetes community. Any application running on any standard Kubernetes environment is fully compatible.

How Can Sumo Logic Help?

Container orchestration tools like Kubernetes are designed to increase flexibility and agility when using containers to deploy your distributed microservice architecture. Amazon EKS simplifies this lifecycle by removing the overhead of having to manually monitor and manage the Kubernetes Control Plane. However, comprehensive and reliable monitoring of your containerized applications running in Amazon EKS is critical to prevent operational blind spots.

While containers help you create fault-tolerant distributed microservices, they also introduce some challenges for monitoring. The ephemeral nature of containers along with the amount of data generated can make it difficult to know the state of your Kubernetes clusters and the applications running inside.

Sumo Logic is a company born in the Amazon Web Services cloud, and our strategic partnership with AWS allows us to further focus on giving our customers a flexible and tailored solution to help them manage and secure their modern applications in the cloud. Sumo Logic provides you with complete visibility into your containerized apps, giving developers the necessary insight into all applications running in EKS, ensuring that those apps are always available to customers.

Cluster Administrators can monitor resource usage for the EKS clusters, ensuring developers have the resources they need for the applications. Security teams can also monitor for any malicious activity across both the Amazon EKS managed service, nodes, containers and pods, and the custom applications running on them.

In this article, we’ll walk you through how to take advantage of the Sumo Logic Amazon EKS App to monitor your EKS environments and turn that log data into valuable Kubernetes performance metrics to optimize your operations.

Monitoring EKS

By managing the Kubernetes Control Plane, Amazon EKS lets you focus on monitoring your applications running inside of it. To effectively monitor your EKS clusters, you need visibility into the application logs and container metrics. Sumo Logic leverages some of the powerful Kubernetes abstractions in order to make it as easy as possible to bring data into our platform. Specifically, we developed an open source FluentD plugin that we leverage for Kubernetes to pull in all application logs from containers.

We also leverage Heapster, a cluster-wide aggregator to monitor resource usage for your cluster, nodes and pods running in Kubernetes. The following diagram shows the overall architecture of how Sumo Logic brings in data from Amazon EKS. There are three primary pieces of data we collect: container logs, container metrics and cluster state information.

Container Logs

At Sumo Logic, we have created an open source FluentD plugin specifically for capturing logs from the

applications running inside your pods. In Kubernetes, the log files from the pods are written to the file system of the node that the pod is running on. Our plugin mounts the file system of the node and uses the FluentD in_tail plugin to read the logs files. Since we run this plugin as a Daemonset, it automatically deploys to all the nodes in the cluster, bringing the logs for all of your pods into your cluster.

We enrich the logs with metadata from Kubernetes, ensuring we capture the namespace and labels of your pod. This gives you access to all the same metadata you have labeled your pods with and provides continuity when looking at all log data. The logs are then sent from FluentD to Sumo Logic via an HTTP source. An HTTP source is part of a hosted collector that we maintain for you. We also provide you with a unique endpoint that you can use to POST your logs to Sumo Logic.

Container Metrics

Kubelet is the process that runs on every node in your Kubernetes cluster and is the component that

communicates with the Control Plane. CAdvisor is bundled with Kubelet and collects the metrics for the containers. Kubelet queries CAdvisor for this information, however it is not designed for long-term storage and only keeps that data in memory for a brief period. Heapster is what you use for long-term storage. Heapster gets the metrics from Kubelet and can export that data to various sinks that are designed to retain the data for long periods of time.

Sumo Logic leverages the Graphite sink from Heapster which translates the metrics to the Graphite format and then sends them to a Graphite receiver. The Sumo Logic collector is a lightweight agent that can be used to ingest log and metrics data from a variety of data sources. It can be run locally on the host, and also as a container. To collect container metrics from Heapster, we simply create the Sumo Logic collector with a Graphite source.

The collector is run as a deployment, so we can have at least two replicas for failover, and runs a Graphite source to receive Graphite data. There is a Kubernetes service so Heapster can send the metrics to the Sumo Logic collector. Heapster can then be configured with a Graphite sink that points to that Kubernetes service, which will send all of the cluster level, node and pod metrics.

Cluster State

The final bit of important information to monitor is cluster state information that we can get from the Kubernetes API. This provides visibility into node and pod health and the current state of what is running inside the Kubernetes cluster. We have created a simple script that retrieves this data from the Kubernetes API and then sends it to Sumo Logic via an HTTP source, just like FluentD does for the container logs. This script can be run as a CronJob in Kubernetes, delivering the data to Sumo Logic at any desired cadence.

Sumo Logic Amazon EKS App

Once we have all that data in Sumo Logic, with just a few clicks we can install our Amazon EKS app, giving you instant visibility into your cluster and the applications running in it. The app provides multiple dashboards built to help you manage your cluster and ensure everything is running as expected. The app quickly informs you of the state of your Kubernetes cluster, nodes and pods.

Sumo Logic Dashboard Displaying Kubernetes Overview

Sumo Logic Dashboard Displaying Kubernetes Node Overview

Sumo Logic Dashboard Displaying Kubernetes Pod-level Metrics

Setup EKS Collection For Sumo Logic

We have pre-packaged all the configuration you need to get visibility into your Amazon EKS clusters. Simply follow our help documentation to collect all your data and install the Sumo Logic app for Amazon EKS!

Future Updates

Many Kubernetes users may be aware that Heapster is going to be deprecated in a future Kubernetes release. However, we are hard at work on a revamped metrics collection that will provide even more detail on what is happening inside of Kubernetes. The collection process will enable customers to ingest the Prometheus formatted metrics that Kubernetes exposes into Sumo Logic. This includes support for the container, kube-state and the control plane metrics. This same data will be available for Kubernetes clusters powered by Amazon EKS. Keep an eye on the Sumo Logic blog for more on that soon.

Additional Resources

• Sumo Logic for Amazon Elastic Container Service for Kubernetes (EKS) landing page
• Comparing Kubernetes Services on AWS vs. Azure vs. GCP blog
• Monitoring Kubernetes-powered Apps with Sumo Logic blog
• Sign up for Sumo Logic for free