New Relic has always been a firm believer in the quote “Monitor all the things!” Through the years, that credo has led us down many diverse paths ranging from surfacing deep, technical insights about how your application’s database is performing all the way to showing you high-level, business-rich data on how users are interacting with your software. But until today, there was a noticeable blind spot in the fast-growing containerization space.
So we are proud to announce a public beta of an important addition to the ever-growing family of “things” New Relic can monitor: Docker. This is the story of how we got there, and a request for feedback on the most important next steps.
Docker is a portable, lightweight runtime and packaging tool that enables apps to be quickly assembled from components and eliminates the friction between development, QA, and production environments. As a result, IT can ship faster and run the same app, largely unchanged, on everything from laptops to data center virtual machines to any cloud.
Visibility into users’ containers is important, but core functionality in the New Relic experience was lost or degraded due to the abstractions inherent to Docker’s architecture. Further, there was no visibility into how Docker itself was performing on New Relic user’s hosts. As Docker users ourselves, we knew how useful it is to have visibility into containers, so we set out to make your lives (as well as our own) a little easier.
A holistic solution
We wanted to make Docker feel like a first-class entity in New Relic, not “just another thing we monitor.” So we got to work identifying and improving the features in New Relic that didn’t play nice with Docker, introducing good monitoring practices that deliver visibility into containers from the app’s point of view.
First up was New Relic APM’s drilldown page. When viewing an application in New Relic, a powerful feature at the bottom of the page immediately shows you a summary of how many servers your application is load-balanced across. This can help you decide whether you have enough resources backing your application, and gives you the option to jump into a New Relic Servers instance to get more details about how your application is performing on a given host.
When Docker is involved, however, the connection between the application and server is lost. From your application’s perspective, only the container is visible, so instead of a list of servers you’re stuck with a list of unfriendly container GUIDs that are linked to nothing:
We decided to switch this view to a hierarchical table that would make it easier to tell a container from a host. If your application is inside of a Docker container, you’ll be able to navigate to the host machine to see how things are performing:
Next up was New Relic Server’s drilldown page. For a given server, the page indicates all of the New Relic instrumented applications you have hosted there. This can help you understand what will be affected if you take a host out of rotation or if a noisy neighbor is hogging all the resources.
Again, we weren’t able to surface this content for applications inside of a Docker container. Nothing too fancy here, we simply fixed what was broken with Docker in the middle, so now Docker users can get the same visibility that anyone running on bare metal would see for a given host:
At that point we felt we could honestly say that New Relic supported Docker-based application deployments. New Relic users who have Docker in their appstack could now enjoy the same navigational features and visibility as those using bare metal hardware or traditional virtualization.
Tracking how Docker performs
However, we soon realized that that wasn’t enough. We needed detail into how Docker itself was actually performing. But we didn’t want to create a small side-project to monitor Docker and just regurgitate all the same data that the Docker APIs provided. Instead, we opted to embed Docker directly into our core product. We feel that this approach has laid the groundwork to grow into a robust, convenient, and most important, useful Docker monitoring integration for New Relic.
So, without further ado, let’s take a look at the New Relic Virtualization dashboard:
The Virtualization dashboard summarizes the impact of your Docker images on their host server. Currently you can view running containers on a given host rolled up to the image level, filtered to the top 20 most CPU-intensive and memory-intensive ones. Just as important, the dashboard provides high-level information on how Docker itself is performing in your infrastructure. And it’s another key step in building out a fully-fledged Docker monitoring solution.
Now is where you come in! We’ve finished the first round of implementation and we want to hear from you. We’d like to know how useful it is, how easy it is to set up, or even if we used the wrong font in our charts. Your feedback is a critical ingredient in our process, so if you have time and you’re looking to see what Docker-related goodies we’ve been working on, please follow the instructions here to check out our latest beta release!
If you have questions or comments, post them in the “Docker Beta” category in the New Relic Community Forum. If there’s anything you’d like us to know, we’d like to hear it (e.g., the data is or is not helpful, the experience is confusing, we spelled “Docker” wrong). It will all help us plan the next steps of this project and ultimately help us help a lot of people write better software.
Thank you for taking the time!
For more information, see:
- The Docker press release
- New Relic Servers for Docker
- Enabling New Relic Servers for Docker
- Docker beta in the New Relic Community Forum