One of the greatest challenges in building an application performance management system is having a suite of applications to manage. APM is all about collecting the right metrics about an application’s performance, and presenting that information in the most effective, intuitive manner possible.
This is particularly challenging because web applications vary so broadly, not only based on the frameworks they use (J2EE, Spring, Grails, Rails, Sinatra, etc) but also the backend datastore selections (Relational Database, NoSQL, SOAP, Rest, etc) and a myriad of other factors that make each web application unique.
The on-premise APM Vendor therefore has to guess at how web applications work at run-time, how to instrument them and how to present the data. To build out new features and test their instrumentation and visualization, they have access to a very limited suite of applications to test against. A perennial favorite is Pet Store, the original demo app for J2EE, first built in around 1999. Several years later, Spring introduced a demo app for their framework, which was – a Pet Store. The demo app for .NET? You guessed it. Pet Store.
As a result, most APM offerings are remarkably good at managing Pet Store. Too bad for you that you don’t sell pets.
By contrast, New Relic has the worlds largest application performance database, by a factor of at least 100 times our nearest competitor. As a multi-tenant SaaS offering, we collect data from over 40,000 Java or Ruby instances every minute. That’s 1.4 million metric data points per minute, 60 billion per month. This data is a game changer for us because it enables us to continually improve our data collection and presentation, which in turn improves how well RPM manages your applications and environments.
Last October, we released RPM V2, which was a complete overhaul of the user interface. The changes significantly improved RPM’s intuitiveness and usability. Since that change, our customers spend 50% more time on RPM than they did on V1. Without a rich data set of real application performance data, I am positive we would not have enjoyed this kind of success.
Absent a large warehouse of production application performance data, APM features built in the lab hit snags in the real-wold. At New Relic, we catch many of these problems before the feature is released to our customers. When we are building a new feature, we try it out on dozens of real world applications and iterate on it before publishing it. Contrast that to the waterfall process of putting a new version of on-premise software in your environment, telling your vendor that the instrumentation is inadequate or the data presentation is poor, and then waiting for your vendor to build a fix for you to reinstall in your environment and hopefully get it right.
With such a massive resource of real-life production data, the application performance metrics we capture are also valuable to the Java and Rails community. We make this data available to the developer community in a couple different ways. Every few months we publish a benchmark of the most commonly used application components in a report called State of the Stack on our RailsLab site. This enables everyone using Rails to compare their application stack to others. Expect us to do similar things for the Java community. We also enable every Rails customer to share his performance data with the Rails Core Team who are responsible for moving the platform forward.
The net result for our customers is software that is more intuitive, and will likely just work in their environment. If it doesn’t, let us know, and we’ll have a look at your app right away. Chances are, the fix will be in next week’s push.