Inside New Relic Engineering: Slip Charts

By: Bjorn Freeman-Benson | Posted in: Company News, Top Post, Sep. 17th, 2012

Just as you’d never run a production web app without performance monitoring, I never want to run an engineering organization without some level of monitoring. And while web apps have a standard measurement (response time), there isn’t an industry standard measurement of engineering productivity. So what do we measure in our internal engineering dashboard? I use Slip Charts. (Although I thought I had invented them, a quick Google search shows that I was probably influenced by many of the project management books I’ve read over the years including Quality Software Management: First Order Measurement and the Project Management Toolbox.)

Here’s an example of a short project that was very well estimated — no slips, no padding, delivered a day early:

Here’s a more typical project that was re-estimated a few times, including once where the scope decreased a bit. I don’t expect perfection, so I consider this project to be in pretty good shape:

Here’s another typical pattern. When engineers are scared of management, they pad their schedules. Unfortunately, padded schedules cause other dysfunctions in a fast-paced company. If I see this pattern, I talk to the project managers about better estimates in the future:

And then we get into the bad cases. Take this one for example. Three days from release the project suddenly slipped to more than 150% the original estimate:

But this following one is even worse. This project was completely out of control: scope creep, estimation problems, technical complexity … there are a lot of reasons for this project’s inability to deliver. The key point is that the project’s flailing status is easily visible in the slip chart.

Slip charts are incredibly easy to implement. We use both Pivotal Tracker and Atlassian’s JIRA; Tracker for smaller projects, JIRA for larger ones. In both tools, we use Epics to define “trackable projects/features”. In JIRA we use the Due Date field as the estimated delivery date; in Tracker we use a comment formatted as “ETA: <date>”. My Heroku-based dashboard uses the Tracker and JIRA APIs to grab a snapshot of every project every night, storing (recorded date, estimated date) pairs in the database. The slip chart is just a fancy <canvas>-based view of those pairs.

Now if only I could invent a predictive slip chart or perhaps a time machine…