Inside New Relic Engineering: Slip Charts

Just as you’d never run a production web app without performance monitoring, I never want to run an engineering organization without some level of monitoring. And while web apps have a standard measurement (response time), there isn’t an industry standard measurement of engineering productivity. So what do we measure in our internal engineering dashboard? I use Slip Charts. (Although I thought I had invented them, a quick Google search shows that I was probably influenced by many of the project management books I’ve read over the years including Quality Software Management: First Order Measurement and the Project Management Toolbox.)

Here’s an example of a short project that was very well estimated — no slips, no padding, delivered a day early:

Slip Chart: Well planned project; no slips

Here’s a more typical project that was re-estimated a few times, including once where the scope decreased a bit. I don’t expect perfection, so I consider this project to be in pretty good shape:

Slip Chart: Project re-estimated a few times; in good shape

Here’s another typical pattern. When engineers are scared of management, they pad their schedules. Unfortunately, padded schedules cause other dysfunctions in a fast-paced company. If I see this pattern, I talk to the project managers about better estimates in the future:

Slip Chart: Project with too much padding

And then we get into the bad cases. Take this one for example. Three days from release the project suddenly slipped to more than 150% the original estimate:

Slip Chart: Project slips suddenly

But this following one is even worse. This project was completely out of control: scope creep, estimation problems, technical complexity … there are a lot of reasons for this project’s inability to deliver. The key point is that the project’s flailing status is easily visible in the slip chart.

Slip Chart: Project fail

Slip charts are incredibly easy to implement. We use both Pivotal Tracker and Atlassian’s JIRA;┬áTracker for smaller projects, JIRA for larger ones. In both tools, we use Epics to define “trackable projects/features”. In JIRA we use the Due Date field as the estimated delivery date; in Tracker we use a comment formatted as “ETA: <date>”. My Heroku-based dashboard uses the Tracker and JIRA APIs to grab a snapshot of every project every night, storing (recorded date, estimated date) pairs in the database. The slip chart is just a fancy <canvas>-based view of those pairs.

Now if only I could invent a predictive slip chart or perhaps a time machine…

bjorn@newrelic.com'

View posts by .

Interested in writing for New Relic Blog? Send us a pitch!