Don’t Get Demoralized by Bad Estimates!

By Posted in Tech Topics, Top Post 3 July 2013

Sometimes it’s hard to estimate how much time you need to complete a project. Developers often underestimate how much time we need to deliver a new feature, especially with changes and iterations involved. It can be demoralizing to get it wrong, and we often do.

As the lead on a recent project, I learned the hard way after a steady stream of bad estimates, changing requirements and scope creep caused a one-month project to be delayed – repeatedly – until it had been five months.

Case Study: Tagging
I recently built out a feature here at New Relic that enables users to tag their applications, servers and key transactions. Tagging is a very simple idea and a very simple feature. To keep things basic and incremental, we planned to ship the tagging feature for servers first. I gave an estimate of about one month to complete the project – divided up into four reasonable weeks – to do the following:

* Implement a simple backend and a basic UI for assigning tags

* Implement a basic UI for filtering servers by tags and tagging multiple servers at once

* Incorporate UI/UX feedback from our design team, release to early preview users

* Release to everybody and fix up any reported issues

The first two weeks went by seamlessly. Then the inevitable, painful changes began and started to derail the project’s timeline. Fortunately, pain often leads to lessons.

Lesson 1: The Greater of Two Estimates
In week three, meetings with our design team made it clear that I had grossly underestimated the amount of time it would take to collect and implement feedback. I had initially allowed a week for this discourse given the simplicity of the feature, but with the amount of feedback I received, I suddenly found myself off schedule.

This brings me to the first lesson: assume something could take longer than expected, and think about the reasons why. It’s better to overestimate and ship something early than to underestimate and cause a delay.

Lesson 2: When to Re-estimate
We make it a priority at New Relic to constantly improve our site performance, and this priority introduced scope creep into the Tagging project. The pages I worked on for the Tagging UI weren’t speedy enough, so to improve performance and reduce technical debt, we decided to rewrite them in a client-side MVC framework (Backbone).

Although this rewrite was outside my project’s scope, I agreed to the added requirements. I saw the potential for making the rest of the project easier on me. I thought about how portable and reusable I could make the Backbone views for the Tagging interface – they would make it much easier to port Tagging to applications and key transactions. I relished the thought of searching through a Backbone collection rather than searching for table rows directly on the DOM. It made me giddy, in fact. So I added two weeks to my project, one week each to rewrite two pages in Backbone.

I don’t regret adding this requirement to the project. Rewriting the UI for these pages did make it easier to do future work on the project, and the code behind these pages is also much cleaner now. What I do regret is not taking a step back at that time to look at my plan again as a whole.

So I find myself at the second lesson: any non-trivial changes to your requirements warrant re-estimation. I should have gone through another quick planning/estimation phase to revisit the project with this new requirement. Instead, I simply allowed extra requirements to be tacked onto the current project plan.

Lesson 3: Changes are Okay
This is cliché, but don’t be afraid of change. It happens in life, it happens at work, and it happens in our projects. Change is inevitable. What’s important is how we address change: with careful consideration.

If the changes make the initial release more useful, or reduce technical debt and make future work/maintenance easier, consider adding them to the project plan. When you don’t think the proposed changes are a good idea, turn them down. Maybe they’re out of your project’s scope or they could be put on hold and done incrementally after an initial release.

It’s extremely important to re-estimate the entire project accordingly lest you find yourself with a chain reaction of changing requirements further down the line. The Backbone rewrite later caused a few changes that I hadn’t considered at the outset. These caused only minor delays but, by that point, there already were enough delays that I felt demoralized.

A single change – and the resulting added cost – cannot be considered on its own, as it will likely affect every requirement thereafter.

To Recap
A feature that I once believed would only take a month ended up taking five. I lost confidence in my ability to deliver expectations. Now I can’t help but wonder if I’d used more caution, I could have avoided feeling like I failed.

1. Think something might take longer? It probably will. Think about why, and give yourself more time accordingly.

2. Non-trivial changes warrant re-estimating your project. Think about the long-term implications of added or changed requirements.

3. Instead of trying to avoid change, be cognizant of how much change you can handle and how that affects the overall cost of your project.

I’d love to hear stories of how you approach changing requirements and what you’ve learned. As always, the comment field is below.

About the author

\ No newline at end of bio

Tell us your thoughts Or Send us an internal high five

Talk to @newrelic