Estimating Software Projects with the ••• System

Like most programmers (and most people generally), I have a tendency to underestimate the time it will take to finish complex tasks.¹ To counteract this bias, I came up with an estimation technique that I’ve been applying to all of my software projects. I call it the “••• (dot dot dot) system”.

Hold on, are estimates even necessary?

It’s not hard to find where estimates have failed in the physical world: public works projects that went billions of dollars over budget, hardware Kickstarters arriving years late, building renovations that fall weeks or months behind schedule. Software of course is no exception, and perhaps may be even more prone to poor estimation. I’m sure we’ve all witnessed a software project that took much longer than planned.

If estimates are so often notoriously wrong, why do we bother estimating software projects at all? For me, it is not really up for debate: working as a freelancer, my estimates are critical to setting expectations with my clients. Starting a project without an estimate is like asking the client for a blank check: good luck with that! Likewise, seriously low-balling estimates and then blowing through budgets is not a good way to satisfy customers or win referrals.

Therefore the question I ask myself is not “are estimates necessary?” but rather “how can I improve my estimates?”. To answer that, I’ve set out to overcome my natural underestimation bias with a little bit of mind-hackery.

Here’s how my ••• system works:

1. Draw a site map
2. Break it down
3. Spike it
4. Assign dots based on difficulty
5. Compute
6. Trust the estimate

1. Draw a site map

The biggest risk in planning a project is forgetting to account for a major piece of functionality. Therefore I find it incredibly helpful to produce some sort of schematic first, to ensure that the client and I are on the same page.

For a web development project, the classic deliverable for this is a site map. Here are some tips for creating a site map for the purpose of estimating a web project:

• It doesn’t have to be perfect or hi-fidelity; just a rough thumbnail for each page is usually good enough.
• I refer to past projects or competitors to get an idea of what pages are needed.
• I include all pages I can think of, including boilerplate like login, password reset, etc.
• On each page, I ask myself, “where can the user go from here?” or “as a user, what would I want to do next?”. That helps me uncover pages I might otherwise forget to include.

If building the site map is difficult, or if there are big gaps, that is a giant red flag. In my experience it means there is simply not enough information to produce an accurate estimate. In this case I’ll suggest a small starter project or discovery phase where the client and I can first hash out the requirements.

2. Break it down

Next I’ll list all the steps and tasks I need to perform to build the actual product, based on the site map and other requirements. Again, referring to past projects is a huge help here.

• It’s important not to make the tasks too large. If it seems like a task will take more than a couple days to complete, that’s a sign that it should be broken down further.
• List all tasks involved in the project. Don’t forget testing and deployment!

3. Spike it

Sometimes a feature or task will have me completely stumped. Maybe I haven’t built something like it before, or maybe it’s a third-party integration I’ve never encountered. In these situations I will build a “spike solution”.

A spike solution (or just “spike”) is a coding experiment designed to mitigate risk or uncertainty. The goal of a spike is not necessarily to build a production-quality feature, but to simply prove that a solution is feasible.

For example, I might review the documentation of the third-party integration and do a tutorial or two to get a feel for it. With this experience, it becomes easier to gauge how difficult the feature will be to implement.

4. Assign dots based on difficulty

Now it’s time for the mind hack.

It’s difficult to look at tasks in aggregate, or even individually, and come up with an estimate in terms of dollars or hours. But comparing one task to another and deciding which takes more effort: that’s easy!

I’ll start with the easiest tasks first and assign them • (one dot). Tasks that are slightly harder than the easiest tasks get •• (two dots), and so on. I like to think of the dots as representing “• trivial”, “•• easy”, “••• medium”, and “•••• hard”.²

The mind hack is that I am essentially creating estimates without using numbers. By sorting the tasks into groups based on relative difficulty, and identifying them with an abstract, qualitative measurement of “dots”, I am bypassing the project manager part of my brain that wants to sum things up and think in terms of quantities of dollars and hours. By avoiding numbers, I prevent the tendencies of second-guessing and underestimation from creeping in.

Accounting for risk

If a task seems particularly risky, or if the task depends on an external system or API that may or may not work as I expect, I’ll add an extra • to account for that risk.

5. Compute

Now it’s time to take those abstract dots and turn them into a concrete estimate.

I’ll look through all the • (single dot) tasks and choose one that I can most confidently translate into actual time. Let’s say I’m 100% sure that this task will take no more than 1 hour to complete.

Then I’ll look at the •• tasks. How many • tasks could I get done in the space of time that a •• task demands? Likewise for ••• tasks, and so on.

Usually it works out to be powers of two. I’ll make a legend like this:

• • = 1 hour
• •• = 2 hours
• ••• = 4 hours
• •••• = 8 hours

Each individual task may take more or less time than the legend indicates, but that’s OK. It will come out in the wash.

Now it’s just a matter of translating the dot estimates into hours based on this legend, and adding them all up. The estimate is complete!

Uncertainty factor

I also take one final step when calculating the estimate: I multiply the total by an uncertainty factor, usually between 1.0 and 2.0. Arriving at this factor comes with experience, and it depends on the client. When choosing an uncertainty factor, I’ll consider:

• How similar is this project to work I’ve done before?
• Is the client confident in the requirements, or will things change?
• How hands-on is the client? Do I expect the client will be satisfied with my first design, or will there be many revision/feedback cycles?

6. Trust the estimate

When I’m done with this exercise, the resulting number almost always looks too big. In fact, my first reaction is “wow, that is much more time than I expected!”

I’ve learned that this is completely normal, and it’s one of those situations where my “gut feeling” is wrong.

Here’s why. When I’m asked to make an estimate (i.e. deliver a proposal), the client or stakeholder often phrases the request like this:

Hey Matt, we’d like you to design and build features A, B, and C – is that something that you could get done for us in, say, two weeks?

The client is already “framing” the discussion: even if two weeks is impossibly optimistic, my mind latches onto it as the baseline, even when I’m not conscious of it. If my ••• system indicates the work will take closer to 5 weeks, it feels like I’ve miscalculated. Surely the “right” estimate should be smaller than that?

This framing comes in many forms: budget considerations, the client’s desire (understandably) to get a “good deal”, etc. Once you add in human nature to underestimate, there is a lot of pressure that makes the calculated estimate seem “wrong”. Resist that pressure and trust the estimate!

Parting thoughts

Estimating software projects is incredibly difficult. No two projects are the same, which makes it hard to consistently apply past experience to future estimates. And even if two products did have identical requirements, the pace of changing technology means building that product in 2014 is going to be a much different experience than building it again in 2015.

As software developers, we’re expected to incorporate the latest and greatest while building things that have never existed before, and do it all on time and on budget. It’s hard work! Thankfully the ••• system is working well for me and helps me keep my sanity.