12 Problems with Software Estimation

Software estimation is hard, really hard, but still many software companies rely heavily on software estimations. I will present twelve big problems that come with software estimation. The list could easily have 20 or 30 problems, but I limited the list to only twelve problems.

I will not suggest any solutions to these problems yet; that is the topic of another post.

1. It is impossible to estimate the unknown

Every software is unique, and it is impossible to estimate a software accurately, when it’s being done for the first time. Even if the requirements are given, it is really hard to estimate a complex system that is going to be built with the given requirements.

Software can be programmed in so many different ways, and usually design decisions, which are not known during the time of estimation, have a huge impact on the duration of the project.

There are, of course, different kind models and methods to make the estimates more accurate, but even with the perfect method, the estimates are still only guesses, because of all the things that are unknown when the estimates are needed, which is usually before the development starts.

2. Estimates are usually optimistic

Most programmers make optimistic estimations. They either think they are more productive than they actually are, or they do not understand the complexity of the software.

This usually leads to padding. The person who receives estimates from the programmers might multiply the estimates with a magic number just to make the estimates less optimistic. This is also very bad, because everyone doesn’t make optimistic estimations.

3. Estimated time is always used

Most programmers also tend to spend the allocated time on a task. If something is estimated to take two days, the programmer makes sure that it takes two days. Even if he finishes early, he will tune and polish his solution, or just slack off, until the allocated time is spent.

This creates a situation, where nothing gets done faster than the estimates, but some things will take longer than the estimates.

4. Business relies on estimations too heavily

Product launches are scheduled based on estimates. Trade shows are booked based on estimates. In some companies the whole sales and marketing departments rely so heavily on estimations that any kind of delay will cause major problems for the company.

5. A lot of time is spent on things that are not estimated

Even if the estimates are perfect, a lot of time must be spent on things that are not estimated. Programmers need to do some maintenance for older versions, someone gets sick, and new tasks come up during the project.

Many companies add another padding for these things, but it does not make the art of estimation any better. If estimates are based on adding padding, and padding on top of guesses, it does not sound something that can be relied on very much.

6. Estimates are not treated as estimates

People do not understand the meaning of the word estimate. It should be taken literally, but the true meaning of the word is not understood nor is it respected. There shouldn’t be too many plans that rely on the estimates. If things that have estimates depend on other estimated things, it creates a chain reaction of delays, when an estimate cannot be reached.

7. Estimates do not consider productivity variations between programmers

This adds another level of difficulty for estimations. In traditional industry, you know the speed of the average bridge builder, but programmers differ so much in productivity that it is impossible to estimate accurately something when you have two unknown variables: the speed of development, and the required amount of work.

The situation is not that bad when programmers estimate their own work, because they might have a good guess on their own productivity, but when someone else does the estimation for the programmers, they could get equally believable numbers with a Stetson-Harrison method.

8. Changes in requirements do not reflect on estimations

Most projects have changes in requirements through out the project, but the estimations for the whole project are not changed. Every single feature added to project must have an effect on the estimates, otherwise the whole process of estimating things is a complete joke.

9. Truth is told when it’s too late

When programmers make their own estimations, they blindly believe that the estimates can be reached and the software will be finished on time. They won’t accept that their estimates were optimistic, until very late in the project, when telling the truth is already too late.

10. Estimates are not updated when they cannot be reached

When estimates are not reached, other estimates should also be corrected, because missing estimates is usually a sign of optimistic estimation. However, in these situations programmers blindly believe that they can do the rest of the tasks faster to make up for the lost time, when the opposite is usually true.

11. Programmers are blamed when estimates cannot be reached

When the whole success of the company depends on estimates, someone must be blamed when the estimates cannot be reached. Usually the finger is pointed to the people, who could not keep up with the estimates: the programmers. This will create a very unhealthy work environment, and programmers will pad the estimates just to keep their jobs safe.

12. Wrong people do the estimates

Estimates should be made by the programmers themselves. Sometimes team leads can do the estimates for them, but the game is lost, if someone with an MBA does the estimates. Run and don’t look back, if this is happening in your company.

  • Share/Bookmark

25 Comments

  1. Amber Shah says:

    I find it amusing that people get surprised that estimates are so often wrong. After all, estimating is basically trying to predict the future. If you are truly good at predicting the future, then building software is probably the wrong business for you (try the stock market!).

    One thing I find is that most people equate “estimate” with a “commitment”. When most businesspeople ask for an estimate, they are actually asking for a commitment, that it would be done BY THEN or at least be done shortly after. A developer offers his estimate for development time only, not taking into account roadblocks, communication delay and just overall life in general.

    Developers need to be aware and verbal about which one they are giving when they just throw out a number. I would say, though, that if you’re working for a company that likes to play the blame game, then you’ve already lost.

  2. Agile offers a solution for this, which is not to rely on large estimates. Just estimate the next 2-4 weeks and you’ll learn the speed of your team as you go. This is a very good way to go in some projects. Just make sure you always deliver the most important functionality to the customer first. Then just iterate until you’ve reached the point where adding more features doesn’t add enough value to the customer.

    However, if you have to do a large estimate, the first thing to do is to find out if you have the actual time spent from another similar project, or projects. This is probably the most reliable way to do the estimate.

    The last method I use is to break the project down to small tasks, down to the level of 2-16 hours each. This should give you a reasonable estimate if you’re honest about how things usually really go when you estimate each of these tasks. Please also keep in mind that people don’t work at 100% efficiency.

  3. Jeff says:

    Estimates are definitely hard, and there is a wealth of information out there providing guidelines on how to improve ones estimations as well as how to interpret another’s estimations.

    The great equalizer here is historical data. Keep track of all of your estimates as well as the final actuals, and ensure that this information is available to the entire team. There will always be outliers, but as a rule, this historical data will guide all team members to improve their estimates such that horrid overruns and farcical padding becomes less of an issue. If anything, spotting trends becomes much easier with 3, 6, even 12 month’s data easily available.

  4. roger says:

    Folks,

    Let me disillusion you. What you have here, and unfortunately, it has become a norm, is a major failure of program/project management. In fact, one of the rules in PM world states ‘don’t publish estimates, only 2 or 3 revision.’ Programmers must be empowered with PM techniques and methodologies to decompose work enough to understand cost/time ramifications. Again, unfortunately, modern so-called project managers act like mere coordinators fully relying on the information furnished to them by the same programmers. Instead, project managers must be fluent in project controls to recognize trends and raise issues when things are going haywire. Of course, if street standard these days is PMP designation that came out of the construction industry, no wonder that project managers can’t determine a simple critical path of a project, much less they can effectively shield programmers from dates-minded business community.

    DAWIA
    Roger

  5. JoeBlob says:

    A corrollary to no. 4 is of course that companies often just use the launch date as the estimate. We want it in 3 weeks therefore it will take 3 weeks….

  6. All good points. Still, in explaining them, I found some blaming that may be extended. Point 9 for instance, says engineers are the ones to estimate and blindly believe they will succeed in time. Sometimes are not engineers but managers the ones that put experienced engineers to estimate, and junior engineers to execute, or managers that put the estimate as a contract engineers must honor, thus no further adjustments are allowed, etc.

    @Susana: Great tips. Still, there is a problem with estimates, not said here, that will also affect the agile technique: defining the tasks.
    You see, I know broadly what should I do. Then I try to break that into small tasks, hopefully no one more that one day. If it is greater than one day, it is probably a breakable task. If it is too small (1 hour) it may be too unrealistic (usually it takes 2 hours to complete the coding, check it, integrate it, plus all other interruptions you may have during that period).

    The most common problem when breaking the task, is that you tend to think on the most important sub-tasks, ignoring some easy or trivial ones that would add time at the end.

    The best way of detecting what is keeping the engineer from succeeding with estimates is to keep a log. The developer’s journal is a quick document where the developer writes down the problem faced, and the solution. It helps a lot to reconstruct the work and see what things killed the estimate, plus helps the developer to focus on problem and ends up helping him to discover the solution faster (and helps others that may find the same problem later on). You will find that maybe 2 hours were just right to code, but making that code work due to bugs or problems with other tools, takes 3 additional hours!

    Cheers!

  7. I don’t think the problem will be solved by making more accurate estimations. Even if you slice and dice everything and use all the historical data in the world, your first estimate won’t be very accurate, unless you always overestimate things and use the allocated time.

    My main point was that the whole concept of relying on estimations is flawed. That’s why I like more the agile approach, where you do stuff in prioritized order and then someone else can decide when to ship.

    It’s impossible to answer the question, how long will it take to get a fixed scope done with a set of resources. That question should never be asked.

  8. Vesa Suontama says:

    @Tuomas; Estimation question should never be asked? I must disagree with you on this one.

    PM (project or product manager) needs to compare different tasks. They need decide the value of feature X against the features (Y,Z,Y, U V), and compare the costs.

    Usually it is the programmer who can provide most acurate estimations.

    This comparison of value of features should be done in both agile and “traditional” projects. Without programmers help it is harder to do cost/value based priorization.

    All parties just have to acknowledge that estimations are inaccurate.

    It is also revealing to study why estimations were inaccurate. It gives important and interesting insights into the code base, people, feelings and technologies used.

    I have seen programmers who try to avoid answering this questions, and it is just selfish because then the PM needs to do his value estimations on bad data.

    So go on and admit that you are just wrong on this one, if you can :)

  9. I am not wrong. I never am. ;)

    If you need to consider the cost of features X, Y and Z, you should estimate them relatively, not by guessing that X takes 2 weeks, Y 3 weeks, and Z 4 weeks.

    It’s a different thing to estimate the absolute time it will take to do something than to say something will take two times longer than something else.

  10. Martin Smith says:

    @Tuomas – the thing that we most often miss out when saying use relative sizing is that business are deciding where to invest their money not just between different software projects but between opening a new store, running a marketing campaign. We are not good at showing the relative value of these ventures and we are going to struggle to relative size these.

    The key is to start to think of projects as new ventures or companies and start to deliver similar information a start-up would give to a financial backer. We must adapt to the needs of the backer and, if they need an estimate we must find a way of delivering one. Agile techniques can help, but we need to take this list of the challenges and have more than approach to mitigate the risk laid out in this list.

  11. Dave says:

    Every project I’ve worked on in IT has contained at least one hidden “Gotcha”. Every single one. It’s these Gotchas that have caused the overrun of projects, whether it’s because of a single piece of code that calls a program that is no longer supported, or that the data isn’t compatible with how it’ll be used, or because the logic applied turns out to be more complex than believed.

    The problem is that PMs always seem to think that these gotchas can be seen up front, and mistake the fact that even in the cases when they can, that they should have been prevented.

    Consider this logic for a second. Consider what this is actually asking of the person doing the estimation. The person is faced with, more than likely, a massive distributed system, that uses hundreds of thousands of lines of code, built on software that uses magnitudes more, any of which can be called multiple times in various combinations and if just one of these lines of code contains a gotcha, then to notice this before the system itself reveals the gotcha – either by a malfunction or an unexpected behaviour – then the estimator would have had to have selected this single line, out of the numerous other hundreds of thousands, applied the same data that will eventually cause this problem – out of the infinite possibilities of combinations – and by sheer luck, manage to foresee the problem.

    It’s just not going to happen.

    The only way around this therefore, is to ask that a programmer or team of programmers, mentally go through the multitudinous logic in their minds, effectively applying every piece of logic that the systems/computers/languages would, to pinpoint where a hidden gotcha awaits – all before the system is even built. In other words, it’s asking a human to do a machine’s work with the exact same precision, speed and an appropriately comparable conceptual ability. This is fundamentally impossible.

    For the love of all that is sane, it’s the reason why computers exist in the first place: Humans can’t logicise this effectively.

    Every single gotcha in a system is due to conditional logic being applied that is not suitable to the system. Every single piece, whether it’s a version not being supported by the platform and Windows/Unix checking this, or if a driver doesn’t exist on a 64-bit architecture and an application can’t find it – it all comes down to a gargantuan number of conditional logic statements, applied sequentially, where only one failure with halt the system.

    If you are a PM in IT, and you don’t understand this, then you shouldn’t be a PM in IT.
    If you’re an accountant and you don’t understand this, you shouldn’t have anything to do with IT projects.
    If you’re a CEO and you don’t understand this, then you’re going to have a lot of very unhappy shareholders.

    The only way to deal with this absolute, inescapable, fundamental fact of reality, is to look at the reasons a definite development time is required, and remove them.
    In practise, this isn’t going to be easy. In reality it’s going to be very difficult. But in reality, there are no tails wagging the dog, and because of this no dog owners getting upset that the dog isn’t happy.

  12. Thankfulness to my father who shared with me on the topic
    of this webpage, this weblog is in fact awesome.

  13. Lonny says:

    This article is very informative but it took me a long time
    to find it in google. I found it on 15 spot,
    you should focus on quality backlinks building, it will help you to
    rank to google top 10. And i know how to help you, just
    search in google – k2 seo tips and tricks

  14. Lowell says:

    I see a lot of interesting content on your blog.
    You have to spend a lot of time writing, i know how to
    save you a lot of time, there is a tool that creates unique, google friendly articles in couple of minutes, just
    search in google – laranita’s free content source

Trackbacks / Pingbacks

  1. uberVU - social comments
  2. Auto Post 04/09/2010 (p.m.) « PN.Truong Phuc's Blog
  3. Blog post 04/09/2010 (p.m.) « PN.Truong Phuc's Blog
  4. Weekly Link Post 140 « Rhonda Tipton’s WebLog
  5. Yazılım Projelerinin Zaman Planmasında 12 Problem | engin güller
  6. FollowSteph.com – Lazy Friday Reading Assignments
  7. Pedro Newsletter 06-11.04.2010 « Pragmatic Programmer Issues – pietrowski.info
  8. How to Manage your Interaction with Clients - WebsitesMadeRight.com
  9. Ketterästi sikaa säkissä, osa I | Celkee Oy
  10. Yazılım Projelerinin Zaman Planmasında 12 Problem | Oğuzhan Çetinkaya

Leave a Reply