12 Problems with Software Estimation
April 8, 2010
|
Posted by Tuomas Pelkonen
Software estimation is hard, really hard, but still many software companies rely heavily on software estimations. I will present twelve big problems that come with software estimation. The list could easily have 20 or 30 problems, but I limited the list to only twelve problems.
I will not suggest any solutions to these problems yet; that is the topic of another post.
1. It is impossible to estimate the unknown
Every software is unique, and it is impossible to estimate a software accurately, when it’s being done for the first time. Even if the requirements are given, it is really hard to estimate a complex system that is going to be built with the given requirements.
Software can be programmed in so many different ways, and usually design decisions, which are not known during the time of estimation, have a huge impact on the duration of the project.
There are, of course, different kind models and methods to make the estimates more accurate, but even with the perfect method, the estimates are still only guesses, because of all the things that are unknown when the estimates are needed, which is usually before the development starts.
2. Estimates are usually optimistic
Most programmers make optimistic estimations. They either think they are more productive than they actually are, or they do not understand the complexity of the software.
This usually leads to padding. The person who receives estimates from the programmers might multiply the estimates with a magic number just to make the estimates less optimistic. This is also very bad, because everyone doesn’t make optimistic estimations.
3. Estimated time is always used
Most programmers also tend to spend the allocated time on a task. If something is estimated to take two days, the programmer makes sure that it takes two days. Even if he finishes early, he will tune and polish his solution, or just slack off, until the allocated time is spent.
This creates a situation, where nothing gets done faster than the estimates, but some things will take longer than the estimates.
4. Business relies on estimations too heavily
Product launches are scheduled based on estimates. Trade shows are booked based on estimates. In some companies the whole sales and marketing departments rely so heavily on estimations that any kind of delay will cause major problems for the company.
5. A lot of time is spent on things that are not estimated
Even if the estimates are perfect, a lot of time must be spent on things that are not estimated. Programmers need to do some maintenance for older versions, someone gets sick, and new tasks come up during the project.
Many companies add another padding for these things, but it does not make the art of estimation any better. If estimates are based on adding padding, and padding on top of guesses, it does not sound something that can be relied on very much.
6. Estimates are not treated as estimates
People do not understand the meaning of the word estimate. It should be taken literally, but the true meaning of the word is not understood nor is it respected. There shouldn’t be too many plans that rely on the estimates. If things that have estimates depend on other estimated things, it creates a chain reaction of delays, when an estimate cannot be reached.
7. Estimates do not consider productivity variations between programmers
This adds another level of difficulty for estimations. In traditional industry, you know the speed of the average bridge builder, but programmers differ so much in productivity that it is impossible to estimate accurately something when you have two unknown variables: the speed of development, and the required amount of work.
The situation is not that bad when programmers estimate their own work, because they might have a good guess on their own productivity, but when someone else does the estimation for the programmers, they could get equally believable numbers with a Stetson-Harrison method.
8. Changes in requirements do not reflect on estimations
Most projects have changes in requirements through out the project, but the estimations for the whole project are not changed. Every single feature added to project must have an effect on the estimates, otherwise the whole process of estimating things is a complete joke.
9. Truth is told when it’s too late
When programmers make their own estimations, they blindly believe that the estimates can be reached and the software will be finished on time. They won’t accept that their estimates were optimistic, until very late in the project, when telling the truth is already too late.
10. Estimates are not updated when they cannot be reached
When estimates are not reached, other estimates should also be corrected, because missing estimates is usually a sign of optimistic estimation. However, in these situations programmers blindly believe that they can do the rest of the tasks faster to make up for the lost time, when the opposite is usually true.
11. Programmers are blamed when estimates cannot be reached
When the whole success of the company depends on estimates, someone must be blamed when the estimates cannot be reached. Usually the finger is pointed to the people, who could not keep up with the estimates: the programmers. This will create a very unhealthy work environment, and programmers will pad the estimates just to keep their jobs safe.
12. Wrong people do the estimates
Estimates should be made by the programmers themselves. Sometimes team leads can do the estimates for them, but the game is lost, if someone with an MBA does the estimates. Run and don’t look back, if this is happening in your company.
Categories: Programming, Rant, Software Development
| 
Tags: Blog, Estimation, Programming, Rant
|
Leave a reply
I find it amusing that people get surprised that estimates are so often wrong. After all, estimating is basically trying to predict the future. If you are truly good at predicting the future, then building software is probably the wrong business for you (try the stock market!).
One thing I find is that most people equate “estimate” with a “commitment”. When most businesspeople ask for an estimate, they are actually asking for a commitment, that it would be done BY THEN or at least be done shortly after. A developer offers his estimate for development time only, not taking into account roadblocks, communication delay and just overall life in general.
Developers need to be aware and verbal about which one they are giving when they just throw out a number. I would say, though, that if you’re working for a company that likes to play the blame game, then you’ve already lost.
Agile offers a solution for this, which is not to rely on large estimates. Just estimate the next 2-4 weeks and you’ll learn the speed of your team as you go. This is a very good way to go in some projects. Just make sure you always deliver the most important functionality to the customer first. Then just iterate until you’ve reached the point where adding more features doesn’t add enough value to the customer.
However, if you have to do a large estimate, the first thing to do is to find out if you have the actual time spent from another similar project, or projects. This is probably the most reliable way to do the estimate.
The last method I use is to break the project down to small tasks, down to the level of 2-16 hours each. This should give you a reasonable estimate if you’re honest about how things usually really go when you estimate each of these tasks. Please also keep in mind that people don’t work at 100% efficiency.
Estimates are definitely hard, and there is a wealth of information out there providing guidelines on how to improve ones estimations as well as how to interpret another’s estimations.
The great equalizer here is historical data. Keep track of all of your estimates as well as the final actuals, and ensure that this information is available to the entire team. There will always be outliers, but as a rule, this historical data will guide all team members to improve their estimates such that horrid overruns and farcical padding becomes less of an issue. If anything, spotting trends becomes much easier with 3, 6, even 12 month’s data easily available.
Folks,
Let me disillusion you. What you have here, and unfortunately, it has become a norm, is a major failure of program/project management. In fact, one of the rules in PM world states ‘don’t publish estimates, only 2 or 3 revision.’ Programmers must be empowered with PM techniques and methodologies to decompose work enough to understand cost/time ramifications. Again, unfortunately, modern so-called project managers act like mere coordinators fully relying on the information furnished to them by the same programmers. Instead, project managers must be fluent in project controls to recognize trends and raise issues when things are going haywire. Of course, if street standard these days is PMP designation that came out of the construction industry, no wonder that project managers can’t determine a simple critical path of a project, much less they can effectively shield programmers from dates-minded business community.
DAWIA
Roger
A corrollary to no. 4 is of course that companies often just use the launch date as the estimate. We want it in 3 weeks therefore it will take 3 weeks….
All good points. Still, in explaining them, I found some blaming that may be extended. Point 9 for instance, says engineers are the ones to estimate and blindly believe they will succeed in time. Sometimes are not engineers but managers the ones that put experienced engineers to estimate, and junior engineers to execute, or managers that put the estimate as a contract engineers must honor, thus no further adjustments are allowed, etc.
@Susana: Great tips. Still, there is a problem with estimates, not said here, that will also affect the agile technique: defining the tasks.
You see, I know broadly what should I do. Then I try to break that into small tasks, hopefully no one more that one day. If it is greater than one day, it is probably a breakable task. If it is too small (1 hour) it may be too unrealistic (usually it takes 2 hours to complete the coding, check it, integrate it, plus all other interruptions you may have during that period).
The most common problem when breaking the task, is that you tend to think on the most important sub-tasks, ignoring some easy or trivial ones that would add time at the end.
The best way of detecting what is keeping the engineer from succeeding with estimates is to keep a log. The developer’s journal is a quick document where the developer writes down the problem faced, and the solution. It helps a lot to reconstruct the work and see what things killed the estimate, plus helps the developer to focus on problem and ends up helping him to discover the solution faster (and helps others that may find the same problem later on). You will find that maybe 2 hours were just right to code, but making that code work due to bugs or problems with other tools, takes 3 additional hours!
Cheers!
I don’t think the problem will be solved by making more accurate estimations. Even if you slice and dice everything and use all the historical data in the world, your first estimate won’t be very accurate, unless you always overestimate things and use the allocated time.
My main point was that the whole concept of relying on estimations is flawed. That’s why I like more the agile approach, where you do stuff in prioritized order and then someone else can decide when to ship.
It’s impossible to answer the question, how long will it take to get a fixed scope done with a set of resources. That question should never be asked.
@Tuomas; Estimation question should never be asked? I must disagree with you on this one.
PM (project or product manager) needs to compare different tasks. They need decide the value of feature X against the features (Y,Z,Y, U V), and compare the costs.
Usually it is the programmer who can provide most acurate estimations.
This comparison of value of features should be done in both agile and “traditional” projects. Without programmers help it is harder to do cost/value based priorization.
All parties just have to acknowledge that estimations are inaccurate.
It is also revealing to study why estimations were inaccurate. It gives important and interesting insights into the code base, people, feelings and technologies used.
I have seen programmers who try to avoid answering this questions, and it is just selfish because then the PM needs to do his value estimations on bad data.
So go on and admit that you are just wrong on this one, if you can
I am not wrong. I never am.
If you need to consider the cost of features X, Y and Z, you should estimate them relatively, not by guessing that X takes 2 weeks, Y 3 weeks, and Z 4 weeks.
It’s a different thing to estimate the absolute time it will take to do something than to say something will take two times longer than something else.
goolge
@Tuomas – the thing that we most often miss out when saying use relative sizing is that business are deciding where to invest their money not just between different software projects but between opening a new store, running a marketing campaign. We are not good at showing the relative value of these ventures and we are going to struggle to relative size these.
The key is to start to think of projects as new ventures or companies and start to deliver similar information a start-up would give to a financial backer. We must adapt to the needs of the backer and, if they need an estimate we must find a way of delivering one. Agile techniques can help, but we need to take this list of the challenges and have more than approach to mitigate the risk laid out in this list.