Why is software development work so difficult to make reliable estimates about?
If you asked someone to plaster a wall in your house, they would look at it; feel the surface; test it for moisture and loose material; think about the weather; then measure the wall and see how many square metres it was. Then they would tell you how long it would take and if it was wrong by more than 10% then you would probably be unhappy.
If you did some software development work and you missed your deadline by 10% no one would be surprised. Software projects don’t hit their deadlines; in fact deadlines might not even be important. In fact, if it worked at all they will probably be quite pleased.
The question is: why not? One conversation I had about this led to the insightful comment: when you do software development and you think “I’m going to plaster this wall” you make some estimate but as soon as you start working you find out that it isn’t a wall at all! You quickly find out that the wall has become a ceiling made of rubber. Or it’s a door that leads into a lift shaft. The concepts in software are — well — soft! My wife took an alternative, less fanciful, view: that it was like trying to get someone to estimate the job when they only hear a description on the phone. The person that is describing the work doesn’t realise what all the important things are; doesn’t know how to find them and even if they are told what to look for, they don’t know how to see the important things.
So, what do we do? Give up making estimates or try and make the software process mechanical? I think that the kind of problems that we see with estimating software development are more akin to things experienced in creative design work. Software isn’t changing that fast; even though the technologies evolve rapidly the core ideas are the same as they have been for 30 years. The question is why isn’t that producing repeatability? The answer is, I think, that there isn’t a clear body of knowledge like there is for civil engineering and there isn’t a professional development program that is embedding it.
Thanks to Jaspreet for his insight into plastering and software development!