First things first, second things never

The title says it all. Everyone is trying to show they are good at Getting Things Done. People invented the waterfall system of project management because it was a hell of a lot better than no system at all. What agile does is cope with a piece of software that is never done, but always improving.

It works by forcing the users to decide what things are first and should be done before anything else. It not only puts first things first; it allows the things that are first this month to be different to the things that are first next month. Of course it isn’t easy,  it requires software developers to give up the pretence that they run the software projects. After all, the person who truly runs the project is the one that gets to decide if it was a success, not the one doing the work; but you can pretend otherwise – for a while – when you are the “project manager”; it sounds like you are in charge. It also requires developers to learn how to make software that is flexible and changeable and testable; but if they can’t do that then the whole future is buggered anyway because whether you are doing agile or not, your requirements will change.

The title is a quote from the 1970’s household productivity classic SuperWoman,  by Shirley Conran. She also said “Life is too short to stuff a mushroom”; but that seemed slightly less relevant.

C.S. Lewis, sly Christian propagandist that he was *, put it slightly less clearly, but more elegantly:

You can’t get second things by putting them first; you can get second things only by putting first things first.

You can’t always get people to believe that they can’t have everything, but at least with agile they get to have something now and something else later, rather than having to wait 8 months to find that the whole thing is missing the point as the new business is now over there <points to the other side of the world/>.

* I must say that the whole Christian metaphor of The Lion, the Witch and the wardrobe didn’t pass me by when I read the book, but it didn’t occur to me to be offended; after all, it was only a story and the whole bible thing, well that was just another story.

My first real agile experience

I had a spontaneous agile moment and I can confirm to those at the back: Agile Works.

Working with two other guys on a project, we agreed with the manager that in one month from now (ooh agile, a sprint) we would demo it (ooh agile, a sprint demo). The demo would consist of putting simple data into a lightweight  front-end and going all the way through to the database and back (ooh agile, a “full slice of cake” like a user story). During that month, we evolved (ooh agile) a way of working that was very effective; we would meet up each day at about 9:30 (ooh agile, daily standup meeting) and talk about the results of yesterday and what we would do today. At least 25% of the time was spent working together at the same computer (ooh agile, pairing). We had a continuous integration build using CruiseControl.Net (ooh agile) running unit tests (ooh agile) and some integration tests (ooh agile). The CI build was also doing some static code analysis (ooh agile) using Simian.

And it was really working. OK we didn’t really have the code coverage from the tests, and the code was, in places, not only a bit dirrrrrty but it was a little heavy on the singletons (the hackers favourite). And we had a few code smells like interacting tests due to all the tests running off the same database. However, we kept each other honest with the pairing and even if we weren’t doing TDD then we were writing fairly OO code that had at least a few tests per class. And, we were really cranking it out. 3 people,  1 month and nearly several thousand lines of code written. It was classic “three guys in a garage” stuff.

And then, of course, the wheels come off.

I got pulled onto something else, and the project got split in 2 so everyone now working on their own decoupled piece. And when I checked back after a few days I find that; quelle horreur it’s all gone wrong. 80% casualty rate on the automated tests; 20% progress on a deliverable and the rest of the time spent changing the structure and putting out the resulting fires. OK the changes were legit; required to get the foundations right and even agilistas like that when there is a business case for it (and there was).

So what did I learn:

• Focus matters: One person doing one task is much more productive than one person doing two tasks. It sounds obvious but I really believe that the difference is huge.
• Teamwork matters: Two people doing one task are much more productive that two people doing two tasks. In this “controlled” experiment with the same people doing the same tasks the difference was significant. The extra person keeps you honest and keeps you on track. They stop you getting distracted by other non-project tasks but more subtly they give you moral support when you get stuck.
• Test-first matters: working alone it is easy to slip and not test first; or even have a test in mind. Headphones on and in the zone you can be hundreds of lines of code in trouble before you realise that you aren’t even going to be able to test this. If you can’t make a test, you don’t have working code.
• Deadlines matter: A 4-person team working on a one-week deadline work harder than one person on a one-month deadline. The tight deadlines of agile are important; they Keep It Real and keep it scary. You won’t get scared when you have a 3 month deadline until you run the numbers and see that – even if it all goes to plan – you have 100 days of work to do. And even then you won’t be scared until you realise that in each calendar week you probably only get 3.5 days of solid work in.
• Mood matters: when we were cooking with gas, it felt good. We were all upbeat and had a great positive feedback loop going; the more productive we were the more we wanted to get done.

The beginning of the end?

So. Last week I had an experience which I think was a significant step in my programming life/career/whatever. I realised that I was about to rewrite my first big application. And when I say re-write, I don’t mean v2.0. I mean: I was faced with a problem; I came up with a solution to it that was quite specific; after a few seconds of thought about how to implement it I realised that I had already implemented it years before. As far as I know that system is still in 24×7 operation with the kernel of the code untouched by other developers. So it was clearly good, so why not just re-use it? That is good, right? We want to re-use code. Or even if we can’t re-use the code we want to re-use the pattern (in this case it was using .NET attributes to decorate strongly-typed classes so that they could be turned into loosely-typed data-transfer objects). All good, right? Even if I couldn’t use the code, I knew that the concept would solve the problem and solve it well enough to last years of subsequent tweaks.

Well, in spite of that, it didn’t feel good. It felt like the chill of old age. It felt like this is the beginning of the end. There are no new problems to solve, they have already been done, even by me. All the rest of programming is just repetition, automation and abstraction. Of course, this is probably what it has always been; ever since Turing became complete people have probably been sighing over the fact that they are doing the same thing as they were last year only faster. The shocking thing is that my father used to say the same thing; having worked on minicomputers in the 60s for doing critical path analysis he was forever telling me that “computers haven’t changed”. He was wrong of course; the scope of the problems had changed because there was 100,000 times more processing power, but I suspect that the programming problems were just the same.

The question is, do I still want to keep doing the same thing over and over? Just a little bit faster with less bugs than before. I wrote another application this week. It was, I have to say, well structured using all the tweaks that ReSharper could throw at it. The methods were tiny and readable; the code was flexible enough to cope with change but it wasn’t over-designed; I cranked it out in 4 days but it ran first time (or maybe second..). There are, as I’ve said before, no points for difficulty and the business people who wanted it would probably be happy that it got done in a week with no problems. Next patient, the doctor is IN!

But how long can it go on? The same tired old business problems; more reporting tools; more transaction processing; more stupid little GUIs with one more button to handle special case X. I mean, you can’t avoid that. The world moves on and programs rot. You can’t be writing earth-shattering middleware every week, and as I’ve said before, the stuff that becomes the new backbone of an architecture doesn’t get written, it gets extracted from a running application (e.g., Rails). But seriously, how long can people stand to do the same thing over and over. A little more abstraction isn’t enough; to get from c++ to MFC to c# in, what? 20 years?

All I can say is that I won’t be doing this in 20 years time.

http://en.wikipedia.org/wiki/Critical_path_method

The problem is not the problem

When someone starts to tell you about something that they need to get done, already you are thinking about how they can solve the problem. You are thinking about technology and all the things you can do with it; they are thinking about the task that they need to perform.

Actually, they really aren’t thinking about that. They are thinking about what they need to do to get a pay rise; they want to get a promotion; they want to improve the performance of their team so that their team doesn’t get taken over. So with all these things in mind, is it any wonder that they can’t turn out a decent spec, or a decent set of requirements; they can’t even answer questions like “what do you want it to do?”. They do know what they want it to do: they want it to get their boss fired so they can get their job; they want it to get their kid a dog; they want it to enable them to leave on time.

As my old metalwork (*) teacher used to say, first define the problem. The real issue here is getting people to tell you what they already know, then capturing it in a structured way that you can turn into a list of the nouns and verbs of the system; and while you are at if, figure out where the gaps are. And you have to do this in a language that you can both understand and you have to do it while carrying on a normal conversation.

Mostly, people fall into two groups; those who think that computers are amazing things made of spells and spiderwebs that can do anything and the people who program them are able to get the computer to do anything. The other group is still in the dark ages of desk calculators and any time the application does anything they want it to spit out everything that is related and not do anything until it has a good few clicks to encourage it; they are the ones who always want another “are you sure?” dialog box.  There is more than a grain of truth in both camps, so be gentle with them.

Actually, the real, real problem is getting them to engage with you at all. The real problem is making them see that if you don’t know what you want, you definitely won’t get it. The problem is, as all interesting problems are, a people business. Getting people to tell you what they want and then explaining that they can’t have that, but they can have this other really interesting thing that they didn’t even know that computers could do is all part of the fun. Actually, it is all part of the fun working on any kind of agile team. I pity the poor suckers whose only contact with the client is the bug reports that are fired at them by level 2 support (and they don’t even get to see level 2 support, they don’t even get an email from them; they just get a bug feed).

As with all people problems, there is no no simple solution and rarely is there some sort of break point where it all becomes clear. With people problems it’s uphill both ways and the wind is in your face. We should be grateful that all we need to do is get people to talk about what they want out of their computer systems and not what they want out of life. Anyone who is a real manager – or even better an HR manager – knows that is the real secret sauce of a productive organization.

In any case, the problem is not the problem; the problem is getting people to realise that unless they face their problems and take responsibility for the solution instead of handing it over the IT butler to sort out, then they won’t get what they really want.

* – Actually, he wasn’t my metalwork teacher; by the time I was doing it, the subject was called Design & Technology. He was a rather odd chap was Mr Tooze. He preferred to sharpen his pencils with a piece of sandpaper and then wipe it on his sock; oh and it isn’t called sandpaper it’s glasspaper; and he had a tic in his face tha made him look like he was blinking; and he spread his glue using carpet cutoffs that he kept for the purpose. And whenever he heard car tires screeching he would always say “I know that driver is under 21 years old” and then proceed to tell a story about how he had been the first on the scene of an accident where a motorcycle had hit a sportscar head on; the motorcycle had cut the car in half and the irst par of the motorcyclist he found was his foot.