Too busy to change: Experiment to find upsides, limit the downsides

OK so you’ve learned something new; you want to start doing it for real. How do you find the time?

Orson Scott Card said that

“The essence of training is to allow error without consequence.”

Learning stuff is great, but it’s only useful if we can put it into action. Unfortunately, we can’t rehearse the relevant skills in isolation. There is no training arena to test out project management systems, so we have to blend our training with our work.

Agile Vs Waterfall.

We have to find a way to learn and try things out while doing “real work”. We must look for options that have an upside, but find a way to control the downsides. If we can’t control the consequences, then we will be too afraid of making errors to try anything new. And the longer that goes on, the more risk averse we become and the easier it is to say “it’s worked up until now, why change?”

Also: manage upwards; get them to buy you some time and help you control downsides; the payoffs will come.

Here’s a few suggestions, all rather obvious by now:

  • Book some time. Pick a spot in your diary and block some time every week.
  • Start small. Pick something isolated, where you can take a risk. And deliberately take a risk and do something new.
  • Don’t delay. Waiting for the right moment is probably not a good idea, it’s unlikely to ever come along. OK yes, don’t pick an absolute crisis either, but everything is relative; most of our work crises are “first-world problems”
  • Learn with others, learning can be social experience. If you are learning together you can reinforce each other’s commitment to try new things
  • Learn from others on the same path if you have to do all the experiments yourself, you’ll age to death before you figure it out. Share experiences. It’s hard to know how much their experiences apply, of course.
  • Don’t stop many small experiments is better than one big change. Don’t settle for just trying something new once.


Be patient. Maybe the new thing won’t work first time? That doesn’t mean that it won’t ever work, or that doing new things is a waste of time. Share your experiences with your peers, be honest with your manager and your team about the fact you tried something and it didn’t quite work. Ask for forgiveness, rinse and repeat.

Moral Map of the Web

Enjoy this.

I liked this part, where MS is a dark, satanic mill, vomiting poison into the pastures of the web. Harsh.

And I also learned a new phrase: Paving the cow path

Mostly I think of technology as a job. Then when I read something like the UK government digital task force blog, it seems like a noble quest. Making the world a better place, one crappy government department at a time.

PowerShell: quick custom object

TL;DR: I parse some filenames and pack the results into a quick custom object.

I found that I had a lot of files to go through, they were all dated very nicely:

Sadly, some of the dates in the filenames didn’t match the date modified, so I can’t use that to sort them.

And I can’t use the filename because of the date format doesn’t sort correctly. If only they had nicely formatted them as 2014-02-28, then all would be good.

PoSh it is then:

#get the files into some data 
$files = gci "\\folder"

# grab the names
# pick out the date with a regex, remember to out-null to get rid of the True, True, True, True... that will come out of the pipeline
# then take up the matches using the $matches object, packing them into a little hash table, the hash will work as a hobo object
$result = $ |% {$_ -imatch ".+-(\d\d)\.(\d\d)\.(\d\d\d\d)\..+" | `
Out-Null; @{ "date" = (get-date -Day $Matches[1] -Month $Matches[2] -Year $Matches[3] ); "filename"= $_}}

#select that out, like a boss, find the file you've been dreaming of
$result | select {$_.filename}, @{e={$}; l="date"} | sort date

PowerShell : slow data in a file

Say you have a script that gets data from a lot of different places, it’s slow to run when you are developing but you probably want to connect to them all when you run for real. If that’s you, then you can save the results of your slow connections in a file with just one line of script and keep using that file unless you have to re-get fresh data.

I’ve found using PoSh’s Export-CliXml and Import-CliXml to store the results from a slow operation nice and easy, but it can output any PoSh object.

 = (join-path $PSScriptRoot
 $resultsFileRedo = $false
$bigListOfThings = get-ListOfThings
 write-host "about to get the slow data from $($bigListOfThings.count) things"
 foreach($eachThing in $bigListOfThings)
 write-host "doing $eachThing"
 $eachThingData = Get-SlowDataThatTakesAges $eachThing
 $slowData.add($eachThing , $eachThingData)
 del $resultFile
 $slowData | export-clixml $resultFile
 $slowData = import-clixml $resultFile

Essence of Agile

You can, of course read the 12 principles of the Agile alliance; sometimes I find them hard to swallow. I also can’t remember more than 5 things.

Here’s my take:

  • Smaller packages of work delivered more frequently
  • Subject matter experts working directly with technology builders reduces the risk of waste

And that’s it.

Of course, it isn’t. I can’t say anything succinctly.

Smaller packages of work, delivered more frequently

“smaller” is a relative term, and Agile is a relative concept. The idea is that you try to make and deliver the smallest thing you can. The smallest piece of work you can do independently of other pieces of work. Why do we do that? Well firstly and most importantly is that it seems to work; and works much better than any other approach. Smaller packages of work means that there is less overhead in tracking and staying focussed, less problems of scope creep, shorter meetings and shorter documents.

Also, builders don’t try and solve problems which you aren’t trying to solve, so code can stay lean. People who are smart may start solving other problems that they “know” will come up; in Agile where the problem is evolving then you don’t “know” anything, you are doing the wrong work.

So here’s where some people get lost. They think: “whoah. You should stop the problem from evolving and nail it down before you start to work!” Well that sounds great, but it’s proven not to work in practice.

The problem is evolving for 2 reasons:

  1. The world is changing and you can’t keep up by doing everything up front, you just can’t.
  2. People actually don’t know everything about the problem when they start

You might think that it’s the best thing to dive in deeper and deeper until they do understand it all, but you are wrong. Making people wait for easy and urgent parts of the system while you analyse the unimportant but very complex parts of the system means two things:

  1. You end up with a design that is bloated and complex and heavily clipped to the problem as it is today, it’s not going to evolve to vNext.
  2. Your company will go bust because you don’t have a system that allows them to track orders or trades or whatever.

“delivered” is also a subjective term. What you “deliver” in your project as scoped, designed, coded and tested might not be what goes live this week. You might have 2 different definitions of “delivered”, with 2 different sizes of a package of work. Really what might be happening there is that there are 2 back to back agile teams, one making features and one rolling them out. Or it might not; see Kanban.

“delivered” also means like really delivered, you have to understand that being Agile doesn’t mean cutting corners. It doesn’t mean doing less design or less testing or less documentation of those things. You just do them in a smaller cycle. It’s not just about being done, it’s about being done done. Which means really cleaning up after you, so when that piece of work has to change and adapt, as it probably will, they aren’t flying blind. The absence of a spec and a BDUF means that you have to back fill as you go.

“more frequently” is also a relative term, and it doesn’t mean faster (see next point). It means that you “ship early”, yes; but you also “ship often”. You have to repeatedly deliver, only the act of actually delivering something means you understand the process of delivery and didn’t just get lucky. If you don’t deliver repeatedly you aren’t removing risk. You have to give yourself a chance to learn from your mistakes. And, repetition makes you automate and automate and automate.

Work that is in the pipeline and not delivered is the same as Amazon having 5 million TVs in the warehouse; Amazon had to pay to get them but they aren’t paid for them while those goods sit there, depreciating in value.

Subject matter experts working directly with technology builders reduces the risk of waste

Really this is where Agile is faster; it isn’t any faster to design and build software incrementally. It isn’t slower either, not really. The real advantages are in the lack of waste. You never build the wrong thing, even if you need to redesign it.

“subject matter experts” are not analysts or testers, though they have to do a bit of both in the micro scale. In the classic waterfall model, you would have subject matter experts (what we sometimes call “the business”) working with analysts who are skilled at analysing things (e.g., MECE, 5 whys, etc) but aren’t software people. The analysts work with architects and testers, and they work with developers… (and no one works with the sysadmins who run it in production, of course J). What we change in Agile is to shorten the supply lines and bring the work suppliers (the business) closer to the consumers of work (the builders). That’s great as there is less confusion and better communication.

It’s harder though because everyone needs to be better at communication, more multi skilled, they need to be “requirements engineers” and not be drawn into too many corner cases, they must be their own best architect, a solid tester and they must cut code as well. Not everyone needs to be a master in them all, but you need awareness and close proximity to someone who can help.

“the risk of waste” is high in a waterfall project. You build too much without checking it works together and in live. What makes Agile work better is the small package of work. There is less waste because you stay focussed on what’s important and that smaller package of work is easier to analyse and easier to code and test correctly.

In technology all decisions are reversible*, it’s just a matter of how fast you can reverse and how much you waste. Rework is essential to Agile, and not something to avoid. Rework isn’t waste, it’s sharpening the saw.

That’s the thing: software work isn’t like building it’s like designing. And if you’ve ever worked with a house architect or a graphic designer you’ll know that you repeat as often as you need to and feel happy that you are getting what you want!

“working directly” means direct, face to face conversations, ideally in the same room. Sitting together doesn’t mean that you get the trust, openness, candour, belief in each other and sense of fun that you need to make an outstanding Agile team. But being together and repeatedly delivering small pieces of work gives that confidence in each other, and its real confidence. Not acquired on a rock-climbing course or a boozy night out, but real trust in each other’s ability.

What goes wrong?

Well nothing’s perfect is it?

Working in the microscale can cause some problems. If you are just starting a project, operating with a team that is unskilled at Agile and there is little to guide you technically, then you will make some wrong turns, for sure. Doing Agile design is a skill, and when you do it for the first time you might get it wrong and design too much or too little. But if you pick a tech and a design that is “normal” and composable, you will probably be able to fix it.

You might not need an overall architecture, though that will help if you are in steady state agile delivery. You might not need an overall “guardian” of the quality, but then again, you might. It depends on the relative pressure for delivery from the business, the overall skills mix of your builders (i.e., their tendency to under- or over-design).

One of the things that can go wrong when the team is new to Agile and the tech/builder team is politically much weaker than the business team is that the business team thinks that this is a licence to do whatever they want and get something for nothing. Separating out tech work into separate stories that never get done is a sign of this. Plus all the normal bad smells of imbalance: project crunches, tech people of low status, blame games, etc. Just like waterfall projects, you will not know that you have a problem until the end and you look back over many cycles with code quality declining and support issues draining the team.

Correcting that balance can’t be done with a process only with communication and the building of trust. But the iterative process gives you a chance to revisit the start of work over and over again, and start fresh.



*as opposed to medicine, where limbs don’t grow back. Let’s also assume that the bad decision doesn’t bankrupt the company.


There are 2 types of meeting. Discuss.

Meetings. Terrible things; but like democracy they are the least-worst way of solving the problem*. They can be a terrible waste of time and they are the only way of really getting the best from those terrible, frustrating, wonderful, inspiring, brilliant people that are our co-workers. They are the best high-bandwidth, bidirectional, audio-physical media that works for our 3d simian brains. How can we get the best out of our meetings? I think that the answer lies in that excellent reasoning device: the barbell (i.e., there’s something at either end and nothing in the middle). There are 2 types of meeting:

  • The Decision meeting
  • The Discussion meeting

Once I’ve said the names, there isn’t much more to say, but let’s grind through it.

1. The Decision meeting

The Decision meeting is pointy. The ultimate example I’ve seen is the decision meeting for an investment buy/sell. There are defining characteristics of the decision meeting:

  • The point of the meeting is to make a decision, the decision makers should know who they are before the meeting
  • In order to stop debate, the decision should be laid out in a document, before the meeting
  • In order to stop groupthink, each decision maker should make and submit their decision before the meeting
  • There must be strict record-keeping; minutes and decision notes must be recorded consistently
  • In IT, we like data. Recorded decisions should be mine-able and have a confidence level. Or use the Apache voting system.
  • There must be rules of engagement; the format of the meeting should be known and every point covered.

If all this is done before the meeting, what is the point of getting together? Well, you establish the agenda, set your own opinions and then drill into those opinions and see if anyone needs to change their mind. If you’ve ever watched The Apprentice, you know what happens when you get people together to make a decision without a strong, reasoned opinion. You get a shouting match and then vicious reversals and selective memory after the fact. Don’t do this. Of course, all this is great but you must do a lot of work to get there. The hardest thing is building a decision to decide on. You have to put together an exhaustively documented plan of action/architecture/strategy to vote on. That must be close to what people want or you’ll be lost in debate. PGMs are good because the answer is always buy/no buy. The other thing is, this decision meeting doesn’t have to be a democracy: the decision maker might be one person who ignores everyone else, but if things go bad then the documentation is there to fall back on. Meeting minutes are a cleft stick; vital and yet timewasting they deserve a post of their own. One thing is certain, publishing meeting minutes is not a way to communicate what you are doing, you have to do something to make them digestible.

2. The Discussion meeting

The Discussion meeting is schmoozy, smooth and free-flowing. In order to get the best out of this you really need an agenda, if you don’t have an agenda you better all know the problem well. You might be having some early stage meetings where you don’t even know what the agenda might be, but you better have an opinion before you walk in the room. These meetings will be useful when you are hammering out what the options are, or for finding out what the problems are. I’ve found that if the meeting group is always the same and you keep talking about the same focussed subject-area, you soon develop a team spirit and a shared context and vocabulary that enables a productive sharing of ideas; if the meeting attendance changes or meetings are infrequent productivity drops drastically as we need to reset each time.

There is a special case of the discussion meeting that is, in my opinion, the best type of meeting: the one-to-one.

These are the meetings where you share ideas freely and inspire each other. This is one case where it might be OK to not have an agenda, the very act of having to talk it out will clarify things and throw up angles and aspects that you would never find alone. See also Programmer’s dog. This is where I and the core values split: I think that brainstorming without focus is a disaster, but balanced, thoughtful, egoless collaboration, where you abandon your idea when a better one comes along… for a certain type of creativity,  there is nothing like it.

And of course, finding out how people tick will only happen in this type of meeting.

Types of meeting to avoid

  • The status update: Utter fail. Send me an email. Or don’t, even better.
  • The boxing match: One person talks, someone else argues with them. Everyone else listens and checks email on blackberry.
  • Any meeting that DOESN’T START ON TIME
  • Any meeting where people check emails other than to make the point that “this meeting is pointless”. TBH, sometimes I do send myself reminders from a berry, but until we have in-office multi-user Siri in every room…

Conclusion Know your meeting, value your colleagues time. Choose one or the other and make sure they know which it is.

*  Churchill said that. Amazingly here are the Hansard minutes that say so. Meeting minutes are a wonderful thing. (see first few lines in the linked page for comments on democracy)