Tuesday, January 27, 2009

IT Development & the Tory Proposal

Today's announcement by the Tory party on the capping of expenditure for IT projects and reviewing the way they are run is good news for some, including those consultants who've made a living out of plugging the ISO 9000 standards for systems documentation.

I'm all for this proposal by the Opposition, for the sake of future generations we're going to need some Agile thinking to reduce the burden of debt left by the incumbent spendthrifts.

Government projects are great at delivering 2,000 page documents that describe, in somewhat eye-watering detail, what the system is supposed to deliver. Apart from creating a physical hazard for those designated to carry these tomes around, or those unlucky enough to trip over 4 inches of high quality paper left lying in a corridor, this type of document does little to deliver a quality system. Studies claim that delivering this type of traditional requirements only 25% of the specified features are used frequently, and the rest of the system is used either infrequently or not at all.

The basic problem is that it's very difficult to write down in unambiguous detail how someone is going to interact with a system. You often find that words don't work well and even the best pictures only tell part of the story. Developing software is a creative process, more like creating a piece of art or writing a novel or poetry. There's a great deal of skill and craft involved in asking the right questions, capturing the, sometimes non-obvious, interactions between the people in their various roles and their use of the system.

An example of this interaction is the beloved ATM, as told by a friend of mine who's done this before. Ever noticed that when you go to the machine and take out cash, you get your card back first? Think about what would happen if the cash came out first? What are the chances you would take the cash and leave the card behind in the machine? Pretty high right? Think about how many times you've left a receipt behind when you asked for one to be printed.

Think of building software in the same way that a director takes a great book and turns it into a movie, or perhaps a long running TV drama. The written word does not translate well to the visual. You can't always reflect the nuances of a character, nor can you burden the viewer with the drudgery of a narrative discussing how tortured the hero of our story is. A good director is sympathetic to the original text but savvy enough to make it work in the visual world. There needs to be compromises made along the way that allow those who've read the book to feel that the story is a fulfilling realisation of the story. Compare the film Pride and Prejudice with the book; a reasonably good rendition of a great book. On the other side, think about Frank Herbert's Dune. A great book scared by David Lynch's bizarre and poor interpretation.

A big issue that faces software development is that managers believe you can take engineering project management disciplines and apply them to developing software. This is the traditional "deliver it by this date" approach. Sponsors want detailed estimates that go out for months and years in advance. They want certainty that your plan will deliver what you said, when you said and then they flog you when it all goes wrong at delivery. Time and time again the waterfall approach of extended analysis, design, development and testing fails to deliver the goods.

The reason for this is that building software is completely unlike building a house or office block, yet we in the projects industry still try and pretend that we can apply the same rules.

Firstly, people have built houses for about 10,000 years so there's an agreed set of steps that you need to take. To build a decent sized house, you need a foundation, or it falls down. You need properly supported walls, or it falls down. You need a good roof or you get wet in the winter, and then it falls down. And so the list goes on.

For tall buildings, it's a little more complicated and cost overruns occur because the wrong things happen at the wrong time. For example, you want the electricians to cable up a floor and this needs to be done before the dry rising is installed. If someone decides to install the dry-rising first, then the management contractor will then need to sort out who did what in the wrong sequence and deal with it. The upside is that without the electrics, you can't use the building. The contrast with software is that we try to pretend that delivery 0.1 that doesn't have any electricity is useful. At this point the battle's lost.

One of the big differences with buildings is that until the entire house or floor is fitted out it's difficult to use it. With software, you should deliver working components along the way that are ready for use; this also allows both the developers and business users to ensure that what's delivered meets the specification. If not, and you should discover this long before you reach the test driven UAT (User Acceptance Testing) stage, then you need to decide if you're going to fix it in the current cycle or defer it to the next one if it's too much work.

This might sound easy but it's not. Developers always underestimate how much they will deliver and business users also overestimate what they're going to get. A smart and agile project manager will ensure that both sides expectations are met.

No comments: