The Agile Waiting Game

This post was originally written as a guest post on the MagRails Conference blog. You can find it here.

Agile practices (pair programming, TDD, continuous integration, etc) and methodologies (Scrum, XP etc) are intended to increase the productivity of teams that adopt them. In one sense you can think of it as “team + practice = improved team”. The bit that gets the most focus is “practice”; what we will explore here is the nature of “+”.

Filling up

If you want to take a bath, initially the bath will be empty. You make an intervention to fix this undesirable state: put the bath plug in the plughole and turn the taps on. Is the bath ready now? I mean, now? Unless you’re reading this over a modem from 1983 and it took you an hour to get this far, the answer is no: the bath hasn’t had time to fill yet. There’s a delay between turning the taps on and the bath being full, somewhere in the region of 10 minutes. This is significant: it means you can go and make a cup of tea without fear of flooding the house. It also means that if the bath isn’t full in half an hour, something is wrong. (Perhaps the plug is leaking slightly, or the flow through the taps is too slow.) Whatever happens, we intuitively know how to use this delay to manage “bath + running taps = full bath”.

The same idea is equally – if not more – important in software teams. Without a sense of the delay between the original state and the desired state given a certain intervention, it is easy to over- or under-manage.

Say your team currently integrates and releases on a 6-month cycle, at the end of which usually follows a month of mad scrambling and tail-chasing to deploy the code. You want to improve this, so you intervene by introducing fortnightly iterations, where code is supposed to be fully integrated at the end of each iteration. If after a month of this (2 iterations), the team’s productivity is down, changes are not being fully integrated, and not all integrated features are fully working, has the intervention failed? At this stage, there isn’t really enough evidence to suggest this. An understanding of the nature of the intervention and its inherent delays helps here.

Moving to frequent iterations reduces the batch size of work being integrated. A manual integration process twice a year may be fine; the same process every 2 weeks will show enormous waste. Therefore the team will need to automate its integration process, but this will take time. This effort will take capacity away from existing development, so in the short term you would expect measured productivity to go down. (I’ll ignore any debate on actual productivity for the purpose of this example.) Also, with more rapid integration, team members will also be forced to communicate more frequently, and meetings will need to be made more focused and resolve conflict faster.

If 3 months later, the same pain is still being felt, has the intervention failed? Now the answer looks more like yes: it should not take most teams three months to achieve significant improvements in a deployment process. However, the the real answer is context-specific, or “it depends” as it’s known in the consulting trade.

There is no need to pick just one or the other out of the two (arbitrary) time-points: reacting at 1 month or 3 months with no information in between is still less than ideal. When filling a bath, you don’t need to wait until the end to see if it’s ever going to fill: by checking occasionally, you can see if it filling as expected. You may be able to see water leaking as it becomes half-full, or you may see it filling slowly and realise you only half-opened the taps. This extra feedback lets you manage more effectively. But if the bath takes 15 minutes to fill under ideal conditions, and you’re only 10 minutes in, don’t expect it to be full yet.

You wouldn’t blame a bath for not filling quickly enough, so don’t blame a team for not improving fast enough, unless they are actually goofing off. Slow progress may be due to a new practice coming into conflict with existing policies or mindsets (which you should watch for), but even under ideal conditions will take time.

Draining out

The reverse situation is also important. When you take the plug out of a bath, it takes time to empty. Imagine you filled the bath and got in, but half an hour later – to your bemusement – you realise the water level has dropped significantly. What gives? It may be that there’s a tiny crack you didn’t notice when filling it, which has allowed water to seep out. Again, the inherent delay and your own perception means it took a while for the problem to become apparent.

Software development teams can have leaks too. Any time a process which maintains long-term productivity of is dropped or reduced, the team’s performance will also start to fall – but it will take time for the problem to appear. Reducing TDD will not cause the defect rate to rise sharply, nor the ease of adding features to fall sharply. Instead, what will happen is that 6 months down the line, customers are complaining about the increasing number of bugs, and developers are complaining about the increasing difficulty of fixing them. The delays between different people’s responses will not be uniform, either, so watch for the early complainers – far from being a nuisance, they’re the system giving you a hint of what the future holds.

There’s a particularly insidious situation that can emerge when practices are dropped, known in systems thinking as “shifting the burden”. Here, some team members will pick up the slack, increasing the delay before the fundamental problem becomes apparent – which it will do so dramatically.

Again, you wouldn’t pull the bath plug out of a bath and exclaim “The bath isn’t empty! We didn’t need the bath plug after all!”, so don’t let a team reduce or drop practices it knows to maintain productivity, only to act surprised when things fall apart later.

A few delays

Start looking for delays in your team and software development process. As a starter, here are some examples:

  • learning a new language or framework
  • changing the practices in use (including learning a new one)
  • changing the members of a team
  • changing suppliers
  • taking on or dropping clients
  • learning to look for delays

If you’re too busy in the day to think about the delays around you, think about it next time you’re running a bath. What are your thoughts? Feel free to leave your comments below. I’m happy to wait for them.

My name is Ash Moran. I’m a software developer and agile coach, and owner of PatchSpace Ltd (Twitter). If you have any feedback, questions, or would like to know more about my services, feel free to contact me at ash.moran@patchspace.co.uk.