You can read the intro to this series here: On Agile Planning - An Intro

Why is prediction of project duration so tricky?

Just a walk in the park

Imagine if you will, a pleasant mountain scene. It's the summer, and I have invited you on a brisk walk in the crisp air.

As we walk, I give you a challenge. I want you to predict for me how long it will take you to walk from the spot where we currently stand to a tree about twenty metres away. The terrain is good, flat, solid and the weather conditions excellent. Whatsmore, you can take as long as you want in planning. The only rule is, you have to walk in even measured steps - you can't run!

I've issued this challenge (although to be fair not in the same beautiful settings) enough times now that I’m confident people are actually pretty good at estimating this. Usually they come in within 10% of the actual time and steps.

And frankly, this kind of makes sense. We humans spend a lot of our life predicting things in controlled, highly local settings, and we've developed pretty good heuristics that help us ascertain how quickly and safely we can move from one side of a room to another.

Bravo us.

The impending storm

Hang on though - what's this? After successfully completing the "to the tree and back" challenge a few times, a category 2 hurricane has just rocked up. The winds are gusting up to 177 km/h in an effectively random pattern, for several seconds there will be no wind at all and then a gust will come through so strongly that you'll be unable to proceed.

The introduction of this single factory of uncertainty into the system is going to wreak havoc on your ability to predict the time and number of steps required to reach the tree.

As you brace against the wind, you realise that the best you can do is make an estimate of the range of time it might take you - but given the gusts can entirely stop you in your tracks (or worse, push you backwards) the short distance could in effect take you hours, or days! Your range, if you attempt to estimate one, is going to be so broad it becomes worthless.

Welcome, entropy

The storm in this instance, is an example of what in complex systems theory is called entropy. Before the storm, the system (made up of yourself, the mountain, and all the atoms it entails) was, from your human perspective, in a relatively stable state that you as a human being are highly experienced in predicting.

If you think of it like an equation, the result (number of steps taken) would have depended on a number of static (from the perspective of a human being) variables. The floor is solid and flat, your locomotion, whilst insanely complex at a molecular level is, at this macro level and without further external modifiers, understood, and could be represented by some straightforward velocity parameter. The grass holds such a tiny possibility of spontaneously evolving into molten lava that it can be safely ignored. This equation would have been relatively straightforward, and you were capable of solving it without too much grief.

Now that the storm has arrived, the equation has suddenly become much more complicated, featuring as it does the non-linear (read, random) effect of the gusts of wind. And of course, note that we are, for the purposes of this simple example focusing entirely on the direct impact rather than considering things like stamina or your propensity to become injured.

Higher entropy, simply means a higher complexity to the equation. More (and more complex) moving parts, so to speak.

As it happens I've witnessed such an attempt first hand. In an alpine ski trip several years ago my good friend Ross Jackson and I were snow boarding along a relatively simple piste (blue on the European scale) and enjoying the good progress we'd expected to make. Suddenly the weather, which had been reasonable but not fantastic up to that point became much worse such that we were forced to take shelter next to an old mountain building. Ice wind battered us like needles and visibility had dropped to just a few metres.

As we'd skied this particular terrain a number of times, Ross and I knew that we were precious little distance from the entrance to the funicular train which could have taken us safely down the mountain, perhaps thirty metres at most. However, the entrance was entirely invisible due to the conditions and the wind was whipping icy snow into our face that even with the shelter felt like it was piercing the skin.

Ross wagered that we could make the walk and wanted to give it a shot. I disagreed. He took a breath and bravely pushed forwards into the storm. I cowered in the small dug out that was our makeshift bunker.

At first, he made quite good progress, but just as he reached the edge of sight from our place of safety I saw the wind literally pick him up and throw him backwards.

Not perturbed, he braved on and for a minute or so entirely disappeared from view. I was beginning to ponder how I would explain his demise to his girlfriend when suddenly he reappeared and recounted several more mid air heroics before he'd decided this wasn’t a viable plan at all.

In normal conditions, this would have been a simple prediction for Ross with very limited chance of failure - the increased entropy brought about by the storm however, made Ross's prediction completely off. The truth was that Ross had never experienced a similar physical context. His very human brain however told him this didn't matter and incorrectly inferred from prior experiences that success was within reach. However bold and impressive the effort, it was not.

Once you step outside of your everyday, highly stable set of experiences (walking, breathing etc) you are immediately exposed to these types of random external modifiers to the system.

If the system is your project then this is going to trip up any projection you might attempt - and the more moving parts (people, tools, markets) the less likely you are to be able to predict progress - just like the weather.

But... we can predict the weather?

So the wind is causing us problems. So what?

Weather, of which wind is clearly a subset, is deterministic. By deterministic I mean that it is governed by a number of macro level laws well understood by physics. Provide exactly the same inputs and you will get exactly the same gusts of wind. Every time. So technically, if we could somehow measure the wind velocity, pressures and temperatures at the edge of the system (maybe the edge of the mountain), we might be able to predict the exact timing of the gusts and therefore predict the exact time it would take us to reach our target tree.

I told you that you could take as long as you wanted in planning.. so we have time to make those measurements and projections and get an accurate estimate, right?

This would also be a nice parallel to our projects - if we put our minds to enumerating and measuring all possible variables up front then, assuming everything is deterministic like the weather, we could predict it!

Chaos Theory (wind isn't really random, but that doesn't mean it's predictable)

It seems this is a good moment to introduce Edward Lorenz, the father of chaos theory and rather usefully for our alpine storm example, a meteorologist.

Lorenz made his initial discovery trying to predict something highly aligned with our example. In 1961 he was making some weather predictions using an early computer called a Royal McBeeLGP-30 (side note: this was called a desktop computer, not because it fit on desk but because it was the size of one!).

Lorenz had already modelled a simulation of a particular storm previously and wanted to see the sequence of results again. This meant running the whole program again, which due to limited computational power was going to take some considerable time.

Luckily he had some mid point calculations from the prior run recorded and so to save time, he started the simulation there - inputing this initial starting data from his saved records. Given the algorithm was entirely deterministic he was confident he would see the same results.

Except he didn't. In fact the result was completely, wildly different to his previous calculation. Lorenz, checked and double checked his input data, but the result was always the same.

Eventually, after considerable debugging, Lorenz discovered that the numbers on the print-outs he'd used to save the data from the prior run were at 3-digit precision. However, the computer was tracking data at 6-digit precision. Most readers will have heard, perhaps on an almost daily basis, the utterance "it's a rounding error, we can safely ignore it". Sometimes, this might be true - but not, as Lorenz discovered, when you're entering the world of complex systems, systems with high entropy where the result is determined by a complicated interaction of elements.

What Lorenz eventually realised was that these tiny differences in input precision, say from 0.506428 to 0.506 on the printouts were, for the first few days of prediction giving the same answers, but then, suddenly would diverge massively from each other.

And so chaos theory was born, the observation that due to the exponential ability of multipliers in complex systems to alter the result, tiny changes in inputs can dramatically change the outcome. As Lorenz put it in his presentation: a butterfly flapping its wings in Brazil could eventually cause a hurricane in Florida - the butterfly effect.

The less Hollywood definition however might be even more useful to us here:

Chaos: When the present determines the future, but the approximate present does not approximately determine the future.

The result of all this, was that Lorenz realised that the weather was not predictable (even by modern computing standards) beyond just a few days. At that point even tiny, subtle changes to input (that are too small, or perhaps too numerous for us to track) effect the result leading to wildly different weather predictions.

So what does this mean for us? It means that some complex systems (of which our projects are an example) are not predictable beyond a short period of time - after which they may diverge from our plan a little, or indeed, exponentially.

Chaos is hitting you from two angles at the same time:

1. Anyone of Lorenz’s butterflies could right now be causing an emerging storm (physical or metaphorical) that will massively derail your project - for example a market deviation, loss of a key team member or internal or investor revolt
2. Your project is full of human beings who do not estimate things like task duration very well - despite best efforts, people's estimates are usually wildly off - including mine and especially in tech.

The more complexity in terms of people and tools, the more complex the jigsaw puzzle, which in turn allows a chaos to emerge.

External elements build on internal ambiguity in terms of scope, estimation and external modifiers, combining to create a web of variability that will, over time, combine and interact with each other to cause truly unpredictable results.

All is not lost however. Some things can be planned without hitting these obstacles we just need to make sure we understand which things, and what the rules are if we venture outside the bounds.

Part II - (what can be planned, and what can’t) - coming soon.