Wanted: A Simple Measure of Success in a Complex World

Speaking at the World Bank, Ben Ramalingam, author of Aid on the Edge of Chaos set out a challenge to those working in the humanitarian and development fields: Move away from a narrow focus on what we think is important and take a more wide-angle approach to the issues we’re dealing with.

“Responses to complex challenges need to be adaptive,” he said. “Rather than strategies for best practice we should be looking at strategies for best fit.”

Ben discussed his plans to push the debate on change in the development system during a lunch we had about two years ago where we continued a conversation we had started years earlier when both of us worked on evaluating humanitarian assistance.

We both agreed that in a world riddled by unpredictability the usefulness of linear models was limited and that there was a pressing need for rethinking how development assistance works.

Ben’s book argues that our models are based on simplifications that make assumptions, eliminate real life factors, and that fail to reflect that the world is a complex maze of interrelationships that affect each other.

Tools like the logical framework, as Ben says, can if “[d]one right...make[s] users think carefully and systematically about their plans, and how activities will contribute to goals.” Drawing on many evaluations, he observes, that the tool is often used mechanistically: results-based management and M&E systems are typically focused at input-output level and are based on linear relations that ignore rather than recognize complexity.

Is the tool to blame? You might think this is a funny question for an evaluator to ask. After all, don’t we use logframes as the basis for our assessments? Yet, as people struggle to put together meaningful results frameworks, the question is inescapable.

Arguments that the logframe is too limiting, that it doesn’t take into account other factors, or cater to the complexity of situations are true. But only in part.

The tool actually requires planners to clarify their assumptions and assess risks. In other words: think about a networked and chaotic reality and choose a more linear set of goals, objectives, and outputs. Without it, one is left trying to develop from first principles what might be the appropriate systems for adequate planning, learning and evaluation under complex circumstances.

For us at the World Bank Group the challenge is two-fold:

- We understand the world is complex. The new model – the Solutions Bank Group – has been conceived precisely to correspond to this reality and aims to bring about transformational change in how we work. - To support these changes, we need practical measures to demonstrate – and evaluate independently – whether multi-dimensional development solutions are working, what changes they bring about, and how problems are fixed as they arise.

So how do we get there and what are the risks? Three stand out:

Oversimplification. Past experience is riddled with examples of results that are simply outputs. Take road construction. The simplest measure is the distance that has been built. But, how will this tell us what the road will achieve? In an earlier role I have evaluated road projects. Some resulted in transformational change, economic empowerment and in reduction roadside robberies. Others ended up as the roads less traveled with no economic or social value. So what then is a measure that is simple enough to add up and yet meaningful enough to tell us about results?

Over-Abstraction. Wouldn’t it be great if we had a simple index that tells us whether things are improving or not? It’s a very seductive thought - a number that indicates how well or how badly things are going. But, will the new construct again revert to simplifying models – the ones that Ben points to as the crux of the development matter – in order to make it possible to capture in a number what is a complex process?

Undefined. So, if an iterative learning process is more appropriate in this age of complexity, should we not simply leave our targets undefined and figure things out as we go along? If so, how would we manage the risk, aptly discussed in Ben’s book, of errors that might creep in because we are unaware of our assumptions, have a tendency to simplify models, and to repeatedly follow the same path? How will we know if we are wasting valuable time, effort, and resources instead of investing in them effectively?

During the next week’s Spring Meetings, we will be sponsoring a panel of eminent thinkers and posing this challenge to them so that we can take a practical approach to the new science of delivery. I urge you to make your voice heard.

Add comment

Plain text

  • Allowed HTML tags: <p> <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Latest Blogs

What is Safe Monitoring and Evaluation Practice in the 21st Century?

Monitoring and evaluation practice can do harm. It can harm:

the environment by prioritizing economic gain over species that have no voice
people who are invisible to us when we are in a position of power
by asking for...

The GEF and Cleantech: Evaluating Efforts to Build an Innovation Ecosystem

Global factors such as technological advances, lowered costs, available capital, consumer demand and climate change have been encouraging the development and deployment of clean technologies as part of low...

Recent FAO evaluation highlights the role of GEF in programmes addressing integrated natural resource management for sustainable food and agriculture systems

In November 2018, the Food and Agriculture Organization of the United Nations (FAO) has released an evaluation of its contributions to integrated natural resource management for sustainable agriculture.

CIF Evaluation & Learning: What Are We Learning About the Big Issues in Climate Finance?

The scale and urgency of the climate crisis demands rapid action to prevent a range of consequences – from increased droughts and floods to huge economic losses and more extreme natural disasters.  

All major climate funds and...

Rethinking resilience, adaptation and transformation in a time of change

“How the world deals with large and serious problems is almost certainly going to shape our collective future” is the second line – and a promising start – in this edited Springer volume titled...