The Challenge of Evaluation

khanalrc's picture

Ram Chandra Khanal discusses the challenges in the evaluation of climate change based on the climate change adaptation session of the Evaluation Conclave 2013 held on February 26 to March 1 in Kathmandu, Nepal. The article was originally published in the features section of the Climate & Development Knowledge Network.

Evaluation of climate change interventions in South Asia was one of the themes discussed at the recent South Asia Evaluation Conclave held in Kathmandu, Nepal.

Evaluation is a critical part of development interventions – for accountability, for learning, to see what worked and what didn't, for influence and so on. Evaluation is particularly critical in the context of South Asia, home to complex social structures, high rates of poverty, gender discrimination, dynamic forces of globalization sweeping traditional societies and numerous development projects for the large populations of this region. Innovative evaluation approaches and practices are particularly important in such complex contexts.

The aim of the conclave was to bring together thinkers, commissioners and practitioners of evaluation in an interactive forum to discuss, deliberate and share advancements in the theory and practice of evaluation in the South Asia region and abroad.

Organized by the Community of Evaluators (CoE) in collaboration with the Community of Evaluators Nepal (CoE-Nepal), this was the second such conclave to be held in South Asia – the first was in October 2010 in Delhi. The CoE is a group of evaluators from South Asia (India, Sri Lanka, Bangladesh, Afghanistan, Nepal and Pakistan), working together to strengthen the field of evaluation. This second conclave from 26 February to 1 March was attended by over 300 experts and evaluators from different parts of the world including representatives of the World Bank, Asian Development Bank, IDRC and universities. Robert Chambers, the 'guru' of participatory evaluation, and Michael Quinn Patton gave keynote speeches in the workshop.

Climate change adaptation strategies contain a wide variety of interventions with different scale considerations, processes and types of activities, targeting different sectors and levels. Some focus on building adaptive capacity, others on direct interventions such as development of physical infrastructure; many will include a mix of these elements. Evaluation of climate change related interventions thus poses particular challenges. It can be difficult to distinguish the effects of climate change adaptation strategies from those of other sectoral activities, and thus to attribute outcomes or impact to a single intervention. How then to evaluate and measure impacts of climate change adaptation interventions?

This was the question I attempted to answer in the session on climate change and M&E, organized by SEA Change CoP - an Asian community of practice on the monitoring and evaluation of climate change interventions, with financial support provided by the Rockefeller Foundations and technical and logistical support from PACT. I began by looking at evaluation institutions and processes related to climate change adaptation in Nepal. My conclusion was that there are good mechanisms instituted, but the focus is more on monitoring than evaluation. Moreover evaluation activities by multiple players and at different levels are not well-connected and harmonized in practice.

In the second part of my presentation I used CDKN and the Pilot Programme for Climate Resilience (PPCR) as case studies of evaluation systems for CCA projects. Examples of indicators used by CDKN and PPCR include:

Numbers of people supported by the project to cope with effects of climate change;Degree of integration of climate change in national, including sector, planning;Extent to which vulnerable households, communities, businesses and public sector services use improved (the project) supported tools, instruments, strategies, activities to respond to climate vulnerability and climate change;Evidence of strengthened government capacity and coordination mechanism to mainstream climate resilience;Negotiation skill (on climate change) enhanced in international forums by using knowledge and recommendations generated through the project.

The CDKN and PPCR case studies highlighted the challenges and complexities in evaluating CCA related projects. How to distinguish between development and adaptation? How to take into account the fact that people often respond to CC in autonomous ways? How to measure elements such as adaptive capacity, transformative change and resilience? There has been a move towards result-based and participatory M&E activities, but efforts are fragmented and not sufficient to address current challenges. The CDKN approach of focusing on practical aspects and flexibility were well appreciated and so was that of PPCR.

The main conclusion from the session was that there is no fixed model: evaluation needs to be adaptive, based on the context and project needs. There was also general agreement on the fact that additional precautions (such as focusing on a programmatic approach and building capacity of stakeholders) need to be taken while developing evaluation systems for CCA projects. The complementarities of adaptation vs. development have to be explored. Continuous assessment and sharing learning among stakeholders are also important. Perhaps the main conclusion was that evaluation of CC adaptation projects is evolving and further research is needed.

Ram Chandra Khanal is CDKN's Engagement leader in Nepal. He is also the former Natural Resources Management Specialist / National consultant within the Ministry of Environment for the Strategic Program for Climate Resilience (SPCR) in Nepal, and can be contacted at khanalrc@gmail.com

Add comment

Plain text

  • Allowed HTML tags: <p> <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Latest Blogs

What is Safe Monitoring and Evaluation Practice in the 21st Century?

Monitoring and evaluation practice can do harm. It can harm:

the environment by prioritizing economic gain over species that have no voice
people who are invisible to us when we are in a position of power
by asking for...

The GEF and Cleantech: Evaluating Efforts to Build an Innovation Ecosystem

Global factors such as technological advances, lowered costs, available capital, consumer demand and climate change have been encouraging the development and deployment of clean technologies as part of low...

Recent FAO evaluation highlights the role of GEF in programmes addressing integrated natural resource management for sustainable food and agriculture systems

In November 2018, the Food and Agriculture Organization of the United Nations (FAO) has released an evaluation of its contributions to integrated natural resource management for sustainable agriculture.

CIF Evaluation & Learning: What Are We Learning About the Big Issues in Climate Finance?

The scale and urgency of the climate crisis demands rapid action to prevent a range of consequences – from increased droughts and floods to huge economic losses and more extreme natural disasters.  

All major climate funds and...

Rethinking resilience, adaptation and transformation in a time of change

“How the world deals with large and serious problems is almost certainly going to shape our collective future” is the second line – and a promising start – in this edited Springer volume titled...