Looking beyond project outputs in climate mitigation evaluation

Kelly Hewitt delivered the following message for the Climate Change M&E Panel of the Evaluation Conclave 2013 in Kathmandu, Nepal:

Please, let me start by saying that though I and my colleague Kapil Thukral, serve on the Evaluation Oversight Committee for the current CIFs global evaluation, it is my separate, independent and individual professional observations as an evaluator that guide my comments this morning. So with that let's move on.

I believe that we all can agree that carbon investment funds aim to bring about additionality, leveraging, and transformation. In the evaluator's world we're speaking to core sub-components of efficiency and sustainability. I had originally planned to talk about how complex it is to assess additionality, leverage, and transformation because of inconsistent definitions and because any assessment involves measuring for the most part capacity building results and real private sector market transformation. Instead, in an attempt to dissolve some of the complexity and to provide some focus to the concept of transformation - I will speak solely to that term.

Most of what we talk about when we discuss additionality, leverage, and transformation in the evaluation context is quite different from the typical private sector definitions of the same terms.

Now it's important to realize that when I speak of private sector definitions, I am not referring to the private sector arm or arms of the five multilateral development banks that design, approve, and implement CIF funded projects. When I use the term private sector, I'm speaking of real markets - supply - product and service quantities, pricing, demand; market entry/access, competition (both firm and product), market penetration, market power - all of which go into real sustainable market transformation.

And despite these real variables, there measurement, and the need to include them in evaluation assessments of publicly funded climate investments, today's evaluation approach of climate investments appears to mention very little if anything at all with regard to the importance of knowing, understanding, and measuring these particular real market variables.

Instead, evaluation has focused on output measurement - for example - How effective have public finance mechanisms to scale up private sector investment in climate solutions been - i.e. number of private sector investments in renewables or energy efficiency before and after an intervention or - another example - number and type of policies produced to incentivize renewable or clean energy investments and/or to create an enabling environment for private investment. And, often, the evaluation stops there.

We can't stop there. Back in the early 90's when we were having the same discussions regarding M&E of energy efficiency initiatives and demand-side management in the US and Europe - there were assumptions made, conclusions drawn regarding outcomes - project contributions to energy use reduction - energy demand - assumptions of outcomes that could not or either were not verified.

Today - we cannot assume that because a project was designed to reduce X amount GHGs annually that because the project output materialized that, indeed, the reduction occurred.

Equally, we cannot assume that because a project was designed to result in market transformation that because the project output of de facto conducive policies and/or private investments materialized - that, indeed, market transformation has occurred.

To measure real transformation we must look at and quantify, among other things, real market variables that provide necessary information on - product and service supply, pricing, competition (both firm and product), market penetration, market power- all of which goes into real sustainable market transformation.

With that I'll stop here hoping that in our interactive discussion we can chat about real word experience with trying to quantify these variables, along with limitations in evaluation scope and application of lessons learned. Thank you.

Ms. Kelly Hewitt, Evaluation Specialist with the Independent Evaluation Department of the Asian Development Bank, the Philippines. Commencing work in the energy sector in 1990, among other things, she was responsible for carrying through design, market and competitive analyses of demand-side efficiency energy sector products and services for private sector energy companies and for public sector regulators. She currently focuses on independent country energy sector assessments and special evaluation studies - including renewable energy intervention assessments. BA -Public Policy; MA - Economics, Jurist Doctorate, MPA - Harvard Kennedy School

Add comment

Plain text

  • Allowed HTML tags: <p> <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Latest Blogs

What is Safe Monitoring and Evaluation Practice in the 21st Century?

Monitoring and evaluation practice can do harm. It can harm:

the environment by prioritizing economic gain over species that have no voice
people who are invisible to us when we are in a position of power
by asking for...

The GEF and Cleantech: Evaluating Efforts to Build an Innovation Ecosystem

Global factors such as technological advances, lowered costs, available capital, consumer demand and climate change have been encouraging the development and deployment of clean technologies as part of low...

Recent FAO evaluation highlights the role of GEF in programmes addressing integrated natural resource management for sustainable food and agriculture systems

In November 2018, the Food and Agriculture Organization of the United Nations (FAO) has released an evaluation of its contributions to integrated natural resource management for sustainable agriculture.

CIF Evaluation & Learning: What Are We Learning About the Big Issues in Climate Finance?

The scale and urgency of the climate crisis demands rapid action to prevent a range of consequences – from increased droughts and floods to huge economic losses and more extreme natural disasters.  

All major climate funds and...

Rethinking resilience, adaptation and transformation in a time of change

“How the world deals with large and serious problems is almost certainly going to shape our collective future” is the second line – and a promising start – in this edited Springer volume titled...

Looking beyond project outputs in climate mitigation evaluation | Earth-Eval

Error

The website encountered an unexpected error. Please try again later.