Scoping Study of Evaluations of NRM Interventions Linked to Climate Change

Building on its successes and increasing membership, Climate-Eval has broadened its scope to take on a wider set of issues and development interventions around evaluation in the field of Natural Resource Management (NRM). This intent was first communicated to the online community in a blog last year as well as a news item earlier this year.

In light of this expanded focus, a scoping study has now been commissioned on evaluations of NRM interventions with linkages to climate change. The purpose of the study is to map out the landscape of evaluations and literature on this topic and highlight the gaps in knowledge, challenges, innovative methodologies and avenues for further research. More detailed information on this study can be found in the approach paper.

The study has just passed through its initial stages and is moving along to the more substantial, analytical parts. The desk review of relevant literature and evaluations from databases and libraries has been completed, and the process is outlined below. As a reference point, Arksey & O’ Malleys’ methodological framework for scoping studies was used to guide this process:

  1. Identify research question
  2. Identify relevant studies
  3. Study selection
  4. Charting the data
  5. Collating, summarizing and reporting the results (Arksey & O’Malley, 2005)

Literature Review: Rather than having a highly focused research question at the outset, scoping studies aim to identify all relevant literature related to a topic regardless of study design (Arksey&O’Malley, 2005). Although this study aimed to be as comprehensive as possible in the coverage of available literature, some limits had to be applied to the extent of the search due to time constraints. As such, the literature review was limited to all evaluation journals available through the World Bank Group’s online library network, and also included other articles which had been cited in articles found in the journal search (“snowballing”). Peer-reviewed journal articles were collected from these journals using the following Boolean searches (published between 1990 – 2014):

“Natural Resource Management”; “NRM”; “Sustainable Management of Natural Resources”; “Sustainable Forest Management”; “Protected Areas”; “Soil and Water Conservation”; “Conservation”; “Payments for Ecosystem Services” ; “Land Rehabilitation”; ”Watershed Management”; “Sustainable Land Management”; “Livelihoods”; “Food Security”; “Biodiversity Conservation”; “Ecosystems Management” AND “Climate Change Mitigation”; ”Mitigation”; ”Green House Gases”; ”GHG”; ”Carbon Sequestration”; ”Carbon Markets”; ”Soil Carbon”; ”Landscape Change”; ”Biomass”; ”Carbon Sink”; ”REDD+”; ”Emissions”; “LULUCF”; “Land Use Change”; “Land-cover”; OR “Climate Change Adaptation”; “Adaptation”; ”Community-based Adaptation”; ”Resilience”; ”Vulnerability”.

Evaluation databases: Other than the literature review, evaluation databases of key organizations involved in evaluation of climate change and development/NRM were searched. These include first and foremost the Climate-Eval library, followed by the GEF IEO’s Terminal Evaluation database, the World Bank IEG’s evaluations, UNDP’s Evaluation Resource Center, ADB Independent Evaluation Resources, and USAID’s Development Experience Clearinghouse. To simplify the searches, only the keywords ‘Natural Resource Management’ and ‘NRM’ were used.

Reports, guidelines and other frameworks: To complement the desk review, some relevant guidelines, toolboxes, and frameworks were also added as possible additional sources. These were collected through a simple google search for ‘Natural Resource Management guidelines/frameworks/ toolboxes’ or different combinations of the same phrase.

Results: The completed desk review came up with over 115 evaluations, 135 journal articles, and 105 other documents (guides, reports/reviews, frameworks, and tools, etc.). To further refine the group of evaluations collected, a monitoring table was created to rank the documents in terms of number of keywords that they contained, and whether or not they discussed methodology, logical frameworks, sustainability, limitations, etc. The keywords that were searched within each document were: “climate change”, “adaptation”, “mitigation”, “carbon”, “emission(s)”, and “resilience”. The evaluations which did not have explicit linkages to climate change were removed from the study.

The next step in the study involves a thorough analysis of the shortlisted documents to uncover trends in knowledge and practice gaps. The evaluations reviewed to date have in most cases been summative evaluations carried out for accountability purposes, which are required as a condition of donor funding. A small number of impact evaluations and systematic or rigorous literature reviews have also been identified, which probe in greater detail the evidence base for various conclusions concerning NRM interventions. The literature review has revealed an extensive and lively debate concerning appropriate methodologies, data quality, internal and external validity, and causal inferences.

Stay tuned for our next blog update on this study in a couple of weeks!

Questions or comments about this study? Comment below or e-mail us at: Climate-Eval@Climate-Eval.org

References:

Arksey,H & O’Malley, L (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology. Vol 8, Pp 19-32.

Add comment

Plain text

  • Allowed HTML tags: <p> <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Latest Blogs

What is Safe Monitoring and Evaluation Practice in the 21st Century?

Monitoring and evaluation practice can do harm. It can harm:

the environment by prioritizing economic gain over species that have no voice
people who are invisible to us when we are in a position of power
by asking for...

The GEF and Cleantech: Evaluating Efforts to Build an Innovation Ecosystem

Global factors such as technological advances, lowered costs, available capital, consumer demand and climate change have been encouraging the development and deployment of clean technologies as part of low...

Recent FAO evaluation highlights the role of GEF in programmes addressing integrated natural resource management for sustainable food and agriculture systems

In November 2018, the Food and Agriculture Organization of the United Nations (FAO) has released an evaluation of its contributions to integrated natural resource management for sustainable agriculture.

CIF Evaluation & Learning: What Are We Learning About the Big Issues in Climate Finance?

The scale and urgency of the climate crisis demands rapid action to prevent a range of consequences – from increased droughts and floods to huge economic losses and more extreme natural disasters.  

All major climate funds and...

Rethinking resilience, adaptation and transformation in a time of change

“How the world deals with large and serious problems is almost certainly going to shape our collective future” is the second line – and a promising start – in this edited Springer volume titled...