Biases in impact evaluation

You’ve finished writing your evaluation report containing a neat LogFrame and objective and verifiable indicators. Rating: highly satisfactory. That’s good, but is it true for the development intervention as a whole?

In a guest post at the World Bank’s Development Impact blog, Martin Ravallion criticizes development evaluation that assesses projects in isolation of the entire development portfolio. He cites common pitfalls in conducting evaluation, such as the assumption of a negligible interaction effects among project components and a tendency for selection bias. But these biases are sometimes inherent to the programs and policies in consideration. Difficulty arises when taking a sample of roads, dams, and other big-ticket projects, or even multifaceted policy reforms.

If we are serious about assessing "develoment impact" then we will have to be more interventionist about what gets evaluated and more pragmatic and eclectic in how that is done.

Ravallion recommends the central coordination of what gets evaluated and thinking of creative ways to evaluate portfolios as a whole that considers interaction effects. You can read the essay here.

Does your evaluation workflow for climate mitigation and adaptation projects encounter and consider these biases? How do you deal with them, if at all?

Source: Development Impact via Smart Aid

Image credit: Smart Toolkit

Add comment

Plain text

  • Allowed HTML tags: <p> <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <p> <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd> <img> <div> <span>
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Latest Blogs

Evaluation and Adaptation Pathways

Policy analysts have a new addition to their toolbox that recognizes the importance of deep uncertainty and the resulting need for adaptive planning approaches. The use of adaptation pathways, as this new addition is known, supports the...

Professionalization With a View to Eval2030

Are we ready, will we ever be? The conversation about professionalization in evaluation has a long history, and often raises much concern and interest. But come to think of it: many other professions took hundreds of years to shape their...

Chance to Make Infrastructure Climate-Friendly

As multilateral development banks, private sources and governments gear up to fill massive infrastructure gaps in Asia, we must not miss a unique opportunity.

We Are All Knowledge Brokers, You Know

We’re all knowledge brokers. As social beings we naturally pass around information, filtering it and combining it with other bits of information that we think are relevant or interesting, then communicating that to others.

What Would You Do with 200 Million US Dollars?

Towards the end of a recent meeting Dr. Melissa Kenney asked me; “What would you do if you had 200 million US Dollars to improve the monitoring and evaluation of climate change adaptation and sustainable development?”