What we need to know about the body of evidence on poverty reduction

Donor governments, international NGOs and agencies have witnessed an increasing flurry of activity in departments that are dealing with programme delivery in recent years. The search is on for finding the most effective programmes that will reduce poverty within a generation (with the World Bank and the IMF setting out an ambitious objective of reducing those who live in extreme poverty to 3 percent by 2030). In some departments this search has been translated into searching for the greatest value for money (VfM) programme in a domestic and organisational environment that was asking for more accountability for domestic tax-payers. In other organisations, the focus has been on the need to be more effective in achieving the moral imperative of reducing poverty by using political networks and social movements more effectively (and these matters were raised by Duncan Green in his blog From Power to Poverty since the start of 2013).

This year has also seen policy exchanges springing up in many development organisations, including DFID, ODI, Oxfam, and World Bank on the best way forward to be more effective in working with aid recipients to ensure poverty reduction. These discussions were brought together in a powerful public debate driven by The Big Push Forward initiative that pitted the practical demand for improving the methods for measuring how international aid is bringing about poverty reduction against the politics of measurement whereby international organisations impose technical metrics of aid recipients more akin to a strait-jacket and which disempower local organisations.

A recent workshop, Institutional Change for Poverty Reduction: Results and Evidence of Impact, co-hosted by the World Bank Institute and the Centre of Development Studies and held at the University of Cambridge, focussed on identifying facts behind successful interventions and moving towards a better understanding of how existing evidence might be used to accelerate poverty reduction. The presenters and participants explored current methods for evaluating poverty reduction programmes and the first questions that emerged from the three days of deliberation were the following: Is there a coherent body of evidence on the link between institutional change and poverty reduction?; and How helpful is existing evidence in helping practitioners and policy makers in designing and implementing development initiatives in complex political, social and cultural contexts?

Newton

Did Newton get it wrong?

The question is an apt and important curtain raiser for a public discussion of whether it is indeed correct to embrace evidence based policy (EBP) framework as best practice to identify institutional change that is most effective to ensure poverty reduction. The protagonists of EBP use every opportunity to point out the improved rigour of this approach, but do not adequately remind us of the limitations of this approach, or the very large cost of methods such as Randomised Controlled Trials (RCTs) and the pre-requirements such as a baseline survey (and large additional financial cost and administrative demand on the aid recipient organisation) as feature highlighted by Professor Armando Barrientos at the workshop, and also in his recent WIDER paper.
The EBP and its specific methods such as RCT and quasi-experimental methods have now become common place in policy evaluation manuals and a necessary requirement in international calls for evaluation consultants and international research grant bids. This makes it crucial that the full set of technical and financial demands imposed by these new methods, originally devised in the field of public medicine for the purpose of testing of new drugs, are set out clearly and understandable in the public domain. The EBP framework derived from the evidence based medicine (EBM) pyramid developed by Dartmouth College and Yale University is a powerful framework for medical service delivery.

pyramid 2

The complexity of using EBP in contexts of large social heterogeneity (e.g., asymmetry of power and hierarchy between international aid donors and recipient organisations highlighted by the Big Push conference) requires us to have more evidence of whether these methods have been significantly successful in key fields of poverty reduction such as slum upgrading and provision of water and sanitation. The recent WBI-CDS workshop reviewed several evaluations of poverty reduction programs (including RCTs and other impact evaluations) in these fields. The review highlighted important information gaps that affect the capacity of these exercises to guide replication and scaling-up of institutional change efforts.
This need for evaluating EBM methods as a basis for policy intervention has also been acknowledged by the medical profession in the UK, with the NHS reviewing the value of these methods (recent paper). A public debate on these matters will help to identify how EBP and other methods can be directed to help advance institutional change for poverty reduction.

Shailaja Fennell
Centre of Development Studies
University of Cambridge

Advertisements

6 thoughts on “What we need to know about the body of evidence on poverty reduction

  1. Thanks for sharing this. A new direction indeed in poverty reduction. I wonder if there is reliable statistics available from lesser developed economies to kick start the EBP strategy?

  2. Interesting article. As Sailaja has mentioned aptly, the heterogeneous social factors in countries like, which are still largely feudal, would demand the evolution of an appropriate measure for poverty and its alleviation.

  3. Hi Shalaja, your post and workshop have interestingly coincided with me moving to Jakarta and start working on a long term project funded by AusAID, called Knowledge Sector Initiative. In a nutshell, this 15 years initiative aims at strengthening the use of research evidence in policy making by working on four fronts: research organizations, government organizations, intermediary organizations (e.g. NGOs), the legislation about the use of research in policy decision making. I am responsible for the government organizations part of the project.
    We are about two months into the project and we have to think from the beginning about what our starting point is and how do we know whether we will be making progress. At the same time we also have to demonstrate value for money.
    While reading your post I was thinking about my current work, indicators, measurements, etc. and I associated all this with the word ‘qualitative’. Changes in policies, changes in ways of doing things, changes on how policies are implemented, changes in knowledge, changes in general are often linked to changes in people. With this I mean changes in attitudes, perceptions and more importantly in behaviour. These changes can be documented using qualitative indicators, indicators that tell the story of how the changes have occurred on not just whether they have occurred.
    The start of an initiative, like the one I am involved, requires the establishment of a ‘library’ (this is the word that comes to my mind), which compiles a description of processes, systems, incentives around policy making and the use of evidence. This can be complemented by maps of power relationships, social maps of existing networks. During the implementation this library, which ultimately is the main source of knowledge and learning out of project, will be filled with more stories, data, information, ideas all of which can be compared at regular intervals with the initial maps and the initial stories. That will form a diagnostic of what has worked and what has not worked during the initiative and why. We will of course also need and will collect more quantitative data, but they are only a part of the story.
    The architect Renzo Piano has been interviewed after the completion of the Shard in London last year (a controversial project) and said something that I think fits well with the discussion we are having: ‘In every interesting job, you are there not to change the world, but to witness the change in the world. You do not get hostility because you are wrong, but because people have a fear of change. Around any job, you can always build a good story and a bad story and the hostility goes away – because if you are right, then the good story comes out.’

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s