Donor governments, international NGOs and agencies have witnessed an increasing flurry of activity in departments that are dealing with programme delivery in recent years. The search is on for finding the most effective programmes that will reduce poverty within a generation (with the World Bank and the IMF setting out an ambitious objective of reducing those who live in extreme poverty to 3 percent by 2030). In some departments this search has been translated into searching for the greatest value for money (VfM) programme in a domestic and organisational environment that was asking for more accountability for domestic tax-payers. In other organisations, the focus has been on the need to be more effective in achieving the moral imperative of reducing poverty by using political networks and social movements more effectively (and these matters were raised by Duncan Green in his blog From Power to Poverty since the start of 2013).
This year has also seen policy exchanges springing up in many development organisations, including DFID, ODI, Oxfam, and World Bank on the best way forward to be more effective in working with aid recipients to ensure poverty reduction. These discussions were brought together in a powerful public debate driven by The Big Push Forward initiative that pitted the practical demand for improving the methods for measuring how international aid is bringing about poverty reduction against the politics of measurement whereby international organisations impose technical metrics of aid recipients more akin to a strait-jacket and which disempower local organisations.
A recent workshop, Institutional Change for Poverty Reduction: Results and Evidence of Impact, co-hosted by the World Bank Institute and the Centre of Development Studies and held at the University of Cambridge, focussed on identifying facts behind successful interventions and moving towards a better understanding of how existing evidence might be used to accelerate poverty reduction. The presenters and participants explored current methods for evaluating poverty reduction programmes and the first questions that emerged from the three days of deliberation were the following: Is there a coherent body of evidence on the link between institutional change and poverty reduction?; and How helpful is existing evidence in helping practitioners and policy makers in designing and implementing development initiatives in complex political, social and cultural contexts?
Did Newton get it wrong?
The question is an apt and important curtain raiser for a public discussion of whether it is indeed correct to embrace evidence based policy (EBP) framework as best practice to identify institutional change that is most effective to ensure poverty reduction. The protagonists of EBP use every opportunity to point out the improved rigour of this approach, but do not adequately remind us of the limitations of this approach, or the very large cost of methods such as Randomised Controlled Trials (RCTs) and the pre-requirements such as a baseline survey (and large additional financial cost and administrative demand on the aid recipient organisation) as feature highlighted by Professor Armando Barrientos at the workshop, and also in his recent WIDER paper.
The EBP and its specific methods such as RCT and quasi-experimental methods have now become common place in policy evaluation manuals and a necessary requirement in international calls for evaluation consultants and international research grant bids. This makes it crucial that the full set of technical and financial demands imposed by these new methods, originally devised in the field of public medicine for the purpose of testing of new drugs, are set out clearly and understandable in the public domain. The EBP framework derived from the evidence based medicine (EBM) pyramid developed by Dartmouth College and Yale University is a powerful framework for medical service delivery.
The complexity of using EBP in contexts of large social heterogeneity (e.g., asymmetry of power and hierarchy between international aid donors and recipient organisations highlighted by the Big Push conference) requires us to have more evidence of whether these methods have been significantly successful in key fields of poverty reduction such as slum upgrading and provision of water and sanitation. The recent WBI-CDS workshop reviewed several evaluations of poverty reduction programs (including RCTs and other impact evaluations) in these fields. The review highlighted important information gaps that affect the capacity of these exercises to guide replication and scaling-up of institutional change efforts.
This need for evaluating EBM methods as a basis for policy intervention has also been acknowledged by the medical profession in the UK, with the NHS reviewing the value of these methods (recent paper). A public debate on these matters will help to identify how EBP and other methods can be directed to help advance institutional change for poverty reduction.
Centre of Development Studies
University of Cambridge