Taking research into account

Many of the targets defined in the Millennium Development Goals (MDGs) will not be met by 2015. Though aid interventions have improved or even saved millions of lives over the last decades, other people are still starving, cases of government corruption still show, and aid practitioners are fighting for their jobs. There is a clear disconnect: over 90 % of published project-impact evaluations are favourable and yet sub-Saharan Africa has only clocked an increase of around 0.07 points on the Human Development Index in the last decade, with 23 of the 24 countries ranked in 2007 as low in human development being in that region.

The task is huge and complex. As Jane Jacobs has noted, development processes involve qualitative changes that “can’t be usefully thought of as a line, or even a collection of open-ended lines. [Development] operates as a web of interdependent co-developments.” Where pulling at one string may unravel the whole system in one context, it may have little effect in another or be highly successful in yet another.

Decisions on aid disbursement are political in nature. Thus the question is not whether or not development cooperation is effective – we know it can be. The Tanzania Essential Health Intervention Project for example helped reduce infant mortality by up to 46 % in two rural districts between 2000 and 2003.

The real question is how development practitioners, researchers, civil society and government officials can influence donors to make decisions and set policies on development cooperation that are informed by research-based evidence and adhere to principles of aid effectiveness.

For the past five years, the Research and Policy in Development (RAPID) programme at the Overseas Development Institute (ODI) in London has examined how better to link research, policy and practice. This analysis has contributed to the development of the RAPID Outcome Mapping Approach (ROMA) to use evidence to inform policy and practice (Young and Mendizabal, 2009).

Following Jacobs’ definition, development is an incremental process that with time gets more complex. This complexity must be understood as a qualitative change that involves – besides economic growth – social as well as cultural change. Knowledge and learning play a central role in informing this qualitative change and illustrate the complex impact of aid on development processes.

Donors spend billions of dollars every year on development research. Not all this research and knowledge is directed to aid flows and aid coordination mechanisms, so the question remains how it influences aid polices and processes. Clemens et al. (2008), argue that development actors seem to suffer from a learning challenge, which is highlighted by the fact that progress has been patchy. So how do we overcome this learning challenge?

RAPID’s work suggests that research-based evidence can improve design, definition and implementation of development polices. A two-step approach is suggested here: First, understand how policy processes work as well as the challenges that research-based evidence faces in influencing and informing those processes. Second, adopt a strategic approach to link research with the policy processes.

Challenges of the policy processes

The research by the RAPID group has highlighted several challenges to making research-based evidence influence policy processes:
– Policy processes are complex and rarely linear. Simply presenting information to decision makers and expecting them to act upon it is very unlikely to work. While many policy processes do involve sequential stages some stages take longer than others, and several may occur more or less simultaneously.
– Many actors are involved in aid decisions: ministers, parliament, civil servants, the private sector, civil society, the media, and all of them are trying to influence the decision-making process and each other.
– Many policy processes are only weakly informed by research-based evidence. Policy-makers have to decide fast, usually work on several themes at the same time, and have to stick to a decision for a reasonable period of time.
– Many policy discussions are held in secret and, as few policy-makers are scientists, they may not appreciate the scientific concept of testing a hypothesis.

Nonetheless, research-based evidence can contribute to policies that have a dramatic impact on lives. For example, the World Bank and the International Monetary Fund launched a new aid strategy in 1999, the Poverty Reduction Strategy Papers (PRSPs).

Three types of evidence influenced the emergence of PRSPs, including academic research, which contributed to a shift in the development discourse towards poverty reduction, and applied policy research undertaken in the late 1990s by, for example, working groups such as the Strategic Partnership with Africa and NGOs on debt relief with a focus on policy recommendations and operational solutions. An extremely powerful demonstration effect was provided by the positive experience of Uganda’s Poverty Eradication Action Plan, which convinced policy-makers and donors of the merits of the poverty reduction strategy model.

Researchers who intend to influence policy need a holistic understanding of the context they are working in. ODI has developed a simple analytical framework identifying four broad groups of factors (Young and Court, 2004).
– External influences are factors outside the specific policy context that affect what happens within. Aid policies can have a huge influence in highly indebted countries and, in general, cultural and social factors might play a large role.
– The political context includes the people, institutions and processes involved in policy-making.
– There are various kinds of research-based evidence in terms of type, quality and contestability of the research. Results are also communicated in different ways, which can matter too.
– Finally, links include all of the other actors and mechanisms that affect how the evidence gets into the policy process.

Crucial skills

Researchers also need to acquire skills to influence policy. They need to understand the politics and identify the key players. They need to be able to derive simple interesting stories from the results of the research. They need to be good networkers and work effectively with all the other stakeholders including policy makers; and they need to be able to build good research projects that pull all of this together.

Moreover, researchers need to really want to influence policy with their research. Therefore turning a researcher into a policy entrepreneur involves a re-orientation towards policy engagement rather than academic achievement and a greater engagement more with the policy and aid community. It also requires researchers to acquire new skills, build multidisciplinary teams and produce a different range of outputs.

Not only are policy processes complex, but they also affect the links between research, policy and practice. Aid policy decisions vary greatly from one country to another, from one sector to another. Planning processes and aid decisions have to also take into account differences in context. The RAPID Outcome Mapping Approach (ROMA) we suggest here is also strongly informed by the principles of Outcome Mapping, developed by the International Development Research Centre of Canada.

Six steps for research

The first step for research is to define a clear policy objective. Influencing objectives need not be limited to changes in aid policies but may also include procedural changes – how something is done, attitudes, perception of key stakeholders and behaviour et cetera. Change may also affect the way something is achieved or approached.

After defining the objective, the second step is to map the policy context around that issue and identify key factors that may influence the policy process (Nash et al, 2006):
– What is the donor’s agenda?
– How do aid policies influence local political context?
– Is there political interest in change in the country?
– How do policy-makers perceive the problem?
– Is there enough of the right sort of evidence to convince others of the need for change? How is it presented?
– Who are the key organisations and individuals with access to policy-makers, are there existing networks to use?

The third step is to identify the key influential stakeholders and distinguish between the ones who are very interested and aligned and can be considered natural allies for change and others who are interested, though not yet aligned and who can yet be brought into the fold of reformers, so they do not present obstacles.

The fourth step is to precisely describe the current behaviour, and the behaviour that is needed if the key actors are to contribute to the desired policy objectives, and short- and medium-term step-changes which can be monitored to ensure that the priority stakeholders are moving in the right direction and responding to the efforts of the programme.

The fifth step is to develop a strategy to achieve those changes. It is about ensuring the engagement team has the competencies required to operationalise the strategy. In other words, it must have the set of skills that can help to inform or involve policy-makers in research. The information gathered thus far should prove useful in the starting tangible action to meet the desired policy objective.

The sixth step is monitoring and tracking progress. All necessary adjustments must be documented and the effectiveness of the approach assessed in order to learn lessons for the future.

The world is changing rapidly in ways that tend to affect poor countries most. The complexity of aid is also increasing. Donors are trying to increase aid effectiveness and achieve greater coordination through agreements such as the Paris Declaration.

The United Nations is piloting in eight countries an initiative to combine the plans, budget, set of guidelines, and leadership of its various agencies in a One UN. As noted by Evans (2008), international development has never been so high up the political agenda and the demand for more scrutiny and accountability never been so high.

In this context it is vital that decision-makers across the world base their decisions on reliable evidence. They need to be able to access the best knowledge and experience available to know what the implications of various policy options are and what works and what doesn’t. In our opinion, greater use of research-based evidence for influencing development policy and practice can make a major contribution to aid effectiveness and accountability.