Involving independent experts

Monitoring plays a major role in development cooperation, especially for designing policy, programmes and financing interventions. It is an important tool for systematically verifying whether specific projects actually have the desired effect. In the light of the debate on Aid Effectiveness, this has become more important than ever before. One example is poverty reduction strategies.

Monitoring in this area mainly serves the purpose of identifying whether a country’s policy in a specific sector is in line with the goals, strategies and indicators stated in its national poverty reduction strategy – and whether it actually results in declining poverty. Monitoring can be politically problematical. Cambodia, for example, is currently witnessing a heated debate on who should write the interim report on the national strategy, for whom this is to be done, and what terms of reference should apply.

The government regards that kind of report as a government document, and thus wants it to be rather upbeat. Experts, however, see it as an exercise in social sciences, which should strive for objectivity. Donors, in turn, want a comprehensive but concise review that identifies all problems relating to poverty reduction, and presents viable solutions – which is simply impossible as donors among themselves do not always agree on problems and on how to approach them.

Relevant indicators

From a methodological perspective, the difficulty lies in the fact that while lots of indicators are required, there is also a risk of generating too much data that provides too little information. A good monitoring system needs to reduce the description of complex, multi-layered phenomena to a small number of relevant indicator in order to provide a solid base for delivering core messages and to effectively gear sector policies to reducing poverty.

There are marked differences in the monitoring of poverty-reduction strategies from one country to another. In Cambodia, for instance, 43 indicators are taken into account; in Tanzania, the total is 68. Data monitoring on this scale makes sense – even the Millennium Development Goals are based on 60 indicators. However, Keith Mackay of the World Bank’s Independent Evaluation Group points to one operational model in Colombia with a 600 indicator base. However, a monitoring system of that magnitude that actually works is the exception, not the rule.

What is crucially important, however, is that the relevant data be actually collected, updated at sufficiently frequent intervals, and differentiated by gender. Targeted pro-poor policies and programmes can best be drafted and implemented if the data are also prepared separately for provinces or even municipalities (poverty mapping).

Rather than to keep on increasing and refining the indicators used, it makes more sense to incorporate more instruments into the monitoring system. Qualitative methods and participatory poverty research are key options.

Monitoring of public expenditure is another important approach. It serves to ensure that the funds required to achieve certain objectives are indeed made available. Relevant instruments include public expenditure reviews (PER), public expenditure tracking surveys (PETS), service delivery surveys (SDS) as well as benefit incidence analysis (BIA). These tools show whether the desired effects have been achieved – for example, whether services have actually been delivered, or who benefited from them.

The results of monitoring are processed in what is known as the policy cycle, which is made up of the following stages:
1. situation analysis,
2. agreement on targets and indicators,
3. policy and strategy development,
4. cost estimation,
5. securing funds through the budget, which ideally includes programme budgets and ODA funds,
6. monitoring and evaluation,
7. assessment of necessary changes and finally
8. confirmation of results or reformulation of goals and indicators, policies and strategies.

Getting the best possible picture of progress or lack of progress is one thing; taking responsibility for the results – which are not always what was hoped for – is another. This is by far the greater challenge, for industrial and developing countries alike.

Because the monitoring results of poverty reduction strategies are of general as well as public interest, governments are loathe to be associated with unpopular findings – especially in times of rising food prices which tend to wipe out progress that was laboriously achieved. In consequence, monitoring is often not done seriously in the first place – or the results are not published. Either way, the scope for improving policies is cancelled out. That said, the question arises whether whether poverty-reduction monitoring should really be the sole responsibility of any given government.

A sensible approach

Such “self- or auto-evaluation” could be usefully supplemented by independent monitoring by third parties. In Latin America, this already seems to be commonplace.

One form it could take is as follows: The responsible government department – in Cambodia it is the Ministy of Planning, but in many other countries those of finance or economy – first drafts a “Green Book” and thus sets a public debate in motion. On the basis of this document – and taking account of public opinion – the cabinet then sets out its official position in a “White Book”.

This procedure would have the advantages of enabling all concerned to present their views and triggering a genuine debate on content. What is more, everyone could learn from the different assessments, avoiding the risk of one-sided perception.

The Green Book-approach would also be important from another viewpoint. Accountability plays a far more important role in poverty-reduction monitoring than in monitoring operations where project performance is of interest only to the commissioning body. After all, the success of poverty reduction strategies matters to government and opposition, parliament, civil society and the entire national and international public – not to mention donors.

Involving independent expertise, for instance that of research institutes, would have another invaluable advantage: governments themselves could focus more intently on their real responsibilities of drafting and implementing policies.