Impact research

From snowball to avalanche

Monitoring and evaluation matter in national politics, but play an even greater role in bilateral and international cooperation. It is therefore astounding how little attention is paid to choosing reliable indicators for impact measurement in the planning stages of interventions. A capacity-building project run by InWEnt in the field of information technology (IT) illustrates that it makes sense to examine what works – and how as many people as possible can be reached.


[ By Balthas Seibold, Winfried Kalhöfer and Markus Wauschkuhn ]

The most prominent example of a problematic selection of indicators is surely the Millennium Development Goals (MDGs). Very few countries have reliable data for the base year 1990, against which the desired effects are supposed to be measured. Monitoring data tend to be insufficient as well. Unfortunately, it is a similar story with many national Poverty Reduction Strategy Papers (PRSPs), on the base of which donors make their funding decisions.

The selection of meaningful indicators is crucial for any M&E system. It is very difficult to correct an inappropriate selection once a programme is underway. InWEnt promotes data collection in accordance with – and on the basis of – existing statistical systems and not to create additional, often donor-driven parallel systems.

It makes sense to assess various details in M&E. How does a single IT trainer succeed in teaching 200 members of a cooperative to handle a computer? The key question in every capacity-building project with a view to dissemination and “training the trainer” is: How can training materials be distributed widely and sustainably? How does a snowball become an avalanche?

“it@coops”

The InWEnt programme “Information Technology for Asian Cooperatives” – it@coops for short – is an example of how this can be done. The programme supported small Southeast Asian cooperatives in training and imparting IT know-how, improving local IT centres and forming national and regional networks. InWEnt and the Asian Women in Co-operative Development Forum (AWCF) conducted an analysis of the programme in retrospect, and discovered snowballing effects:
– The newly created pool of 70 IT instructors trained more than 5,000 members of cooperatives within a single year. Additional qualified instructors were appointed.
– Local IT business development centres in Indonesia, the Philippines and Thailand reached more than 21,000 members of the target group.
– Small and micro businesses belonging to cooperatives in rural areas gained access to the Internet and were connected to important IT services.
– Almost two thirds of the new users of information technologies were women. The newly acquired skills improved their prospects, boosted their self-esteem as well as their productivity, and gave them access to sources of knowledge.

In short, the project fulfilled the brief given by the German Development Ministry (BMZ), which had commissioned it@coops. To what extent has impact monitoring, which was conducted throughout the programme, contributed to success? In physics, the rule applies that you cannot measure the position of a particle without influencing it. In the natural sciences, this effect is often considered to be troublesome, but to programmes such as it@coops, it is most useful. In this case, local partners identified with the objectives because they were involved from the outset in working out and developing quantitative and qualitative indicators.

This approach motivated the people involved and raised their awareness. They carried out their own survey to find out how many women benefited from the computer training course run by newly qualified trainers. This kind of dynamism, of course, is only possible if indicators and benchmarks are worked out participatively and can be adjusted as the programme progresses.

Innovation thanks to IT

Information technologies open up new possibilities for impact measurement. Training material can be placed under an “infectious” open source licence, for example, which is what was done in this case. This means that the material, including any improvements, remains publicly accessible. In principle, it can also be used by any number of people. In the long run, the actual coverage is kept track of by means of an identity number – similar to the way “digital watermarking” works with music and videos.

it@coops was set up as a brand, proving that the approach works. National congresses have already been held under this name, and a Google search using this term brings up over 3000 websites. The snowball has launched an avalanche.

Of course, it is important to constantly check what works and what doesn’t. There are several obvious factors which contributed to the success of it@coops:
– It is important that the participants are selected according to criteria relevant to development.
– The people trained to train others should be members of the respective organisations.
– Emotional ties through national and regional networking are useful.
– The involvement of partners in the field is fundamental – and financial contributions are welcome.
– The training courses have to be tailored precisely to the needs, which requires local expertise.
– The focus of the cooperatives – especially of the AWCF – on the interests of poor women was helpful.

So far, however, the contextual conditions for success are not very well understood. Therefore, the entire programme will be examined by external consultants too. It is already clear, however, that any similar measure in future should pay even more attention to the conditions under which participants work day-to-day and what role their superiors play.

The insight that a successful programme cannot simply be repeated is commonplace. Experience shows, however, that it is worthwhile to set clear objectives and to observe closely how an initiative progresses.