Assessing results

Complex set of factors

Donor agencies are under pressure to prove their worth. Success cannot be measured in business terms alone. The International Finance Corporation (IFC), the member of the World Bank Group that finances private-sector investments, takes account of development impacts and modifies strategy accordingly.


[ By Liane Asta Lohde ]

Today it is widely accepted that the private sector is a key driver of economic development. Accordingly, private-sector investments of multilateral development banks have more than doubled in the past five years. The IFC is the largest multilateral financier in this field. In the past years, it has expanded operations fast. At a time, when aid effectiveness is internationally debated, the IFC understands that monitoring and evaluation systems are essential.

In 2005, the IFC launched the Development Outcome Tracking System (DOTS), which allows the corporation to monitor, track and report developmental results. Building on an existing evaluation system and widely recognized good practice standards, DOTS provides contemporaneous and continuous performance monitoring. It enables the IFC to report at the project level, but also allows for aggregation by sector, country and region of its entire active investment portfolio.

Up to 2007, IFC’s Annual Report only covered financial results. That is no longer so as the new Annual-Report series elaborates on results in terms of finance, sustainability and development impact. To understand the dimensions of this change one has to know
– what type of information the IFC’s M&E outfit generates,
– how that information is used, and
– to what extent it measures the corporation’s contribution to any given country’s economic development.

One way, albeit an approximate one, of showing how IFC-supported investments contribute to a country’s development is reach indicators. They are about economic and other benefits that accrue to various stakeholders. If, for example, an IFC-supported garment manufacturer expands operations and hires additional workers, the host community obviously benefits. That is also the case where companies pay taxes and royalties or buy inputs locally.

On an aggregate basis, the IFC is now in a position to report on this kind of developmental reach. In 2007, IFC-supported investments provided close to 2 million jobs and generated $ 17 billion worth of taxes and other government revenue. On top of that, local purchases of goods and services were triggered to the tune of $ 54 billion. Last year, the IFC also began to collect data on gender indicators, for instance, the number of jobs held by women and the proportion of wages earned by them.

Another dimension of assessing IFC's development impact is to monitor each investment’s progress towards specific outcomes that are defined at the outset. They concern economic, financial, environmental and social results as well as broader private- sector effects. Performance is assessed according to market-based benchmarks for financial results whereas, in social and environmental terms, specific IFC Performance Standards are used. As every single, active investment operation is assessed, it is possible to arrive at an IFC-wide aggregate measure. In 2008, 71 % of IFC projects were considered to have generated high development results.

Learning from experience

Obviously, any meaningful M&E system must generate information with a bearing on ongoing operations, business strategy or the understanding of what makes development happen in general.

Every year, IFC systematically analyses the development results of its activities and feeds this information into its strategy. Thanks to standardised results-tracking across projects, DOTS allows for sectoral, regional and country-level analysis.

Such data indeed inform IFC strategy. In recent years, for example, general manufacturing investments in Africa only led to moderate developmental results. Closer scrutiny of the data revealed that particularly investments in small companies were the reason for this modest performance. As they are inherently susceptible to financial failure these clients needed particularly close supervision and support. In order to improve the development outcome of these projects, closer attention has been paid to such investments since.

Another example of how analysis of results can influence IFC policy lies in the sector of micro, small and medium enterprises (MSMEs). It accounts for a significant proportion of the businesses operational in developing countries, and they typically employ more than half of the working population.

Inititally, the IFC invested directly in MSMEs, but the results were not satisfying. The IFC therefore switched to supporting these firms through financial intermediaries with local knowledge of both markets and clients. IFC’s cumulative microfinance portfolio committed between 2000 and 2008 is $ 942 million. On top of that, the IFC started a new Africa MSME Finance Programme, combining advisory and financial services in 2007.

Development results depend on a complex set of factors, some of which are, for better or worse, beyond IFC control. As the global financial crisis is taking its toll, development outcomes of the corporation’s investments are likely to deteriorate too, despite efforts to support clients.

More generally speaking, one must be humble when claiming responsibility for positive development impact. It is important to differentiate between what can be known about a project’s contribution to development results – as defined through project-specific outcome indicators – and what can only be inferred about an investment’s follow-on contribution to economic development and country-level poverty reduction.

It is inherently more difficult to assess the follow-on effects of any investment – and the IFC’s role in triggering them – than to assess the immediate results. There is a multitude of actors and it is quite difficult to understand how exactly economic and social forces interact.

Consider, for example, a new agribusiness company which sources inputs from farmers in its area. Subsequently, farmer incomes and school enrolment go up. Nonetheless, it is difficult to prove that the latter was caused by incomes rising thanks to the new company. School enrolment may have gone up anyway. By comparison, it is much easier to establish how many jobs were created, or how many farmers sold their produce to the company.

Attributing impact on people’s socio-economic well-being to a company’s operation is easier, moreover, if the operation is new, well-defined and takes place in relative isolation. Many of IFC-supported investments, however, do not fit these characteristics. Promoting growth, after all, is not only necessary in remote rural communities, but also in highly complex megacities, where many causes interact.

Assessing the precise impact of IFC involvement is difficult for yet another reason. There is a limit to the financial support the IFC can provide to private companies. In most cases, the IFC is one of several financiers, and the corporation typically only covers up to 25 % of an investment. This rule makes sense, because the IFC might otherwise crowd out private financiers, thus stifling market dynamics. It implies, however, that, along with the limit on investing, there is a limit on claiming responsibility for a particular impact.

If it is challenging to establish a causal relationship between a company’s activities and poverty reduction in a community, region or country how can IFC know that part of the (intended or unintended) impacts were due to the institution’s involvement? The IFC, like other publicly-funded development agencies, must prove its worth. Rather than competing with the private sector, it complement it wherever markets fail.

Therefore, the IFC aims to support projects where either
– no alternative funding is available – thus supporting the development of markets and the opportunity to escape poverty – or
– where projects are enhanced through IFC’s expertise in the areas of corporate governance, environmental and social standards, industry and global knowledge.
For each project IFC assesses ex ante whether it can add value across these sets of criteria and examines its development results throughout the life of the project.

In order to judge IFC’s development effectiveness, the corporation has adopted an M&E system with which it tries to best approximate the contribution to economic development and poverty reduction. Developmental reach indicators, aggregated development outcome ratings and periodic in-depth project evaluations all provide an indication – as does a recent analysis of how benefits to society compare to benefits to financiers. The data for a host of IFC investments approved between 1999 and 2002 show not only that monetised benefits to host countries exceeded investment costs in the order of ¢ 88 above each dollar invested, but also that these economic benefits accrued equally to the financiers and host societies represented by stakeholders such as employees, governments, suppliers or customers. As other development institutions, IFC continues to work on linking its activities through different M&E approaches to development results and impacts.