The advent of the Sustainable Development Goals, growing dissatisfaction with current economic models, the explosion of impact investing and other innovative finance models coupled with the fourth industrial and the ubiquity of digital technology has created the context in which an overhaul of current monitoring and evaluation practices is required.
“Most areas of the development sector are adapting and innovating but the evaluation discipline hasn’t caught up. Many of the methods and processes have remained unchanged for years. They simply don’t meet the demands of the new stakeholders and new models that are being piloted in the sector at the moment,” says Reana Rossouw Founder of Next Generation & Creator of Investment Impact Index.
But things are starting to change, as Rossouw outlines in the Next Generation 2020 Research Report, Disruption with Impact. The OECD has acknowledged that the current approach to monitoring and evaluation is failing as evidenced by their decision to review their evaluation standards and draft new criteria for evaluation. This process kicked off with extensive stakeholder engagement that showed that the current approach is ineffective and increasingly irrelevant within the current context.
“Its exciting to see new approaches such as participatory evaluation, democratic evaluation, empowerment evaluation, horizontal evaluation, realist evaluation, utilisation focused evaluation, culturally responsive evaluation emerging and this suggests to me that there is a need for evaluation that is tailored to different contexts,” Rossouw elaborates.
The International Finance Corporation is also piloting a new rating system, the Anticipated Impact Measurement and Monitoring (AIMM), to assist with better allocation of finances and support its existing Development Outcomes Tracking System.
These moves by influential players in the sector point to the growing need for a global ‘gold standard’ for monitoring and evaluation that takes into account the shifting context and accommodates the needs of both the traditional development sector as well as the growing impact investment sector.
What is the difference between M&E and impact measurement?
- This has historically looked at monitoring programme implementation and evaluating the outcome of individual programmes.
- Impact measurement:
- This aims to manage and control the process of creating impact to maximise impact while managing costs.
Despite the differences in language and in approach both sectors are facing similar challenges in responding to the shifting context. Below we outline some key trends affecting both sectors.
- The SDGs requires M&E and impact measurement practice that speaks to the 17 goals and the 230 sub-indicators, at a country level, a regional level as well as a specific theme.
- Demand for indicators that measure both financial and social/environmental impact that also aligns with or considers new and emerging innovative/blended financial models.
- The need for standardised, appropriate and effective methodologies and toolkits that respond to the performance management and measurement requirements of a range of new actors including customised measurement methodologies and approaches.
- Until now traditional M&E practice focused on developmental outcomes, now there is a much greater focus on and need for financial indicators to evaluate the financial value and impact of social and impact investments.
- There is now a need for new methodologies that look for the contribution effect (that tracks collective, attribution, contribution and leverage of multiple contributors and funders) rather than the attribution of a single funder.
- Nuanced data is now required that speaks to the entire investment cycle, not just the programme cycle.
- New data collection approaches that use the strengths of digital technologies to capture data more cost effectively and efficiently.
- Ethics and principles that guide behaviour, is an especially important topic in evaluation practice. Principled action and the protection of human rights, data privacy and security have become serious considerations for all stakeholders in the value chain.
Roberto Picciotto also distils how these trends translate into key policy directions for the sector:
- Internationalisation: not just across borders but M&E will increasingly need to be adapted to local contexts while remaining aligned to global standards. This will be supported by the growth of global evaluation associations and networks and increasing professionalisation of the sector.
- Diversification: as the development sector attracts new actors, monitoring and evaluation will need to become more inclusive of a diverse range of needs and stakeholders.
- Digitisation: programme funders and implementers will no longer accept slow and lengthy M&E processes and evaluators will be under increasing pressure to the use new technology, and big data approaches to report on impact quickly and more regularly.
Currently M&E and impact measurement are seen as two distinct subject areas.
What we can see is that both sectors are dealing with the same trends.
In the Next Generation 2020 Research Report Disruption with Impact we advocate for these two branches to come closer together to offer a more cohesive approach to monitoring and evaluation.
With the growth in impact management and measurement approaches (IMM) – there are several implications for practitioners in the sector, most notably:
- New guidelines, frameworks and standards – such as the IFC Principles for Impact Management and Measurement or the Impact Management Project
- New practice requirements, including: investment period, investment portfolios, investment size and multiple organisations, forecast impact and return, manage and measure impact and return, report impact and return on investment publicly, compare and benchmark and new language.
- New methodologies and new metrics/indicators. The development of IRIS+ for instance, brings much greater clarity on what to measure and what unit standards to use in the measurement of impact.
“M&E and impact measurement is on a precipice and needs to evolve to take into account the multidimensional and complex qualitative context of development work and ensure that it speaks to the overarching goals of the SDGS. M&E must also start to account for the growing tendency for programmes to be implemented across borders and by various stakeholders including the public sector, private sector and philanthropic funds while also tracking more long-term, secondary and indirect outcomes,” says Rossouw.