Explore The Five Dimensions Of Impact

Contribution

Did the enterprise’s activities contribute to the outcome achieved? What would have likely been the outcome had the enterprise not done what it did? The impact data categories under the ‘Contribution’ dimension help us answer these questions.

The ‘Contribution’ impact dimension recognises that impact occurs in a dynamic social system, with various stakeholders playing different roles. To understand their own contribution to a social or environmental outcome, enterprises need to consider what would have happened in absence of their activities. This page provides guidance on the ‘Contribution’ data categories that enterprises and investors can use to collect, assess and report data about what would likely happen otherwise. If you are unfamiliar with the impact dimensions (and their respective data categories), we recommend first visiting the short section What is Impact.

Introduction: The 'Contribution' dimension of impact

The data categories under the ‘Contribution’ dimension help enterprises and investors assess an enterprise’s contribution to the social (environmental) outcomes that people (planet) experience, relative to what the market or social system would have done anyway.

Enterprises and investors operate in a dynamic social system, with various actors – from competing firms to government bodies to NGOs – seeking to contribute to the same set of outcomes.

Consider a solar energy company aiming to improve the health of Kibera slum residents by reducing kerosene use. This enterprise operates in an environment where other enterprises, government policies and NGO programmes are all striving to achieve the same outcome but through different mechanisms. Accordingly, if customers’ health improved by 20%, the solar energy company would need to consider how these other initiatives contributed to this percentage change, in order to understand its own contribution to the outcome.

The two categories under the ‘Contribution’ impact dimension help enterprises understand the extent to which they contributed to an outcome by considering what would have otherwise happened in absence of their activities (i.e. a counterfactual scenario). They are:

  • Depth: An enterprise’s contribution to the depth of an outcome by factoring in the estimated degree of change that would have otherwise happened
  • Duration: An enterprise’s contribution to the duration of an outcome by factoring in the estimated duration that the outcome would have otherwise endured

The categories’ insights are essential. If an enterprise finds that its contribution to a particular social or environmental challenge is minimal, then it may decide to (re)allocate resources towards products that show better social value for money. Conversely, if the contribution is significant, then the enterprise may decide to double down on resources.

More broadly, by considering their own contribution, enterprises deepen their awareness of the system they operate in – who the key actors are, how they interact, and what the levers for change may be. Equipped with this understanding, enterprises can work towards optimising the whole system rather than a single intervention. This may mean developing a partnership with an NGO to reach last mile consumers. It could also mean collaborating with government to upskill and increase the resilience of smallholder farmers, at scale.

In impact terminology, ‘Contribution’ overlaps with terms such as ‘additionality’, ‘deadweight’ and ‘attribution’. While they all seek to answer ‘what would have happened anyway?’ to derive an understanding of an enterprise’s contribution to an outcome, these terms often differ in methods and scope. 

We have deliberately chosen the word ‘Contribution’ to cover all of the ways in which the counterfactual can be assessed—from market and evidence-based research to randomised control trials.

The remaining sections provide guidance on the two ‘Contribution’ categories, focusing on how enterprises can assess their contribution to a social or environmental outcome.

How can enterprises evaluate their contribution to the depth of an outcome?

The depth category helps enterprises assess their contribution to the depth of an outcome by accounting for the estimated degree of change that stakeholders would have otherwise experienced.

The depth category under the ‘Contribution’ dimension enables enterprises to understand the extent to which their activities were responsible for the outcome realised. To this end, enterprises need to consider the estimated degree of change that stakeholders would have otherwise experienced (in absence of their business activities). A variety of methods are available to assess depth, from randomised control trials to market research (covered under Methods to calculate depth at the end of the section).

This category should not be mistaken with depth under the ‘How much’ dimension, which covers the significance of the outcome by calculating the difference between the outcome in period and the baseline, without considering the influence of other factors (e.g. other organisations, economic conditions).

As the conceptual diagram illustrates, an enterprise’s contribution to an outcome equals:

An enterprise’s depth contribution =

outcome in period – outcome that would have been observed anyway

Conducting this depth analysis is critical for decision-making. For example, a marginal depth contribution may indicate an ineffective intervention or a crowded market. By contrast, a large depth contribution may reinforce the impact value of the product (or policy). At a higher level, this process can surface valuable information about the dynamics of the social system, including the impact performance of other organisations or external factors that also shape the outcome. Based on these insights, enterprises can (re)allocate resources towards maximising their contribution – whether by tweaking a policy, shutting down an initiative or forging a cross-sector coalition. 

Using market research and stakeholder feedback to understand an enterprise’s contribution

 

By 2011, One Acre Fund (OAF) had reached 78,000 smallholder farmers in Kenya and Rwanda — an impressive 145% year-over-year growth. The rapid scale and impact per farmer ($120 avg. increase in yearly income) convinced the East-African based organisation to explore the Ghanaian market, with the view that it could serve as a potential launchpad for other West African countries.

 

After a few months of scouting, OAF launched a pilot programme with 500 farmers, offering them a loan package with a half-acre of maize seed and fertilizer. To understand the impact of the programme and acquire lessons for scaling up in West Africa, OAF implemented a threefold approach:

 

• Collection of outcome data before the programme and at particular milestones (the indicator being absolute yearly income)

 

• Research on system dynamics

 

• Interviews with farmers and OAF field officers

 

By triangulating these data points, OAF found an unfit-for-purpose environment, making its contribution to the outcome minimal.

 

First, farmer reliance on agriculture was low, with farmers also generating an income from other sources.

 

Second, the maize-based programme was implemented in a region where there was a stronger focus on cash crops. While OAF quickly pivoted to another region with ideal farming conditions, the organisation realised that the addressable population was too small to make the business model work.

 

Third, upon moving to the new area, a lack of rainfall plagued the farming season. OAF soon discovered that these droughts were not an anomaly, but a regional shift towards semi-arid growing conditions.

 

Based on these findings, OAF decided to shut down its Ghanaian operation, but not without gaining important lessons through the process, including the need to: (1) hyper-professionalise country scouting units; (2) constantly adjust and innovate during pilots and scale-ups; and (3) locate operations far from large urban cities when commercial agriculture and non-agricultural activities dominate.

 

One Acre Fund (2014)

Methods to calculate depth

To calculate the counterfactual side of depth, enterprises can use a number of approaches that vary in rigour and costs. Randomised control trials and quasi-experimental methods typically require significant resources but produce higher-quality insights compared to market research and stakeholder feedback. This is not always the case, however, as well-deployed market research and stakeholder feedback (covering large sample sizes) can yield valuable insights for (1) understanding what else may be driving the outcome, (2) building a ‘good enough’ counterfactual scenario, and (3) conducting the depth analysis.

These methods can often be combined to gain complementary findings. The list below covers the main analytical tools:

1. Stakeholder feedback: Stakeholder feedback requires consulting the individuals (or communities) affected by the enterprise’s activities to gain a nuanced understanding of the drivers behind the outcome (e.g. the enterprise’s activities, external factors, government interventions, cultural practices). If deployed well — covering a large enough sample and different points of view —, stakeholder feedback can be used to build a counterfactual. This method should be combined with market research and/or evidence-based research, as they are mutually reinforcing.

2. Market research: By taking a thorough look at an intervention’s context, market research can be used to build a ‘good enough’ counterfactual. This method requires a deep analysis of secondary resources (e.g. industry reports) to identify what else may be driving the outcome — from other organisations, to government interventions, to external factors (weather, economic conditions), to individuals’ unobservable characteristics (self-motivation, cultural practices). Market research should be paired with stakeholder feedback and/or evidence-based research for complementary insights — and for strengthening the credibility of the counterfactual.

3. Evidence-based research: Enterprises can source depth contribution estimates through evidence-based research (i.e. rigorous impact studies of enterprises’ products, services, and other types of interventions conducted by third-party researchers). Often grounded in randomised control trials or quasi-experimental design – two methods that rigorously assess the counterfactual –, evidence-based research produces outcome results that can be extrapolated to gain an understanding of an enterprise’s contribution.

Before extrapolating results from a study, enterprises should first assess the quality of the estimate by considering the study’s methodological rigour, population group, country-setting, and type of intervention. For example, if an Indian enterprise relied on an estimate from an impact study that took place in Argentina – a country with significantly different socio-economic characteristics –, then the quality of this estimate is likely to be low, rendering it unusable. In using this method, enterprises should exercise caution and aim to pair it with stakeholder feedback and market research. Sources of evidence-based research include J-Pal’s evaluations, Innovations for Poverty Action’s research and 3ie’s systematic reviews.

4. Randomised control trials (RCTs): RCTs measure the difference in outcomes over time among two randomly assigned groups:

  • A treatment arm (i.e. receives the intervention such as a product)
  • A control arm (i.e. one that did not receive the intervention, or received a placebo or another type of intervention).

The randomisation ensures that the two groups are similar on observable (income, gender, health) and unobservable (self-motivation, energy) characteristics, creating a robust counterfactual. Although a popular method in international development, RCTs and quasi-experimental methods usually require significant resources. 

5. Quasi-experimental methods: Quasi-experimental methods (e.g. regression discontinuity design, difference-in-difference) cover a range of statistical techniques to build experimental groups. Once these groups are created, practitioners compare the difference in outcomes over time between individuals who received the intervention and those who did not (the counterfactual). In contrast to RCTs, quasi-experimental methods require many more assumptions to develop a credible counterfactual.

Along with depth, enterprises need to consider duration, the second variable of an enterprise’s contribution to an outcome. While this section focused on the depth counterfactual and analysis (i.e. depth category), the remaining module covers the duration counterfactual and analysis (i.e. duration category). The two categories share a similar purpose but differ in the insights they provide to enterprises and investors.

How can enterprises evaluate their contribution to the duration of an outcome?

The duration category under the ‘Contribution’ dimension helps enterprises assess the duration of an outcome relative to what the market would otherwise deliver.

The duration category under the ‘Contribution’ dimension estimates an enterprise’s contribution to the duration of an outcome experienced by its stakeholders. To achieve this, enterprises need to account for the estimated duration that the outcome would have otherwise endured in absence of their activities (i.e. a counterfactual scenario).

Calculating an enterprise’s duration contribution to an outcome yields important sustainability insights. If an enterprise finds its duration contribution to be marginal compared to other enterprises’, it can use this information to activate resources towards improving the endurance of the product (or policy) or shift attention towards a more promising intervention. Conversely, if it finds its contribution to be significant, the enterprise can leverage resources to replicate the success of the intervention elsewhere.

Just as with financial performance, companies need to think of social and environmental performance over the short-, medium- and long-term to make meaningful contributions to society. This thinking entails engaging in initiatives and creating products that plan for and assess the duration of outcomes.

Analysing an enterprise’s duration contribution does not need to be complex. As seen in the illustrative example below, outcomes have different time spans. ‘Improved yearly income’ may endure for 24 months, whereas ‘improved psychological well-being’ may last for nine months. Similarly, some outcomes occur immediately whereas others only materialise in the long-term. For example, ‘improvements in educational attainment’ may only become visible after six months of a launched initiative. Finally, outcomes may persist beyond the end of the intervention (see ‘improved nutritional intake’ and ‘improved yearly income’ in our example).

As with depth, multiple players and factors may be contributing to the duration of the outcome, such as government subsidies, NGOs or other enterprises (see diagram below).

Based on this understanding, assessing an enterprise’s duration contribution equals:

Enterprise’s contribution to the duration of an outcome =

 

Duration of the outcome observed in period – Duration of the outcome that the market or system would otherwise deliver (i.e. a counterfactual scenario)

Estimating the counterfactual does not necessarily require a randomised control trial; enterprises can make use of market research, stakeholder feedback and the evidence base to derive a ‘good enough’ duration counterfactual (more on this in Methods to calculate duration at the end of the section).

This category is not the same as duration under the ‘How much’ dimension, which covers the duration of the outcome without considering what would have otherwise happened.   

Using the evidence base to estimate an enterprise’s contribution to the duration of an outcome

 

Driven by a desire to improve accountability in philanthropic giving, ImpactMatters (IM) conducts impact audits of grant-funded enterprises. IM makes use of existing internal and external data to provide a comprehensive account of the social outcomes achieved, recognising that enterprises may not have the resources to carry out additional assessments.

 

IM estimates an enterprise’s contribution to the duration and depth of an outcome by leveraging evidence from rigorous impact studies sourced from databases, such as J-Pal and Innovations for Poverty Action.

 

For example, to calculate the duration and depth contribution of a business’ support programme to participating companies’ net revenue (i.e. a primary outcome), IM combined internal programme data with a systematic review of 92 studies of similar interventions. Based on this evidence, IM estimated that each company earned an additional $2,600 in annual net revenue for a period of five years (intervention ended in year four), above what the business would have made in absence of the programme.

 

While acknowledging the limitations that come from extrapolating findings from impact studies, IM’s audited enterprises see significant value in the duration and depth estimates, serving as reference points for calibrating future programmes and ensuring that outcomes are deep and long-lasting.

 

Source: ImpactMatters, Impact Audit of TechnoServe MAS Programme (2018)

 

Methods to calculate duration

Calculating the duration counterfactual can be as simple as using market and evidence-based research or as complex as relying on experimental or quasi-experimental methods. While more rigorous and accurate, conducting an RCT with a long time span is inaccessible (and likely un-actionable) for the majority of enterprises. As a starting point, we recommend that enterprises leverage existing research to estimate the duration of the outcome that the market or system would otherwise deliver.

1. Stakeholder feedback: As explained in the previous section, stakeholder feedback gathers insights directly from the people who are experiencing the outcome. Stakeholder feedback could be a useful starting point for understanding the drivers behind the duration of an outcome. Enterprises can complement these findings with market or evidence-based research to further understand the estimated duration that the market or system would otherwise deliver (i.e. the counterfactual).

2. Market research: By taking a thorough look at an intervention’s context, market research can be used to build a ‘good enough’ duration counterfactual. This method requires delving deep into what else may be driving the duration of the outcome, from other organisations, to government interventions, to external factors (weather or economic conditions), to individuals’ unobservable characteristics (self-motivation, cultural practices). Market research should be paired with stakeholder feedback and/or evidence-based research for complementary insights – and strengthening the credibility of the duration counterfactual.

3. Evidence-based research: Relying on evidence-based research (i.e. rigorous impact evaluations, usually RCTs or quasi-experimental studies, explained below) can provide relatively accurate duration contribution estimates. When using this method, enterprises should determine the study’s methodological rigour, population group, country-setting, and type of intervention, in order to understand the quality of the estimate. For example, if an Indian enterprise used an estimate from a study that took place in Argentina (significantly different socio-economic characteristics), then the quality of this estimate would be considered low and likely unusable.

4. Randomised control trials (RCTs): RCTs measure the difference in outcomes over time among two randomly assigned groups: a treatment arm (i.e. receives the intervention such as a product) and a control arm (i.e. one that did not receive the intervention). Academic RCTs usually take place over two or more years and often go beyond the end of the intervention. By extending the evaluation period, RCTs capture reliable estimates of enterprises’ contribution to the duration of an outcome.  

5. Quasi-experimental methods: Quasi-experimental methods (e.g. regression discontinuity analysis, difference-in-difference) cover a range of statistical techniques to build experimental groups. Once these groups are created, practitioners compare the difference in outcomes over time between individuals who received the intervention and those who did not (the counterfactual). Similar to RCTs, this method assess data over two or more years (even after the intervention has ended), producing reliable data on the duration of an outcome.