Measuring and Developing Performance Systems

Measuring Your Impact

• Measuring impacts aids learning and progress and helps to evaluate the effectiveness of investments.
• The easiest way to develop an impact measurement system is to start with a clean slate, rather than being constrained by past choices or sunk costs.
• Table 9 summarizes the steps to develop or refine an impact measurement system.

Developing a Performance Measurement System

• Step 1: Prepare the measurement foundation which Involves linking planned actions to desired results, and inventorying the full range of impacts.
• Step 2: Consider how to use results where the system can be used for control or understanding, serving purposes like learning, action, and accountability.
• Step 3: Identify key impacts and metrics by Identifying significant impacts, deciding which primary and secondary impacts to monitor, and selecting metrics for those impacts.
• Step 4: Develop the measurement system and align it with organizational strategy, gather and analyze data, communicate results, and take action to improve impacts.

Prepare the Measurement Foundation

• A strong measurement foundation ensures valid and useful information.
• The first step involves reviewing the logic model or results chain that links activities and results in a causal model, which is often the most difficult part.
• Clear desired impacts are essential for designing a measurement system that provides needed information.

Linking Planned Actions to Desired Results

• The mission statement defines the social purpose of the organization or intervention and the desired social changes such as maximizing positive social impacts or minimizing negative ones.
• A logic model lays out a clear path for creating changes.
• The logic should define inputs, actions, and outputs, and clarify how outputs will change outcomes and lead to the desired social impacts.

Inventory Your Full Impacts

• When developing a measurement system, organizations need to be aware of the various ways in which they generate impacts, determines which ones are significant enough to include.
• Measurement systems include primary, targeted impacts from the logic model, plus positive and negative impacts for beneficiaries and other stakeholders.
• Stakeholders can be affected via products, services, operations, and passive investments.
• Work creates second-order impacts, which are changes that result from direct impacts.
• Beneficiaries are affected by primary and secondary impacts, as are other stakeholders.
• Direct/indirect beneficiaries should be involved to align interests and explore risks and to ensure buy-in from beneficiaries.
• Stakeholders include supply chain partners, investors, regulators, local authorities, and community members.
• Competitors can be stakeholders, because an organizations ability to attract lower-than-market-rate funding may undercut and damage local for-profit competitors.
• Secondary impacts for beneficiaries can be positive (increased social status) or negative (strained family relationships).

Consider How You'll Use the Results

• Identify impacts to be measured, and consider how to use the results, acknowledging that measurement has costs and requires selectivity.
• Consider actions based on results and how they affect the ability to improve impacts.
• Providing information about impacts and their causes is the most important aspect of measurement and is a key driver in accomplishing the mission.
• Measuring impact is subjective, so intentional use of measures is essential.
• Two basic ways to use measurement systems in organizations are for control and understanding

Measuring for Management

• Using measures for control can have unintended consequences like undermining an organizations impact
• Programs are likely to be managed on metrics if rewards are based on them, rather than on improving outcomes.
• Decision-makers may manage or manipulate measures with a variety of mechanisms such as reclassifying general expenses as program-related expenses.
• Distortions occur based on overhead measures, and social impact can be manipulated.
• An organization might focus efforts on areas where water is already available, even though serving more remote clients would be more difficult and expensive.
• Care when using measures to control performance to ensure that measures don't have dysfunctional consequences.
• Impacts often result from many actions & beyond control of one manager/group .
• Rewarding managers based on results they can't control is problematic.
• Measures can be beneficial if they encourage collaboration and acquisition of the resources.
• Using measures to gain understanding is the essential goal of any performance metrics system. It’s the primary purpose of measurement in the Social Impact Creation Cycle.
• Early assessment of impact is recommended.
• Embed impact measurements throughout the investment and or run simulations to predict which outcomes will likely result from various designs.
• Test the intervention on a small sample.
• Assessments may require identification of a control group to create a counterfactual by comparing beneficiaries who receive intervention with those who didn’t, this creates evidence about a programs responsibility for the results.

Measuring for Understanding

• Understanding social impacts is a central concern for all social purpose which should be identified before program implementation.
• Impact measurement can play a key role in this effort for learning, to guide behavior, and to report performance.
• Good measurement systems can provide insight into effectiveness and results.
• Measurement can be used to compare the effectiveness of programs across populations or geographic regions to identify the differences.
• Organizations often use outputs instead of impacts
• Effective measurement can help organizations develop a better understanding and can be used to evaluate the links between elements in the logic chain and hypothesized relationships.
• Measurement can guide action and make changes in plans, improves organizational performance and increase impacts.
• Emphasize the purpose of the measurement like improving decisions and strategies & focuses on a limited number of questions.
• Measurement can be an effective tool and is used to communicate what areas the organization considers important.
• Measurement can be used for accountability to report evidence to stakeholders.

Identify Key Impacts and Metrics

• Deciding which impacts to measure requires asking the right questions, identify most significant impacts, decide what information is essential, and decide which metrics to use.
• Answering the right questions about impact begins to know whether the plan is working (Am I making an impact?).
• After inventoried the impacts, they are ranked and those of most significance are chosen
• A sound logic model helps be clear on the impacts the organization want to achieve with that being understand the organization’s existence, and questions should be asked that focuses on those impacts

Social Impacts

• Social impacts can vary in desirability, scope, intensity, direction, and duration.
• Metrics are intended to represent something in the real world some are intended to be easy to measure, while others are more difficult (i.e. self-confidence)
• Metrics for advocacy efforts can be even more difficult.
• Many organizations provide lists of sample metrics that can fit various situations.
• The Impact Reporting and Investment Standards (IRIS) metrics are well known in the nonprofit sector, as is the Global Reporting Index (GRI).
• Socially responsible investment funds use ratings to rank companies & MSCI ESG scores, built on the well-known KLD ratings, is one example.

Action Agenda

• Decide which impacts are most important and also the needed information to manage.
• Decide how you'll use information about impacts and their drivers.
• Decide which metrics will best represent the significant impacts.
• Develop the impact measurement system.

Measures for Primary and Secondary Impacts

• Measurement of impacts begins with identifying he broad changes that the organization’s wish to make, this is shown through the organization’s metrics
• An absolute level, such as tons of CO2 emitted can be used to use as an indicator
• A percentage, such as the percentage of children who have learned to read can also be used as an indicator
• A ratio, such as crop yield compared to the amount of fertilizer used can be used as a measure
• There can be comparisons with a standard that will provide the context for interpreting them to target results
• As well as compared to results from the period before and to comparable organizations or to industry benchmarks
• Metrics pursue in each area are provided in Figure 21 and measurements pursue impacts, but are difficult to measure
• Guidance is provided every day, the organizations grapple with elusive outcomes and impacts
• There is examples of measures given in six categories that many social purpose organizations work in

Developing Your Measurement System

• The final step is developing the performance measurement system.
• Consider how defined the system will have to be implemented and put into use, by focusing on the impact measurements.
• Implementing the social impact measurements follows a similar path, except that the strategic nature of this system requires that you take special.
• Key to implement the system throughout all steps is organizational and measurement system goals alignment.

Gathering Data

• A plan for gathering reliable & complete data that will have to be used for the impact report
• Including how when the data will be collected and from the sources of external or internal.

Analysing Data

• Data will have to be thoroughly tested and carefully designed to make sure data is reliable and complete

Communicating Results

• Communication can make a big difference on the measurement system
• Reports aren’t useful if they aren’t used, reports will include the relevant and relevant information
• The information has to be used and discussed consistently
• Investors will require evaluations to verify the reports that are given