Impact Evaluation in Inclusive Finance

Inclusive Finance: Lecture Notes

• Microcredit Miracle? High expectations and the Nobel prize for Muhammad Yunus and Grameen Bank (2006) led to many anecdotal accounts of transformative outcomes from microlending. Early research showed positive impact on borrower welfare (eg., Pitt and Khandker, 1998). High expectations existed for reducing poverty through microcredit. However, recent studies (2015 onwards) criticized early studies for methodology issues and called for randomized evaluations. Later evidence reveals potentially large negative impacts if microfinance is reduced (Breza & Kinnan, 2021). This is linked to reduced wages, household earnings, and consumption due to aggregate demand and reduced business investments.

Impact Evaluation in Microfinance

• Definitions and Concepts Essential concepts and components of impact evaluation are introduced. Econometric foundations, treatment and control groups, analytical concepts, microcredit impact in practice, access to finance in India, consumer lending, and self-assessment techniques are described. Empirical methods, such as General OLS, Diff-in-Diff, and Instrumental Variables (IV), are included in the explanation.

• Why Measure Impact? Accurate impact measurement is critical for resource allocation optimization in microfinance and development initiatives. It helps guide evidence-based policy, resource distribution, and program improvement, including development programs and interventions.

• How to Measure Impact? The need for a "perfect clone" or a counterfactual was outlined to measure the impact of microfinance.

• The Ideal Experiment The presentation shows a comparison of individuals receiving treatment vs. those receiving control in order evaluate treatment impact, including a visualization of outcome for treatment/control before and after the treatment. Visualized comparisons between differing outcome groups are included to show how to observe impact and the need for control groups.

• Reality and Counterfactuals Methods for addressing imperfect scenarios were discussed in the context of evaluating the impact of microfinance. It was emphasized that observing individuals only in one state of the world means that we cannot directly observe the counterfactual.

• Candidate Counterfactual Identifying appropriate counterfactuals was discussed, including considerations of a "younger Subir" vs. an "older Subir" as examples.

• Pre-Post Design and Its Complications A graphical presentation of a pre-post design where the income of a treatment group is contrasted over time with a control group. Visualizations of the control group and the treatment group are shown to indicate how outcomes can vary. There are outlined complications in the pre-post approach.

• Treated Vs. Untreated The difference in outcomes between groups with and without microfinance was demonstrated visually. The presentation highlights that using a treatment group versus a control group to calculate the effect will only work if the two groups behave the same. Complications exist in making these sorts of comparisons where the groups do not behave identically. Figures display outcome differences over time between treatment groups and control groups.

• What If There is No Perfect Match? When a perfect control is not possible, statistical methods can be used to estimate representative effects that provide estimations of outcomes for treatment and control groups. Charts depict the distribution of outcomes for each of the groups. The charts show that the treatment and control groups are distinct groups with unique distributions of outcomes.

• Comparing Distributions Different methods used to compare the distributions of the outcome variables within control and treatment groups were outlined. Techniques to evaluate outcomes, including comparing averages and running regressions, are shown including visualization of these outcomes.

• Econometric Implications The relationship between independent and dependent variables, concepts of endogeneity, omitted variable bias, and reverse causality were elaborated.

• Problems with Ordinary Least Squares (OLS) Endogeneity, reverse causality, and omitted variable bias were highlighted as potential issues in using OLS to evaluate impact in microfinance. These issues lead to inaccurate measurements of the effect of intervention variables that are actually dependent upon other variables.

• Instrumental Variables (IV) Approach Addressing endogeneity was discussed. The instrument variable z should explain the endogenous explanatory variable x without having a direct effect on the outcome y. This will provide a more accurate measure of intervention effect.

• Microfinance Impact The impact of microfinance on income and various factors were analyzed, with figures illustrating the outcomes over time.

• Access to Microfinance in India Studies relating to microfinance expansion programs in India and challenges of random program placement were covered. Different approaches to program placement and identification of treatment/control groups were presented. A sample study with a specific design was profiled.

• Consumer Lending in South Africa Experimental credit scoring is highlighted as a method to study the impacts of expanding microfinance access in a particular context as studied via a particular study.

• Outlook for Microfinance The evolution of financial products to meet the needs of borrowers are discussed with a focus on microsavings and microinsurance as becoming increasingly important, along with a discussion on how microcredit can be better suited for non-entrepreneurial borrowers.

• Self-Assessment Questions relate the content to practical examples with diagrams and tables included. Answers are detailed within questions.

• Econometric Background and Hypothesis Testing Provides an overview of the core concepts and methods used in empirical impact evaluations in microfinance. Detailed discussions in hypotheses testing and a recap of relevant statistical methods.

• OLS Estimation and Objectives The core concepts and rationale of using Ordinary Least Squares (OLS) as a fundamental approach in economic evaluation was explained.

• Hypothesis testing and Concepts A discussion of testing hypothesis against an alternative was included along different possible outcomes and hypothesis errors.

• Interpreting Coefficients Methods to interpret regression coefficients were presented in relation to dependent and independent variables, and examples of interpreting different variable types to account for the impact of an intervention.

• Difference-in-Differences (DiD) This shows how to compare a treatment versus a control group in an experimental research design, demonstrating a visual overview of difference in outcomes, both before and after the treatment intervention.

• DiD Assumptions This discusses what the assumptions need to be to use the difference in differences method and explains common issues of using the method. The presentation highlights issues regarding randomness and other underlying assumptions.

• DiD with Random Assignment This section summarizes how using a randomized design can make using DiD more reliable.

• Instrumental Variables This examines an approach to assess causality when an intervention (e.g., treatment) and an outcome may be linked through other variables that aren't accounted for