Lecture 1: Causal Inference: Potential Outcomes

Roadmap for Causality Lectures

• Focus will be on causality.
• Examination of how economists view causality.
• Discuss the difficulty in estimating causal effects without experimental research design.
• Why randomized trials are considered the ‘golden standard’.
• Guidance on analyzing data from experiments.
• Reasons why even randomized trials can fail to produce causal estimates.
• Examples of randomization in economics research.

Readings for Causality

• Angrist and Pischke, chapters 1 and 2.
• Wooldridge, chapter 1.4.
• Articles.

Why Focus on Causality?

• The most compelling research in economics frequently addresses cause-and-effect relationships.
• A causal relationship is valuable for forecasting the effects of changing policies.
• While it may serve other purposes, purely descriptive research isn't very helpful in that regard.

Causal Relations & Experimental Approach

• The experimental approach plays a crucial role when examining causal relationships.
• Modern microeconomics has increasingly adopted scientific experiments as the "golden standard" for inference.
• Throughout history, numerous scientific advancements have been facilitated through experiments, dating back to the Renaissance with figures like Galileo.

Questions Involving Causality in Economics

• Labour economics seeks to understand how participating in an active labor market program affects employment probabilities.
• Education economics aims to determine whether smaller classes improve study results.
• Health economics explores the impact of absolute/relative income on health and mortality.
• Macroeconomics investigates the effect of unanticipated changes in short-term interest rates on current and future activity.
• Crime economics questions whether more police presence reduces crime rates.

Causal Relations in Economics

• Researchers are interested in the causal effect of participating in a treatment on future outcomes.
• Models for analyzing causal effects are also models for treatment/policy/program evaluation.
• “Treatment” is defined broadly, including actions like attending university or getting married.

Definition of Causality

• An action causes a particular effect if the effect would not have occurred without the action.

Causality Formalized

• A framework for thinking about causality involves a population of N units, such as individuals, firms, or countries.
• An outcome variable Y and a variable D is observed for each unit.
• There is an assumption Y and D are correlated.
• Correlation does not imply causation.
• It is valuable to know under what specific circumstances it is possible to infer that D causes Y.

Causal Relations in Economics Defined

• "i" represents the index for a particular unit in the population.
• Dᵢ symbolizes a "treatment".
• Assume "treatment" is binary, either "yes" or "no".
  • Dᵢ = 1 if unit i is exposed to treatment.
  • Dᵢ = 0 if unit i is not exposed to treatment.
  • Yᵢ(Dᵢ) represents the observed outcome.

Potential Outcome Model

• Defining potential outcomes of unit i creates a counterfactual framework.
• Each unit has two potential outcomes (Y₁i with treatment; Y₀i without).
• Y₁i and Y₀i refer to potential outcomes (treated/non-treated) for unit i whether the unit was actually treated.

Potential Outcome Examples

• An unemployed person seeking a job could participate in a job search program.
• Y₁i: potential labor market outcome if the person participated in the program.
• Y₀i: potential labor market outcome if the person did not participate.
• Those are potential outcomes unrelated to actual treatment status.

Definition of Causal Effect

• In potential outcome framework, only one potential outcome is observed
• The unobserved potential outcome is also called the counterfactual outcome.
• For any given unit, the impact of partaking in the treatment is:
  • Δᵢ = Y₁ᵢ – Y₀ᵢ is the difference in potential outcomes
  • This is also the definition of causal effect at the unit level

Causal Effect Example: Job Search Program

• Example: comparing unemployed workers who went to a job search program and those who didn't may incorrectly assess the program.
• Workers who went to the program may have worse labor market prospects.
• Potential outcomes:
  • Y₁i: the employment outcome of person i going to the program
  • Y₀i: the employment outcome of person i not going to the program
• The causal effect of participating in the program: Δᵢ = Y₁ᵢ – Y₀ᵢ

Fundamental Problem of Causal Interference

• Δᵢ is unobservable because only Y₁ᵢ or Y₀ᵢ can be observed.
• It is impossible to derive causality at the unit level because units cannot receive and not receive treatment at the same time.
• This difficult problem can be approached with either a "scientific“ or a statistical solution."

Overcoming Counterfactual Problems: Scientific Solutions

• Consider a unit i with the assumptions:
  • Temporal Stability: Outcome y does not depend on when the treatment takes place.
  • Causal Transience: Outcome y is independent of any prior treatment.
  • Homogeneity of Units: An assumption for other units j ≠ i such that yi(xi) = yj(xj) for xi = xj
• Assumptions are used in natural sciences to infer causality.
• These are unlikely to hold in social sciences because the environment is is not controllable such as it might be in a lab.

Statistical Solution to the Counterfactual Problem

• Methods that compute the average causal effect for the entire population or subgroups of interest
• In economic literature, the Average Treatment Effect (ATE) is used.
• Formula: ATE = E[Δᵢ] = E[Y₁ᵢ - Y₀ᵢ] = E[Y₁ᵢ] - E[Y₀ᵢ]
  • The ATE of Dᵢ = 1 gauges how much, on average, an individual benefits from receiving the defined treatment.
• ATE compares the potential outcome when all units receive the treatment to the potential outcome when no units receive treatment.
• Neither outcome, however, is typically directly observed.

Average Treatment Effect on the Treated (ATET) in Economics

• The average treatment effect on the treated describes how much on average the individuals who actually received treatment benefited from the treatment:
  • E[∆ᵢ|Dᵢ = 1] = E[Y₁ᵢ - Y₀ᵢ |Dᵢ = 1] = E[Y₁ᵢ |Dᵢ = 1] – E[Y₀ᵢ |Dᵢ = 1] = 1, involves one unobserved counterfactual that indicates the potential outcome as untreated for those who actually received treatment.

ATE and ATET: Job Program Example

• For a job-search program:
  • ATE answers: If all unemployed workers participate in the program, how much would employment increase?
  • ATET answers: How much would employment increase for workers who selected into the program?
• The questions cannot readily be answered, because they require comparison the person's observed outcome to the counterfactual outcome.

Estimating ATE and ATET: Self-Selection

• A self-selection problem must be addressed to estimate average treatment effects:
  • E[Y₁ᵢ|Dᵢ = 1]- E[Y₀ᵢ|Dᵢ = 0].
  • In other words, can't average outcomes can just be compared for those who were treated to outcomes for those not treated?
• This would require the average potential outcome in a non-treated condition to be the same for treated individuals as for non-treated individuals. -E[Y₀ᵢ|Dᵢ = 1] = E[Y₀ᵢ|Dᵢ = 0]
• This is an unlikely event without an experimental variable.

Example of Self-Selection

• Workers motivated to find work are more likely to participate in a job search program.
• The above individuals potential outcomes, treated and untreated, are both more motivating than those less motivated to find work.
• Potential outcomes are not independent of actual treatment status.

Self-Selection Conundrum

• With self-selection:
  • E[Y₀ᵢ | Dᵢ = 1] ≠ E[Y₀ᵢ | Dᵢ = 0] and E[Y₁ᵢ | Dᵢ = 1] ≠ E[Y₁ᵢ | Dᵢ = 0].
• The potential outcomes as untreated for those who were treated are not the same as the potential outcomes as untreated for those who were not treated.
• E[Y₀ᵢ | Dᵢ = 1] is not the same as E[Y₀ᵢ | Dᵢ = 0].
• Likewise the potential outcomes as treated for those who were treated are not the same as the potential outcomes as treated for those who were not treated.
• E[Y₁ᵢ | Dᵢ = 1] is not the same as E[Y₁ᵢ | Dᵢ = 0].

Formalizing Bias Resulting from Self-Selection

• It is possible to formalize self-selection induced bias.
• The Average Treatment Effect (ATE) is naively estimated via:
• E[Y₁ᵢ|Dᵢ = 1] - E[Y₀ᵢ|Dᵢ = 0]
• Adding and subtracting E[Y₀ᵢ|Dᵢ = 1] to the expression above produces:
• E[Y₁ᵢ|Dᵢ = 1] - E[Y₀ᵢ|Dᵢ = 1] + E[Y₀ᵢ|Dᵢ = 1] - E[Y₀ᵢ|Dᵢ = 0]
  • Simplifies the yield: -E[Y1i - Yoi |Di = 1] + E[Yoi|Di = 1] - E[Yoi|Di = 0]
• Which can be written as:
  • E[Y1i - Yoi |Di = 1] + (a bias term)
• A result gives ATET + bias term.

Example of Bias

• The bias term, E[Y₀ᵢ|Dᵢ = 1] - E[Y₀ᵢ|Dᵢ = 0], captures the difference in non-treated employment outcomes between those who did and did not get treatment.
• E[Y₀ᵢ|Dᵢ = 1] - E[Y₀ᵢ|Dᵢ = 0] > 0, since selecting a treatment is in some way dependent on the individual feeling it will improve some aspect of their lives where intervention is desired.

Why Selection Bias Matters

• Selection bias arises from optimizing decisions by rational actors.
  • I am in it if worth it.
  • D = 1 if Y1i - Yoi > c ,
  • C represents the mental and monetary cost of participating in the treatment.

• The above can then be used to write:
  • E[Yoi |Di = 1] = E[Yoi |Yoi < Y1i -c]
  • E[Yoi |Di = 0] = E[Yoi |Yoi > Y1i -c]
• With groups differing in terms of comparative advantages.
• Y1i - Yoi is large for some, small for others.
  • The most simple instances: treatment participants have smaller Yoi, thus larger potential gain.

Other Selection Sources

• May occur due to administrative rules or by selection coming from treatment providers.
  • "Cream-skimming": Choosing “the best” suggests that E[Yoi|Di = 1] > E[Yoi|Di = 0].
• Selection bias also results come from negative treatment:
  • Negative Treatment Example: Placing "weak" kids in small classes produces the formula:
  • E[Yoi|Di = 1] < E[Yoi|Di = 0]

The Key Problem for Treatment Evaluation

• Key unobservable/unestimatable values with the formula E[Y₀ᵢ|Dᵢ = 1], E[Y₁ᵢ|Dᵢ = 0], E[Y₁ᵢ] and E[Y₀ᵢ]. Treatment evaluation's primary issue is isolating something, with exogenous variation, that only influences treatment assignment and not potential outcomes.
• In other words, the treatment status to be independent of potential outcomes and have a valid couterfactual to evaluate treatment

Randomization Solves the Self-Selection Problem

• Social experiments randomize treatment assignment across individuals via a lottery
• Randomization implies treatment assignment that is statistically independent of potential outcomes and other variables:
• (Y0i, Y1i) ┴ Dį
• In the job search example: using a lottery to assign individuals.
• The "treatment" is independent of potential outcomes.

How Randomization Solves the Selection Problem: Example

• In a job search, motivated workers often selected treatment.
• Randomizing treatment creates equal distributions of motivation in treatment/non treatment groups.
• With completely random treatment the groups will look alike, removing selection
• With treatment status the sole group differentiator, each group counteracts the others' factuality.

Randomization Solves The Selection Problem Concluded

• With randomization, treatment status is independent of potential outcomes with these mathematical effects:
  • E[Y1i|Di = 1] = E[Y1i|Di = 0] = E[Y1i]
  • E[Yoi|Di = 1] = E[Yoi|Di = 0] = E[Yoi]
• In a job search program, this implies the following:

Randomization Solves the Selection Problem Concluded 2

• Random assignment of Dᵢ eliminates selection.
  • ATET = E[Y1i|Di = 1] – E[Yoi|Di = 1]=E[Y1i|Di = 1] – E[Yoi|Di = 0]
  • We swap out E[Y1i|Di = 1] for E[Yoi|Di = 0] because treatment status is independent of potential outcomes.
• As a result, E[Y1i|Di = 1] and E[Yoi|Di = 0] are observed.

• E[Y1i]) is the same as the potential outcomes of those who actually took the program E[Y1i|Di = 1]).
• With randomization ATE = ATET as groups are similar despite treatment.

Important Effects of Randomization- World Evidence

• In the social sciences randomization isn't always possible and control strategies (controlling factors that differentiate outcomes) are frequently applied instead
• There are many occasions where both experiments are tried to find a similar result.
• This allows comparison for better alignment with experimental ideal
• Sadly, medicine and economics studies show randomization is superior to control strategies

How Important is Randomization- HRT Medicine Example?

• Recent instances come from hormone replacement therapy investigation
• Evidence from the Nurse's Health Study, a large, influential non experimental survey, showed better health with HRT used with a control strategy
• Results from similar randomized trails showed exceeding, harmful health risks.
• • The prior positive reflection was self-selective, in that health conscious women got selected which explains the ATET + bias term

Important Effects of Randomization World Economics Example

• There are few, non experimental studies with labor market training paradoxically find less earnings from participants compared to nonparticipants.
• One may suspect selection bias from training serving low income individuals leading to naive comparisons
• Randomized training programs, however, boast positive effects.

Explore causality, potential outcomes, and counterfactuals. Understand the notation, significance, and interpretation of causal effects. Learn about observed and unobserved outcomes in causal inference studies.