Causal Inference in RCTs

Causal Inference in Microeconometrics with Applications to Program Evaluation

• Lecture 1: Introduction by Bart Cockx, Ghent University, 2024
• Course Structure
  • 11 three-hour sessions on Fridays from October 4th to December 20th, 2024 (excluding November 1st)
  • Two two-hour sessions per session, each 50 minutes long, with 15-minute breaks and a 1-hour lunch break
  • Low quality recordings available via MS Teams
  • Guest lecturer (William Pariente) covering Randomized Controlled Trials (RCTs) on November 29th, December 6th, and 13th, addressing a particular topic

Evaluation Method

• Attendance is mandatory.
  • -1 point for each unjustified absence.
  • -2 points for any absence from the 3rd absence onwards.
• Assignments (10 points)
  • Groups of 2–3 students
  • Registration on Ufora
  • Model solutions posted on the platform
  • Support from Natalia Bermudez and Giulia Tarullo
• Presentations (10 points)
  • Group presentations on research or lecture-related topics
  • Maximum of 3 students per group
  • Options for selected papers or self-selected research
  • Aim: explaining lecture materials in own words (re-explaining lectures in own words)
  • Time: 15 min presentation + 5 mins discussion
  • Deadline: end of January (date to be confirmed)

Teaching Material

• All lecture materials, including slides, recordings, and documents, are available on Ufora.
• Non-Ghent University Students need official registration on OASIS to access and take the exam.
• Required Reading: Blundell and Costa Dias (2009)
• General Complementary Readings: (specific readings are described in individual lectures)
  • Imbens and Wooldridge (2009) or Angrist and Pischke (2009)
  • Cunningham (2021) (free online resources included)
  • Book of Imbens and Rubin (2015)
  • Abadie and Cattaneo (2018)
  • Huber (2019)
  • Arkhangelsky and Imbens (2024) (recent overviews)
  • Imbens (2022) and Imbens and Xu (2024) (Credibility Revolution)

What is Causality?

• Key questions investigated: smoking and lung cancer, aspirin and heart attacks, unemployment training, schooling, FDI
• Focus is on identifying causal relationships

Example: Training of Unemployed Workers

• Causality: The impact of participation in training on earnings, keeping other earning/training factors constant
• Determinants: Characteristics (education, motivation), predetermined factors (working time reduction, unemployment)
• Fundamental Problem: Observing individuals in one state only (training or not) presents an issue of missing counterfactual data.
• Selection Bias: Replacing missing data with the average earnings of non-trainees can be misleading due to workers with high/low earning capacities.

Random Assignment

• The Statistical Solution: A method for identifying the average causal effect when randomization is possible.
  • Random assignment balances the treatment and control groups.
• Random assignment as "gold standard": Randomization is preferred to avoid bias in identifying/isolating causal effects. Methods of identifying causal effects in cases where randomization isn't possible are required, for example by utilizing "natural experiments." These alternative methods include causal inference methodologies such as (propensity score) matching, re-weighting, regression, (Difference-in-differences (DiD), Regression Discontinuity Design (RDD), and Instrumental Variables (IV).

Limitations and Key Ingredients of Modern Causal Inference

• Multiple methods are needed to detect and exploit hidden forms of randomization when randomization isn't possible
• Non-experimental methods hinge on untestable assumptions.
• Identification of treatment effect is fundamentally non-parametric and requires precise characterization of the affected population.

Overview of Identification Methods

• Methods Based on Unconfoundedness/CIA:
• Difference-in-differences (DID): Comparing outcomes in groups that are not affected by treatment before/after the treatment is introduced.
• Regression Discontinuity Design (RDD): Evaluating treatment effect through comparisons around a treatment threshold, for example, a cutoff
• Instrumental Variables (IV): Identifying causal effects through correlation, using an instrument (Z) correlated with treatment (T) but not with outcome (Y) directly.

Outline of the Course

• Major Topics:
  • The Problem of Causality in Microeconometrics
  • Methods Based on Unconfoundedness
  • Difference-in-Differences (DID), Instrumental Variables (IV), Regression Discontinuity Design (RDD)
  • Randomized Controlled Trials
  • Causal Machine Learning
  • Lecture by a Guest Lecturer covering Randomized Controlled Trials (RCTs)

Outline for the continuation of this lecture

• Focus on the problem of causality
• Formal frameworks to understand causality
• RCTs as a solution
• Causality in a regression framework
• Specification issues with selection
• Critique of OLS Estimation

Sources for the Continuation of this Lecture

• Key sources influencing/used in the lecture discussed in the presented lecture.

A Formal Framework about Causality

• Correlation vs. Causation: Correlation does not imply causation. Confounding factors and reverse causality can confound the relationship.

The Fundamental Problem of Causal Inference

• Definition : Causal effect is the difference in potential outcomes between the treatment and the control group.
• Proposition : The causal effect is logically unobservable without counterfactual evidence

The Statistical Solution

• ATE: Average effect of treatment on the treated.
• ATT: Average treatment effect on the treated

Is Comparison by Treatment Status Informative for ATT?

• Bias: Comparison by treatment status is biased because outcomes of treated and control subjects are not identical in the absense of treatment.
• Solution: The bias caused by the difference of the outcomes for both control and treatment groups in the absence of treatment must be evaluated to accurately calculate the ATT.

RCTs as Solution

• Randomization solves fundamental problems of causal inference.
• Treatment/control groups are statistically equivalent.
• Practical Implementation: Random assignment via lottery, phased implementation, encouragement designs

Some Issues with Implementation of RCTs

• Noncompliance: Not all subjects adhere to the study's planned treatment / control allocation.
• Other Issues: Issues include Hawthorne effects, external validity, spillover effects.

Causality in a regression framework

• Regression equation relating outcome variable Y to treatment variable D.
• Issues with OLS estimation: Selection into treatment, and unobservables that correlate with the treatment or outcome may induce bias, thereby impacting outcomes.
• Law of Iterated Expectations is a key principle for understanding potential issues and suggesting potential solutions

Specification of the selection into treatment

• The participation rule determining individuals' involvement in treatment, which links the participation decision to the outcome(s) of interest.

The model in the compact form

• Compact representation of the causal inference model

The statistical effects of treatment in this model

• Treatment effect (ATE/ATT): the estimated change in the outcome based on the treatment variable
• Idiosyncratic gain from treatment

Problems with OLS Estimation

• Bias in the estimated treatment effect (ATE/ATT) resulting from selection bias.

The Generalized Roy Model

• Model framework: Modeling earnings to isolate the effects of treatment / training.
• Cost of training: Factor that determines the costs or benefits of training, for instance, the costs of attending training; costs dependent on location of event to participation

Alternative reasons for selection bias, for example, participation of individuals in programs for reasons other than economic reasons

Limitations/Critiques of/on the NRH Causal Model

• Potential outcomes are invariant to assignment
• Social or equilibrium interactions are not explicitly accounted

Limitations/Critiques (5)

• Focus on objective outcomes rather than subjective evaluations
• Causal effects of treatment on subjective outcomes/measures of welfare not modelled
• Effects where one event can influence another are not easily captured by the model

Limitations/Critiques (4)

• The causal effect is conceptualised as a black box or opaque process, lacking theoretical explanation
• Historical or previous interventions may not inform / predict future outcomes; limitations associated with external validity, for instance
• External validity limited unless there is a guarantee that interventions are replicated across similar or comparable situations / environments

Limitations/Critiques (3)

• The proposed model only considers realized outcomes; it does not quantify the impact of subjective evaluations of the outcome of interest such as welfare
• Limitations associated with focusing on only objective outcomes rather than considering anticipated outcomes

Identification of F(Y1-Y0) in the (extended) Roy model

• Implications for identifying the proportion of individuals benefiting from treatment in the Roy model; relationships between variables such as Z, Y, and the treatment factor

Some Important Neglected Methods

• Statistical bounds
• Bunching
• Timing of Events
• Designed-Based Identification