Causal Inference in RCTs

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What does ATE stand for in the context of causal inference?

Actual Treatment Estimate

Adjusted Treatment Effect

Average Treatment Effect (correct)

Average Total Effect

What does the letter B represent in the provided equation?

Base effect of training

Benefit of treatment over control

Selection bias between groups (correct)

Bias related to treatment differences

Which implementation design involves a control group receiving training later?

Encouragement design

Random assignment design

Phase in design (correct)

Direct control design

What is a significant issue with the noncompliance in RCTs?

Some control group members participate in the treatment Signup and view all the answers

What does the term LATE refer to in the context of RCTs?

Local Average Treatment Effect Signup and view all the answers

Which of the following is NOT an implementation design for RCTs mentioned?

Sequential design Signup and view all the answers

What type of effect is achieved when the number treated in the treatment group exceeds those in the control group?

Intention-to-treat effect Signup and view all the answers

Which of these is NOT one of the issues associated with the implementation of RCTs?

Confounding variables Signup and view all the answers

What does 'externally changing training status' refer to in the context of causal effects?

Changing a worker's training while keeping other factors constant. Signup and view all the answers

What is the fundamental problem of causal inference highlighted in the content?

Observing individuals only in one training state. Signup and view all the answers

What does ATT stand for in the context of treatment effects?

Average Treatment Effect for Treated Signup and view all the answers

Why is selection bias a critical issue in estimating causal effects?

Not all individuals have the same likelihood of entering training. Signup and view all the answers

How can the average earnings of non-trainees introduce bias?

It ignores the fact that workers with varying capacities might be excluded from training. Signup and view all the answers

What implication can be drawn from the statement 'ATT ≠ ATE'?

The average effect of all individuals differs from the average effect of only those trained. Signup and view all the answers

Why should one avoid controlling for factors determined after training assignment?

They may be directly influenced by the training itself. Signup and view all the answers

What is meant by 'heterogeneous treatment effects' in the content?

Different individuals experience varied impacts from training. Signup and view all the answers

What is the purpose of the Law of Iterated Expectations in causal inference?

To illustrate how smaller information sets dominate larger ones. Signup and view all the answers

In the regression equation $Y_i = \beta + \alpha D_i + \epsilon_i$, what does $\hat{\alpha}_{OLS}$ represent?

The mean difference between treated and untreated groups. Signup and view all the answers

What is a significant challenge with OLS estimation in causal inference?

It might not account for omitted variable bias. Signup and view all the answers

Which statement best describes the relationship between covariance and the OLS coefficient in the given regression?

The OLS coefficient equals the covariance divided by the variance of the treatment variable. Signup and view all the answers

Which aspect is emphasized by the specification of the selection into treatment?

It addresses how individuals are assigned to treatment groups. Signup and view all the answers

What is the implication of $E(D_i) = P(D_i)$ in the context of causal inference?

It indicates the probability of being in the treatment group. Signup and view all the answers

What advantage does the Law of Iterated Expectations provide in statistical analysis?

It allows for both conditional and unconditional expectations. Signup and view all the answers

How does the term $E(Y_i|D_i = 1) - E(Y_i|D_i = 0)$ relate to the OLS estimation?

It equates to the OLS coefficient under specific conditions. Signup and view all the answers

What is the implication of a zero covariance in the extended Roy model?

The proportion of individuals benefiting from treatment can be identified. Signup and view all the answers

Which of the following statements about quantile treatment effects is true?

They are relevant in symmetric social welfare functions. Signup and view all the answers

Which method can identify λ(zγ) in the identification process of the Roy model?

Probit on participation. Signup and view all the answers

What do the residuals of the regression identify in the identification of the Roy model?

Variances of unobserved heterogeneity. Signup and view all the answers

In the context of quantile treatment effects, when do the effects satisfy the equality Q(Y1) − Q(Y0) = Q(Y1 − Y0)?

In cases with perfect rank correlation. Signup and view all the answers

What does the identification process in the extended Roy model help to uncover?

The causal relationship between treatment and outcome. Signup and view all the answers

What is a limitation of quantile treatment effects concerning causal inference?

They may misrepresent individual welfare impact. Signup and view all the answers

Which statistic is necessary to ascertain Cov(U0, U1) in the extended Roy model?

Covariance related to treatment effects. Signup and view all the answers

What is the primary issue that the generalized Roy model addresses regarding training selection?

The correlation between individual characteristics and training outcomes Signup and view all the answers

In the generalized Roy model, what does the variable R represent?

The net benefit of training Signup and view all the answers

Which assumption is crucial for the correctness of a linear specification in the generalized Roy model?

Z is independent of the errors in the model Signup and view all the answers

What does the notation Φ represent in the probability equation provided in the generalized Roy model?

The cumulative distribution function of a normal random variable Signup and view all the answers

How is the cost of training defined in the generalized Roy model?

Based on distance to the training center and subsidies Signup and view all the answers

What does the variable ν represent in the equation R = Zγ + ν?

The cumulative effect of unobservable factors on net benefits Signup and view all the answers

What does the term 'self-selection' imply in the context of training programs?

Individuals decide to pursue training based on perceived benefits Signup and view all the answers

Which of the following is a critical component of the net benefit R computation in the generalized Roy model?

The difference between potential earnings before and after training Signup and view all the answers

What does the equation E(U1 − U0 |D = 1) represent in the context of selection bias?

The average treatment effect on the treated individuals when selection is correlated with unobserved determinants Signup and view all the answers

In the Roy model, what is implied if individuals with high returns tend to self-select into training?

It demonstrates a positive correlation between ability and investment in human capital. Signup and view all the answers

According to the information, what is a consequence of voluntary participation in treatment?

Selection based on expected gains is likely to occur. Signup and view all the answers

What is the effect of OLS estimators in the context of selection bias as suggested in the content?

They overestimate both the ATE and ATT unless significant covariates exist. Signup and view all the answers

What does the note regarding RCTs among voluntary candidates indicate?

They facilitate determining the treatment effect as long as compliance is full. Signup and view all the answers

What limitation is mentioned about RCTs conducted on a full population?

They cannot identify average treatment effects without enforced participation. Signup and view all the answers

How might employment agencies influence selection bias according to the discussion?

Through biased financing based on placement rates. Signup and view all the answers

What does the concept of 'positive discrimination' lead to according to the content?

Negative selection among treatment candidates. Signup and view all the answers

Study Notes

Causal Inference in Microeconometrics with Applications to Program Evaluation

Lecture 1: Introduction by Bart Cockx, Ghent University, 2024
Course Structure
- 11 three-hour sessions on Fridays from October 4th to December 20th, 2024 (excluding November 1st)
- Two two-hour sessions per session, each 50 minutes long, with 15-minute breaks and a 1-hour lunch break
- Low quality recordings available via MS Teams
- Guest lecturer (William Pariente) covering Randomized Controlled Trials (RCTs) on November 29th, December 6th, and 13th, addressing a particular topic

Evaluation Method

Attendance is mandatory.
- -1 point for each unjustified absence.
- -2 points for any absence from the 3rd absence onwards.
Assignments (10 points)
- Groups of 2–3 students
- Registration on Ufora
- Model solutions posted on the platform
- Support from Natalia Bermudez and Giulia Tarullo
Presentations (10 points)
- Group presentations on research or lecture-related topics
- Maximum of 3 students per group
- Options for selected papers or self-selected research
- Aim: explaining lecture materials in own words (re-explaining lectures in own words)
- Time: 15 min presentation + 5 mins discussion
- Deadline: end of January (date to be confirmed)

Teaching Material

All lecture materials, including slides, recordings, and documents, are available on Ufora.
Non-Ghent University Students need official registration on OASIS to access and take the exam.
Required Reading: Blundell and Costa Dias (2009)
General Complementary Readings: (specific readings are described in individual lectures)
- Imbens and Wooldridge (2009) or Angrist and Pischke (2009)
- Cunningham (2021) (free online resources included)
- Book of Imbens and Rubin (2015)
- Abadie and Cattaneo (2018)
- Huber (2019)
- Arkhangelsky and Imbens (2024) (recent overviews)
- Imbens (2022) and Imbens and Xu (2024) (Credibility Revolution)

What is Causality?

Key questions investigated: smoking and lung cancer, aspirin and heart attacks, unemployment training, schooling, FDI
Focus is on identifying causal relationships

Example: Training of Unemployed Workers

Causality: The impact of participation in training on earnings, keeping other earning/training factors constant
Determinants: Characteristics (education, motivation), predetermined factors (working time reduction, unemployment)
Fundamental Problem: Observing individuals in one state only (training or not) presents an issue of missing counterfactual data.
Selection Bias: Replacing missing data with the average earnings of non-trainees can be misleading due to workers with high/low earning capacities.

Random Assignment

The Statistical Solution: A method for identifying the average causal effect when randomization is possible.
- Random assignment balances the treatment and control groups.
Random assignment as "gold standard": Randomization is preferred to avoid bias in identifying/isolating causal effects. Methods of identifying causal effects in cases where randomization isn't possible are required, for example by utilizing "natural experiments." These alternative methods include causal inference methodologies such as (propensity score) matching, re-weighting, regression, (Difference-in-differences (DiD), Regression Discontinuity Design (RDD), and Instrumental Variables (IV).

Limitations and Key Ingredients of Modern Causal Inference

Multiple methods are needed to detect and exploit hidden forms of randomization when randomization isn't possible
Non-experimental methods hinge on untestable assumptions.
Identification of treatment effect is fundamentally non-parametric and requires precise characterization of the affected population.

Overview of Identification Methods

Methods Based on Unconfoundedness/CIA:
Difference-in-differences (DID): Comparing outcomes in groups that are not affected by treatment before/after the treatment is introduced.
Regression Discontinuity Design (RDD): Evaluating treatment effect through comparisons around a treatment threshold, for example, a cutoff
Instrumental Variables (IV): Identifying causal effects through correlation, using an instrument (Z) correlated with treatment (T) but not with outcome (Y) directly.

Outline of the Course

Major Topics:
- The Problem of Causality in Microeconometrics
- Methods Based on Unconfoundedness
- Difference-in-Differences (DID), Instrumental Variables (IV), Regression Discontinuity Design (RDD)
- Randomized Controlled Trials
- Causal Machine Learning
- Lecture by a Guest Lecturer covering Randomized Controlled Trials (RCTs)

Outline for the continuation of this lecture

Focus on the problem of causality
Formal frameworks to understand causality
RCTs as a solution
Causality in a regression framework
Specification issues with selection
Critique of OLS Estimation

Sources for the Continuation of this Lecture

Key sources influencing/used in the lecture discussed in the presented lecture.

A Formal Framework about Causality

Correlation vs. Causation: Correlation does not imply causation. Confounding factors and reverse causality can confound the relationship.

The Fundamental Problem of Causal Inference

Definition : Causal effect is the difference in potential outcomes between the treatment and the control group.
Proposition : The causal effect is logically unobservable without counterfactual evidence

The Statistical Solution

ATE: Average effect of treatment on the treated.
ATT: Average treatment effect on the treated

Is Comparison by Treatment Status Informative for ATT?

Bias: Comparison by treatment status is biased because outcomes of treated and control subjects are not identical in the absense of treatment.
Solution: The bias caused by the difference of the outcomes for both control and treatment groups in the absence of treatment must be evaluated to accurately calculate the ATT.

RCTs as Solution

Randomization solves fundamental problems of causal inference.
Treatment/control groups are statistically equivalent.
Practical Implementation: Random assignment via lottery, phased implementation, encouragement designs

Some Issues with Implementation of RCTs

Noncompliance: Not all subjects adhere to the study's planned treatment / control allocation.
Other Issues: Issues include Hawthorne effects, external validity, spillover effects.

Causality in a regression framework

Regression equation relating outcome variable Y to treatment variable D.
Issues with OLS estimation: Selection into treatment, and unobservables that correlate with the treatment or outcome may induce bias, thereby impacting outcomes.
Law of Iterated Expectations is a key principle for understanding potential issues and suggesting potential solutions

Specification of the selection into treatment

The participation rule determining individuals' involvement in treatment, which links the participation decision to the outcome(s) of interest.

The model in the compact form

Compact representation of the causal inference model

The statistical effects of treatment in this model

Treatment effect (ATE/ATT): the estimated change in the outcome based on the treatment variable
Idiosyncratic gain from treatment

Problems with OLS Estimation

Bias in the estimated treatment effect (ATE/ATT) resulting from selection bias.

The Generalized Roy Model

Model framework: Modeling earnings to isolate the effects of treatment / training.
Cost of training: Factor that determines the costs or benefits of training, for instance, the costs of attending training; costs dependent on location of event to participation

Alternative reasons for selection bias, for example, participation of individuals in programs for reasons other than economic reasons

Limitations/Critiques of/on the NRH Causal Model

Potential outcomes are invariant to assignment
Social or equilibrium interactions are not explicitly accounted

Limitations/Critiques (5)

Focus on objective outcomes rather than subjective evaluations
Causal effects of treatment on subjective outcomes/measures of welfare not modelled
Effects where one event can influence another are not easily captured by the model

Limitations/Critiques (4)

The causal effect is conceptualised as a black box or opaque process, lacking theoretical explanation
Historical or previous interventions may not inform / predict future outcomes; limitations associated with external validity, for instance
External validity limited unless there is a guarantee that interventions are replicated across similar or comparable situations / environments

Limitations/Critiques (3)

The proposed model only considers realized outcomes; it does not quantify the impact of subjective evaluations of the outcome of interest such as welfare
Limitations associated with focusing on only objective outcomes rather than considering anticipated outcomes

Identification of F(Y1-Y0) in the (extended) Roy model

Implications for identifying the proportion of individuals benefiting from treatment in the Roy model; relationships between variables such as Z, Y, and the treatment factor

Some Important Neglected Methods

Statistical bounds
Bunching
Timing of Events
Designed-Based Identification

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

Test your knowledge of causal inference concepts related to randomized controlled trials (RCTs). This quiz covers key terms and issues like ATE, LATE, implementation designs, and selection bias. Understand the nuances of treatment effects and the challenges faced in estimating causal effects.