ECB3AMT Applied Microeconometric Methods PDF

ECB3AMT Applied Microeconometric Methods 3. Instrumental Variables (IV) Dr. Jacopo Mazza Instrumental Variables (IV) 1. The IV Method 2. IV for incomplete experiments 3. 2SLS 4. Limitations Topic 3 IV 2 Concept u...

ECB3AMT Applied Microeconometric Methods 3. Instrumental Variables (IV) Dr. Jacopo Mazza Instrumental Variables (IV) 1. The IV Method 2. IV for incomplete experiments 3. 2SLS 4. Limitations Topic 3 IV 2 Concept u Legend: Y – Outcome Variable D – Treatment Indicator φ λ Z – Instrumental Z D Y Variable ρ u – unobserved (error) λ – Effect of D on Y φ – Effect of Z on D ρ – Effect of Z (via D) on Y If Z affects Y only via D, then we can use Z to estimate the causal effect of D on Y Topic 3 IV 3 The IV Estimator Idea [Effect of Z on Y] = [Effect of Z on D] * [Effect of D on Y] or so that This is often called the Wald estimator Topic 3 IV 4 Example: KIPP KIPP: Knowledge is Power Programme Association of schools in the US Emphasis on discipline and comportment, long school days, extended school year, selective teacher hiring, focus traditional reading and math skills High share of minorities, key role in debate on educational reform Markedly higher test sores for non-white students in KIPP schools Key question: Does the KIPP concept improve learning outcomes of non- white students? Topic 3 Or do better non-white students IV select into KIPP schools? 5 Bias when comparing conditional means Recall: Difference in conditional means = average causal effect + selection bias average causal effect of KIPP on students test scores selection bias of better (or worse) students into KIPP schools Topic 3 IV 6 KIPP Lottery Large growth of number of applicants forced KIPP Lynn to implement a lottery (by law) Actual enrollment is 𝑍 𝑖 =1 𝑍 𝑖 =0 endogenous: Not all applicants actually enroll Some who lose the lottery find a way in However, seat offerings are Topic 3 random IV 7  𝐸 [ 𝑌 𝑖| 𝑍 𝑖 =1 ] − 𝐸 [ 𝑌 𝑖|𝑍 𝑖=0 ] Balance Check and Outcomes 𝐸 [ 𝐷 𝑖|𝑍 𝑖=1 ] − 𝐸 [ 𝐷𝑖| 𝑍 𝑖 =0 ] Topic 3 IV 8 IV Estimate of the KIPP Effect 𝐸 [ 𝑌 𝑖| 𝑍 𝑖 =1 ] − 𝐸 [ 𝑌 𝑖|𝑍 𝑖=0 ] ¿ 𝜆 𝐸 [ 𝐷 𝑖|𝑍 𝑖=1 ] − 𝐸 [ 𝐷𝑖| 𝑍 𝑖 =0 ] Topic 3 IV 9 Assumptions Needed for Internal Validity Identifying Assumptions 1. Relevance: The instrument correlates with the endogenous variable, We can (and should!) test this assumption. 2. Exogeneity / Exclusion Restriction: The instrument does not directly affect the outcome variable We cannot test this assumption. 3. Independence: The IV is (as good as) randomly assigned. i.e. the IV does not affect omitted variables that matter for the outcome. Topic 3 IV 10 Compliance and External Validity External validity depends on who complies with the treatment 1. Compliers: Change their treatment status in line with the IV 2. Always-takers: Always pick the treatment irrespective of the IV. 3. Never-takers: Never pick the treatment, irrespective of the IV. 4. Defiers: Change their treatment status contrary to the IV. One typically assumes that this group does not exist. This assumption is called Monotonicity or No-Defiers Assumption. Topic 3 IV 11 Compliance Topic 3 IV 12 Local Average Treatment Effect (LATE) LATE identifies the average causal effect on compliers i.e. for those who change their treatment status in line with the IV Similar to encouragement designs (see randomization) LATE is not informative for treatment effect on Never-takers and Always-takers. Or: IV is only informative for those who respond to the IV In our example: Defier: unlikely in this lottery Always-taker: attend irrespective of winning the lottery Never-taker: do not attend irrespective of lottery result  Complier-Population likely is large Topic 3 IV 13 Comparison to other treatment effects Average Treatment Effect (ATE) = average of the treatment effects on Never-takers, Always-takers and Compliers. Typically lower, because the effect is usually identified solely via the compliers and the other effects are often zero. Average Treatment Effect on the Treated (ATT) = average of the treatment effects on Always-takers and Compliers. If there are no always takers: LATE = ATT Average Treatment Effect on the Untreated / Control (ATC) = average of the treatment effects on Never-takers and Compliers. If there are no never-takers: LATE = ATC. Topic 3 IV 14 Instrumental Variables (IV) 1. The IV Method 2. IV for incomplete experiments 3. 2SLS 4. Limitations Topic 3 IV 15 Basic Idea Extend the logic: Allocation to treatment in an experiment is random But actual treatment is not random due to incomplete compliance Comparing outcomes between treated and control: ITT But: estimate LATE using IV Topic 3 IV 16 Example: MDVE MDVE: Minneapolis Domestic Violence Experiment Background: unclear how to best deal with batterers Random assignment of treatment status in case of domestic violence (if specific conditions are met) Arrest Advice Separate But: unrealistic to assume that officers would always stick to the experimental protocol due to specific situation on the premise Non-random deviation of delivered treatment from the (randomly) assigned treatment Topic 3 IV 17  Assigned vs. Delivered Treatment First Stage: Recidivism Rate (reocurrence) 𝐷𝑖 =0 𝐷𝑖 =1 Reduced Form: 𝑍 𝑖 =0 Þ Reduced Form is ITT 𝑍 𝑖 =1 LATE: Bia Comparing conditional means: s Almost no always takers (1/92) => LATE ≈ ATT Topic 3 IV 18 Summary Assume that you have an experiment, where Allocation to treatment is random But actual treatment is not random due to incomplete compliance Then: Comparing outcomes between treated and control gives you the ITT But you can estimate the LATE using the IV Method Not discussed so far: how to get Standard Errors? => 2SLS Topic 3 IV 19 Instrumental Variables (IV) 1. The IV Method 2. IV for incomplete experiments 3. 2SLS 4. Limitations Topic 3 IV 20 Extending the IV Method The Two-Stage Least Squares (2SLS) Estimator extends the IV Method: Multiple IVs Control Variables Multiple endogenous variables / treatments Continuous endogenous variables / treatments Continuous instrumental variables Topic 3 IV 21 Repetition When we have one endogenous variable, one IV and no controls, then: Reduced Form: comparing means: as a regression: First Stage: Comparing means: as a regression: Second Stage: predict: regression: On can show that in this case: Topic 3 IV 22 Extension We now can easily add control variables, more IVs, and more endogenous variables First Stage: Second Stage: Reduced Form: Attention: Control Variables X must appear in all equations, 1st & 2nd stage, and reduced form IVs appear only in the first stage (exclusion restriction) Number of IVs must be at least as large as the number of endogenous variables Standard errors for must be corrected for the fact that it is a two-step approach (the built-in 2SLS-commands in statistical software do that) Topic 3 IV 23 Example: Family size and children’s education Do larger families invest less in children’s education? E.g. one child policy in China, family planning programs in Mexico and Indonesia, … Endogeneity: family size is endogenous Family size depends on parents’ education, but parents’ education matters for children's education Potentially large unobserved differences RCT infeasible (ethically and practically) Topic 3 IV 24 Candidates for IVs “as good as” random variation (assumption 3): Twins (but: differences in probability of twins by age of mother and ethnicity) Children’s gender Relevance (assumption 1) Variation in family size due to twins => unexpected increase in family size Variation in family size due to children’s gender => wish for a son (particularly in Asia), or wish for diversified sibling-sex portfolio (Europe, America) Exogeneity / Exclusion restriction (assumption 2) Key question: how could IV affect children‘s education other than via family size? For always-takers and never-takes, the first stage is zero,=0 This imples that for these individuals, (reduced form) i.e. test if the reduced form produces a zero effect on the IV for individuals whom we believe to belong to the always-takers or never-takers group Topic 3 IV 25  𝐷𝑖 =𝛼 1 +𝛼1 𝑋 𝑖 + 𝜙1 𝑍 1 𝑖 + 𝜙2 𝑍 2 𝑖 + 𝑒1 𝑖 First Stage Outcome variable: family size If the effect of the IV changes when adding the control variables, this indicates that the IV is not fully random. Note: the absence of such changes does not imply absence of biases, since we cannot test whether IVs are correlated to unobservables. Topic 3 IV 26  𝑌 𝑖=𝛼2 +𝛼 2 𝑋 𝑖 + 𝜆2 𝑆𝐿𝑆 ^ 𝐷𝑖 + 𝑒2 𝑖 Second Stage OLS-Effect < 2SLS-Effects => downward bias Effects sizes differ across IVs: - IVs capture different variation - Noise - Or exogeneity assumption is violated Imprecise IV estimates: - Either weak first stage (need to test!) - Or no effect of family size on outcomes Topic 3 IV 27 Instrumental Variables (IV) 1. The IV Method 2. IV for incomplete experiments 3. 2SLS 4. Limitations Topic 3 IV 28 Internal Validity Internal validity requires the identifying assumptions to hold 1. Relevance: The instrument correlates with the endogenous Use an F-test to check this (next slide) 2. Exogeneity / Exclusion Restriction: The instrument does not directly affect the outcome variable We cannot test this assumption. Requires careful discussion 3. Independence: The IV is (as good as) randomly assigned. i.e. the IV does not affect omitted variables that matter for the Topic 3 IV 29 outcome Weak Instruments Weak Instruments If an IV does not predict the treatment well, assumption 1 (relevance) is threatened Weak IVs lead to biased estimates; the bias can be large! Intuition: , if , small errors have large effects Test for weak IVs by an F-Test of excluded instruments Regress F-Test on (all instruments have a zero coefficient) Rule of thumb: if F last chapter of this course) Good candidates: Institutional changes, reforms, law changes, etc. Natural variation Historical features Another good candidate are actual experiments with incomplete treatment Use assignment to treatment as an IV for actual treatment (see previous chapter) Topic 3 IV 33 Bartik IVs - Concept Example: estimate the (inverse) wage elasticity of labor supply Local employment growth can be rewritten as Bartik IV: use only national industry growth rates to construct IV for local employment growth Topic 3 IV 34 Bartik IVs - Assumptions National shifts (conditional on controls) can be considered exogenous => e.g. small region or leave out each region‘s contribution to national shifts Shares (conditional on controls) can be considered exogenous => In most settings, identification stems from the shares => Hard to argue, lots of developmens on this on the last years, see Goldsmith-Pinkham et al. (2020, AER 110(9): 2586- 2624) Topic 3 IV 35 Bad Controls Leaving out important variables (confounders) leads to omitted variable bias But: including bad control variables introduces bias Example: Assume estiamte the effect of graduating from university on wages Should you control for occupations? No, because occupation is part of the mechanism by which graduating from university generates higher wages Topic 3 IV 36 How to Identify Bad Controls? In most cases advisible to not control for things that happen after the treatment Previous example: occupation is chosen after graduation But: there are many examples of bad controls, need to know the situation Recommendation: check how adding controls affects the estimates and try to understand why that happens Overview: http://causality.cs.ucla.edu/blog/index.php/category/bad -control/ Topic 3 IV 37 Test for Endogeneity (Durbin–Wu–)Hausman Test Test if an explanatory variable is endogenous Target Regression Auxiliary Regression with an IV Estimate If , then is endogenous Problem: you need an instrumental variable to run the test Topic 3 IV 38 Test for Overidentifying Restrictions If # of instruments = # of endogenous variables => „just identified“ If # of instruments > # of endogenous variables => „overidentified“ Sargan-Hansen Test (J-Test) tests whether the model is „overidentified“, meaning in our case that the excess instruments are bad Often reported by statistical software when you have excess instruments Problem You need more than one instrument You need to assume that at least one instrument is exogenous You cannot test which one is endogenous Topic 3 IV 39 Can We Test for Endogeneity? Previously I said: „No“ The more precise answer is: We can only test for endogeneity if we have an instrumental variable that we can assume to be exogenous. i.e. we need strong assumptions for such a test Topic 3 IV 40 How to read an IV study What is the research question? What is the identification problem? What is exclusion restriction? The authors should carefully discuss whether this assumption is plausible. What does the reduced form look like? „if you can‘t see it in the reduced form, it ain‘t there“ (Angrist/Pischke) What does the first stage look like? Is the first stage reasonable? Do the authors test for weak instruments? How does the IV differ from the OLS? Is the difference plausible? Discussion of external validity How large is the group of compliers relative to the group of always-takers? Topic 3 IV 41 Instrumental Variables (IV) 1. The IV Method 2. IV for incomplete experiments 3. 2SLS 4. Limitations Next Topic: Regression Discontinuity (RD) Designs Topic 3 IV 42