Lecture 5: Research Methods Applied Empirical Economics PDF

Research Methods: Applied Empirical Economics Lecture 5: Differences-in-differences Egbert Jongen | Leiden University September 2024 Discover the world at Leiden University 1 Lecture goals Understand why we need to take a double difference in differences-in-differ...

Research Methods: Applied Empirical Economics Lecture 5: Differences-in-differences Egbert Jongen | Leiden University September 2024 Discover the world at Leiden University 1 Lecture goals Understand why we need to take a double difference in differences-in-differences (DD) Know the basic DD equation Know the key assumptions of DD Know how to test these assumptions Discover the world at Leiden University 2 Outline of the lecture An example Notation Break Key assumptions Key test Another example Questions or comments Discover the world at Leiden University 3 Differences-in-differences Differences-in-differences - Most popular - But not necessarily the most desirable Basically - Take difference treatment and control after - Subtract difference treatment and control before - Hopefully this makes ceteris paribus Again start with an example and introduce notation afterwards Discover the world at Leiden University 4 Video Differences-in-differences Discover the world at Leiden University 5 The natural experiment When a bank run starts, should the Federal Reserve open the flow of credit or turn off the tap? Study the Great Depression (1930s) Look at number of banks in operation Crisis hit banks in 1931 Atlanta Fed=Sixth district: open the flow of credit St. Louis Fed=Eighth district: turn off the tap Sixth district is treatment, Eighth district is control What happened? Discover the world at Leiden University 6 Taking a double difference Number of banks in Sixth district after: 121 Number of banks in Eighth district after: 132 Difference = 121-132 = -11 Discover the world at Leiden University 7 Taking a double difference Number of banks in Sixth district after: 121 Number of banks in Eighth district after: 132 Difference = 121-132 = -11 Number of banks in Sixth district before: 135 Number of banks in Eighth district before: 165 Difference = 135-165 = -30 Discover the world at Leiden University 8 Taking a double difference Number of banks in Sixth district after: 121 Number of banks in Eighth district after: 132 Difference = 121-132 = -11 Number of banks in Sixth district before: 135 Number of banks in Eighth district before: 165 Difference = 135-165 = -30 Difference-in-difference = -11 – (-30) = +19 So opening the flow of credit leads to more surviving banks! Discover the world at Leiden University 9 Taking a double difference Number of banks in Sixth district after: 121 Number of banks in Eighth district after: 132 Difference = 121-132 = -11 Number of banks in Sixth district before: 135 Number of banks in Eighth district before: 165 Difference = 135-165 = -30 Difference-in-difference = -11 – (-30) = +19 So opening the flow of credit leads to more surviving banks! Note: a single difference for the treatment group gives you: 121-135=-14! But this does not control for the general crisis effect on banks Discover the world at Leiden University 10 Graphical analysis Discover the world at Leiden University 11  Graphical analysis Group effect Discover the world at Leiden University 12  Graphical analysis Group Time effect effect Discover the world at Leiden University 13  Graphical analysis Group effect Time effect Discover the world at Leiden University 14  Graphical analysis Group effect Time effect Discover the world at Leiden University 15 Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt Discover the world at Leiden University 16 Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt d indicates the group: Sixth or Eighth district t indicates the time period: before or after the reform Discover the world at Leiden University 17 Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt Constant: outcome for the control group in the pre-reform period Discover the world at Leiden University 18  Ydt = α + β Treatd + γPostt Graphical analysis + δ (Treatd*Postt ) + εdt α Treat=0 Post=0 Discover the world at Leiden University 19  Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt Group dummy: 1 for the treatment group, 0 for the control group (both before and after the reform, groups do not change) Discover the world at Leiden University 20  Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt Group effect: difference in outcome for the treatment and control group in the pre-reform period Discover the world at Leiden University 21  Ydt = α + β Treatd + γPostt Group effect + δ (Treatd*Postt ) + εdt Treat=1 Post=0 Group effect β Discover the world at Leiden University 22  Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt Time dummy: 0 before the reform, 1 after reform Discover the world at Leiden University 23  Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt Time effect: common difference in outcome for treatment and control group in the post-reform period Discover the world at Leiden University 24  Ydt = α + β Treatd + γPostt Time effect + δ (Treatd*Postt ) + εdt Treat=0 Post=1 Time effect γ Discover the world at Leiden University 25  Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt Treatment dummy: 0 before the reform, 1 after reform for the treatment group Discover the world at Leiden University 26  Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt Treatment effect: difference in outcome for treatment and control group in the post-reform period – the difference in outcome for treatment and control group in the pre-reform period Discover the world at Leiden University 27  Ydt = α + β Treatd + γPostt Treatment effect + δ (Treatd*Postt ) + εdt Treat=1 Post=1 δ Discover the world at Leiden University 28 Basic DD regression equation Ydt = α + β Treatd + γPostt + δ (Treatd*Postt ) + εdt Discover the world at Leiden University 29  Extended DD regression equation Ydt = α + β Treatd + ∑sγsPosts + δ (Treatd*Postt ) + εdt Multiple time dummies: 0 in all periods except t=s for which it is 1 Capture individual year effects Discover the world at Leiden University 30  Extended DD regression equation Ydt = α + β Treatd + ∑sγsPosts + ∑u δu(Treatd*Postu) + εdt Multiple treatment dummies: 0 in all periods except t=u for which it is 1 Capture e.g. build up of treatment effect Discover the world at Leiden University 31  Extended DD regression equation Ydt = α + β Treatd + ∑sγsPosts + ∑u δu(Treatd*Postu) + ηtreat(Treatd*t) + ηcontrol(Controld*t) + εdt Differential trends for treatment and control group (Controld is 1 for control group and 0 for treat. group) Discover the world at Leiden University 32  Extended DD regression equation Yidt = α + β Treatd + ∑sγsPosts + ∑u δu(Treatd*Postu) + ηtreat(Treatd*t) + ηcontrol(Controld*t) + πXi + εidt Other control variables (e.g. gender, education etc.) Discover the world at Leiden University 33 Break! Discover the world at Leiden University 34 Key assumptions of DD 1. Common time effects: In the absence of the reform both groups would have evolved parallel 2. Exogeneity of the reform: Reform not related to different development treatment and control group 3. No anticipation of the reform: Treatment group did not respond before the reform Discover the world at Leiden University 35 Again: Heller – Catch 22 Just because you’re paranoid doesn’t mean they are not after you! Discover the world at Leiden University 36 A test πώς τσεκάρουμε τις υποθέσεις της μεθόδου Common method to test these assumptions: Estimate placebo effect for pre-reform dummies If the placebo is statistically significant: Assumptions may be violated! Discover the world at Leiden University 37  Basic DD regression with placebo θα μπορούσε να είναι ένα χρόνο πριν το Ydt = α + β Treatd ριφορμ, ή όχι — νταμι + γpPlacebot + γtPostt + δp(Treatd*Placebot) + δ (Treatd*Postt ) + εdt - Need at least two pre-reform periods - Allow for common time effect in placebo period - Add placebo treatment effect in placebo period Discover the world at Leiden University 38 An example from the Netherlands Leon Bettendorf, Egbert Jongen and Paul Muller (2015) “Childcare subsidies and labour supply - Evidence from a large Dutch reform” Labour Economics, 36, pp. 112-132 Discover the world at Leiden University 39 The natural experiment Massive increase in childcare subsidies over the period 2005-2009 Data period 1995-2009 -Pre-reform: 1995-2004 -Post-reform: 2005-2009 Differences-in-differences -Treatment group: parents with child < 12 years of age -Control group: parents with child 12-17 years of age Discover the world at Leiden University 40 Participation rate women Discover the world at Leiden University 41 Regression results women Discover the world at Leiden University 42 Participation rate men Discover the world at Leiden University 43 Regression results men Discover the world at Leiden University 44 Questions or comments? Discover the world at Leiden University 45 Coming Thursday there is a Q&A for the Mid Term Please send any questions you may have before to: [email protected] Discover the world at Leiden University 46  See you on Thursday! Discover the world at Leiden University 47