TY - JOUR
AU - Bertrand,Marianne
AU - Duflo,Esther
AU - Mullainathan,Sendhil
TI - How Much Should We Trust Differences-in-Differences Estimates?
JF - National Bureau of Economic Research Working Paper Series
VL - No. 8841
PY - 2002
Y2 - March 2002
DO - 10.3386/w8841
UR - http://www.nber.org/papers/w8841
L1 - http://www.nber.org/papers/w8841.pdf
N1 - Author contact info:
Marianne Bertrand
Booth School of Business
University of Chicago
5807 South Woodlawn Avenue
Chicago, IL 60637
Tel: 773/834-5943
Fax: 773/702-0458
E-Mail: marianne.bertrand@chicagobooth.edu
Esther Duflo
Department of Economics, E17-201B
MIT
77 Massachusetts Avenue
Cambridge, MA 02139
Tel: 617/258-7013
Fax: 617/253-6915
E-Mail: eduflo@mit.edu
Sendhil Mullainathan
Department of Economics
Littauer M-18
Harvard University
Cambridge, MA 02138
Tel: 617/496-2720
Fax: 617/495-7730
E-Mail: mullain@fas.harvard.edu
AB - Most Difference-in-Difference (DD) papers rely on many years of data and focus on serially correlated outcomes. Yet almost all these papers ignore the bias in the estimated standard errors that serial correlation introduce4s. This is especially troubling because the independent variable of interest in DD estimation (e.g., the passage of law) is itself very serially correlated, which will exacerbate the bias in standard errors. To illustrate the severity of this issue, we randomly generate placebo laws in state-level data on female wages from the Current Population Survey. For each law, we use OLS to compute the DD estimate of its 'effect' as well as the standard error for this estimate. The standard errors are severely biased: with about 20 years of data, DD estimation finds an 'effect' significant at the 5% level of up to 45% of the placebo laws. Two very simple techniques can solve this problem for large sample sizes. The first technique consists in collapsing the data and ignoring the time-series variation altogether; the second technique is to estimate standard errors while allowing for an arbitrary covariance structure between time periods. We also suggest a third technique, based on randomization inference testing methods, which works well irrespective of sample size. This technique uses the empirical distribution of estimated effects for placebo laws to form the test distribution.
ER -