From Local to Global: External Validity in a Fertility Natural Experiment
We study issues related to external validity for treatment effects using over 100 replications of the Angrist and Evans (1998) natural experiment on the effects of sibling sex composition on fertility and labor supply. The replications are based on census data from around the world going back to 1960. We decompose sources of error in predicting treatment effects in external contexts in terms of macro and micro sources of variation. In our empirical setting, we find that macro covariates dominate over micro covariates for reducing errors in predicting treatments, an issue that past studies of external validity have been unable to evaluate. We develop methods for two applications to evidence-based decision-making, including determining where to locate an experiment and whether policy-makers should commission new experiments or rely on an existing evidence base for making a policy decision.
The authors thank Morris Chow for excellent research assistance; Ali T. Ahmed, Hunt Allcott, Joshua Angrist, Peter Aronow, Neal Beck, James Bisbee, Gary Chamberlain, Drew Dimmery, Michael Gechter, Rachel Glennerster, and Raimundo Undurraga for valuable comments and suggestions; and seminar participants at the BREAD conference, Cowles Econometrics Seminar, EGAP, the Federal Reserve Board of Cleveland, the Federal Reserve Board of New York, Georgetown, GREQAM, IZA, Maastricht, NEUDC 2014, Columbia, Harvard, MIT, NYU, UCLA, UCSD, the World Bank, Yale, the 2014 Stata Texas Empirical Microeconomics Conference, and the Stanford 2015 SITE conference for helpful feedback. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
forthcoming in the Journal of Business & Economic Statistics. citation courtesy of