On the Generalizability of Experimental Results in Economics: With A Response To Camerer
Economists are increasingly turning to the experimental method as a means to estimate causal effects. By using randomization to identify key treatment effects, theories previously viewed as untestable are now scrutinized, efficacy of public policies are now more easily verified, and stakeholders can swiftly add empirical evidence to aid their decision-making. This study provides an overview of experimental methods in economics, with a special focus on developing an economic theory of generalizability. Given that field experiments are in their infancy, our secondary focus pertains to a discussion of the various parameters that they identify, and how they add to scientific knowledge. We conclude that until we conduct more field experiments that build a bridge between the lab and the naturally-occurring settings of interest we cannot begin to make strong conclusions empirically on the crucial question of generalizability from the lab to the field.
We wish to thank Colin Camerer, Marco Castillo, Robert Chambers, David Eil and Andreas Ortmann for helpful comments and discussions. Alec Brandon and David Novgorodsky provided excellent research assistance. Al-Ubaydli: Bahrain Center for Strategic, International and Energy Studies and Department of Economics and Mercatus Center, George Mason University; List: Department of Economics, University of Chicago & NBER. This study extends NBER w17957 "On the generalizability of experimental results in economics," by including a discussion of studies that have commented on our past work. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.