A Theory of Experimenters
This paper proposes a decision-theoretic framework for experiment design. We model experimenters as ambiguity-averse decision-makers, who make trade-offs between subjective expected performance and robustness. This framework accounts for experimenters' preference for randomization, and clarifies the circumstances in which randomization is optimal: when the available sample size is large enough or robustness is an important concern. We illustrate the practical value of such a framework by studying the issue of rerandomization. Rerandomization creates a trade-off between subjective performance and robustness. However, robustness loss grows very slowly with the number of times one randomizes. This argues for rerandomizing in most environments.
We are grateful to Angus Deaton, Pascaline Dupas, Jeff Ely, Guido Imbens, Charles Manski, Pablo Montagnes, Marco Ottaviani, Bruno Strulovici, Alexey Tetenov, Duncan Thomas, Chris Udry, as well as audience members at Bocconi, the Cowles Econometric Conference (2016), the ESSET Meeting at Gerzensee (2017), Emory, INSEAD, MIT, the NBER Development Economics Summer Meeting (2016), the North-American Econometric Society Meeting (2016), the SISL Conference at Caltech (2017), UBC, and the UCL/Cemmap workshop on Econometrics for Public Policy (2016), for several helpful discussions. Chassang and Snowberg gratefully acknowledge the support of NSF grant SES-1156154. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.