Experimenting with Measurement Error: Techniques with Applications to the Caltech Cohort Study
Measurement error is ubiquitous in experimental work. It leads to imperfect statistical controls, attenuated estimated effects of elicited behaviors, and biased correlations between characteristics. We develop simple statistical techniques for dealing with experimental measurement error. These techniques are applied to data from the Caltech Cohort Study, which conducts repeated incentivized surveys of the Caltech student body. We illustrate the impact of measurement error by replicating three classic experiments, and showing that results change substantially when measurement error is taken into account. Collectively, these results show that failing to properly account for measurement error may cause a field-wide bias: it may lead scholars to identify "new" effects and phenomena that are actually similar to those previously documented.
Snowberg gratefully acknowledges the support of NSF grants SES-1156154 and SMA-1329195. Yariv gratefully acknowledges the support of NSF grant SES-0963583 and the Gordon and Betty Moore Foundation grant 1158. We thank Marco Castillo, Yoram Halevy, Muriel Niederle, and Lise Vesterlund for comments and suggestions, as well as seminar audiences at Caltech, SITE, and UBC. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Ben Gillen & Erik Snowberg & Leeat Yariv, 2019. "Experimenting with Measurement Error: Techniques with Applications to the Caltech Cohort Study," Journal of Political Economy, vol 127(4), pages 1826-1863.