Forecasting the Results of Experiments: Piloting an Elicitation Strategy
Forecasts of experimental results can clarify the interpretation of research results, mitigate publication bias, and improve experimental designs. We collect forecasts of the results of three Registered Reports preliminarily accepted to the Journal of Development Economics, randomly varying four features: (1) small versus large reference values; (2) whether predictions are in raw units or standard deviations; (3) text-entry versus slider responses; and (4) small versus large slider bounds. Forecasts are generally robust to elicitation features, though wider slider bounds are associated with higher forecasts throughout the forecast distribution. We make preliminary recommendations on how many forecasts should be gathered.
We are grateful for financial support from the Alfred P. Sloan Foundation (G-2019-12325), and an anonymous foundation. Vivalt is also supported by the John Mitchell Economics of Poverty Lab at the Australian National University. We thank seminar participants at the ASSA 2020, and especially our discussant, David McKenzie. The paper will appear in the 2020 AEA Papers and Proceedings. We also thank the authors of the forecast studies for their support and contributions to the survey and the respondents for their time. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Stefano DellaVigna & Nicholas Otis & Eva Vivalt, 2020. "Forecasting the Results of Experiments: Piloting an Elicitation Strategy," AEA Papers and Proceedings, vol 110, pages 75-79. citation courtesy of