Should the Randomistas (Continue to) Rule?
The rising popularity of randomized controlled trials (RCTs) in development applications has come with continuing debates about the merits of this approach. The paper takes stock of the issues. It argues that an unconditional preference for RCTs is questionable on three main counts. First, the case for such a preference is unclear on a priori grounds. For example, with a given budget, even a biased observational study can come closer to the truth than a costly RCT. Second, the ethical objections to RCTs have not been properly addressed by advocates. Third, there is a risk of distorting the evidence-base for informing policymaking, given that an insistence on RCTs generates selection bias in what gets evaluated. Going forward, pressing knowledge gaps should drive the questions asked and how they are answered, not the methodological preferences of some researchers. The gold standard is the best method for the question at hand.
François Roubaud encouraged the author to write this paper. The author thanks Jason Abaluck, Sarah Baird, Radu Ban, Mary Ann Bronson, Caitlin Brown, Sylvain Chabé-Ferret, Kevin Donovan, Ryan Edwards, Markus Goldstein, Miguel Hernan, Emmanuel Jimenez, Max Kasy, Madhulika Khanna, Nishtha Kochhar, Agnès Labrousse, Andrew Leigh, David McKenzie, Rachael Meager, Berk Özler, Dina Pomeranz, Lant Pritchett, Milan Thomas, Vinod Thomas, Eva Vivalt, Dominique van de Walle, Andrew Zeitlin, and participants at a workshop in Paris, March 2019. The staff of the International Initiative for Impact Evaluation kindly provided an update to their database on published impact evaluations and helped with the author’s questions. The views expressed herein are those of the author and do not necessarily reflect the views of the National Bureau of Economic Research.