NATIONAL BUREAU OF ECONOMIC RESEARCH
NATIONAL BUREAU OF ECONOMIC RESEARCH

Understanding and Misunderstanding Randomized Controlled Trials

Angus Deaton, Nancy Cartwright

NBER Working Paper No. 22595
Issued in September 2016
NBER Program(s):   AG   DEV   HC   HE   LS

The use of RCTs is spreading in economics. They are seen as desirable for discovery and for generating evidence for policy. Yet some of the enthusiasm for RCTs comes from misunderstandings: that randomization provides a fair test by equalizing everything but the treatment and so allows a precise estimate of the treatment alone; that randomization is required to solve selection problems; that lack of blinding does little to compromise inference; and that statistical inference in RCTs is straightforward, because it requires only the comparison of two means. RCTs do indeed require minimal assumptions and prior knowledge, an advantage when persuading distrustful audiences, but a crucial disadvantage for cumulative scientific progress, where randomization undermines precision. It is hard to use them outside of their original context. Yet, once they are seen as part of a cumulative program, they can play a role in building general knowledge and useful predictions, provided they are combined with other methods, including conceptual and theoretical development, to discover not “what works,” but why things work. Unless we are prepared to make assumptions, and to stand on what we know, making statements that will be incredible to some, all the credibility of RCTs is for naught.

download in pdf format
   (999 K)

email paper

The NBER Bulletin on Aging and Health provides summaries of publications like this.  You can sign up to receive the NBER Bulletin on Aging and Health by email.

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w22595

Users who downloaded this paper also downloaded* these:
Jones, Reedy, and Weinberg w19866 Age and Scientific Genius
Gordon w18315 Is U.S. Economic Growth Over? Faltering Innovation Confronts the Six Headwinds
Duranton and Turner w15376 The Fundamental Law of Road Congestion: Evidence from US cities
Deaton w14690 Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development
Fryer w22130 The Production of Human Capital in Developed Countries: Evidence from 196 Randomized Field Experiments
 
Publications
Activities
Meetings
NBER Videos
Themes
Data
People
About

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email: info@nber.org

Contact Us