NATIONAL BUREAU OF ECONOMIC RESEARCH
NATIONAL BUREAU OF ECONOMIC RESEARCH

Understanding and Misunderstanding Randomized Controlled Trials

Angus Deaton, Nancy Cartwright

NBER Working Paper No. 22595
Issued in September 2016, Revised in October 2017
NBER Program(s):Aging, Development Economics, Health Care, Health Economics, Labor Studies

RCTs would be more useful if there were more realistic expectations of them and if their pitfalls were better recognized. For example, and contrary to many claims in the applied literature, randomization does not equalize everything but the treatment across treatments and controls, it does not automatically deliver a precise estimate of the average treatment effect (ATE), and it does not relieve us of the need to think about (observed or unobserved) confounders. Estimates apply to the trial sample only, sometimes a convenience sample, and usually selected; justification is required to extend them to other groups, including any population to which the trial sample belongs. Demanding “external validity” is unhelpful because it expects too much of an RCT while undervaluing its contribution. Statistical inference on ATEs involves hazards that are not always recognized. RCTs do indeed require minimal assumptions and can operate with little prior knowledge. This is an advantage when persuading distrustful audiences, but it is a disadvantage for cumulative scientific progress, where prior knowledge should be built upon and not discarded. RCTs can play a role in building scientific knowledge and useful predictions but they can only do so as part of a cumulative program, combining with other methods, including conceptual and theoretical development, to discover not “what works,” but “why things work”.

download in pdf format
   (265 K)

email paper

The NBER Bulletin on Aging and Health provides summaries of publications like this.  You can sign up to receive the NBER Bulletin on Aging and Health by email.

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w22595

Users who downloaded this paper also downloaded* these:
Gordon w18315 Is U.S. Economic Growth Over? Faltering Innovation Confronts the Six Headwinds
Duranton and Turner w15376 The Fundamental Law of Road Congestion: Evidence from US cities
Deaton w14690 Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development
Jones, Reedy, and Weinberg w19866 Age and Scientific Genius
Fryer w22130 The Production of Human Capital in Developed Countries: Evidence from 196 Randomized Field Experiments
 
Publications
Activities
Meetings
NBER Videos
Themes
Data
People
About

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email: info@nber.org

Contact Us