NATIONAL BUREAU OF ECONOMIC RESEARCH
NATIONAL BUREAU OF ECONOMIC RESEARCH

Biased Beliefs About Random Samples: Evidence from Two Integrated Experiments

Daniel J. Benjamin, Don A. Moore, Matthew Rabin

NBER Working Paper No. 23927
Issued in October 2017
NBER Program(s):AG

This paper describes results of a pair of incentivized experiments on biases in judgments about random samples. Consistent with the Law of Small Numbers (LSN), participants exaggerated the likelihood that short sequences and random subsets of coin flips would be balanced between heads and tails. Consistent with the Non-Belief in the Law of Large Numbers (NBLLN), participants underestimated the likelihood that large samples would be close to 50% heads. However, we identify some shortcomings of existing models of LSN, and we find that NBLLN may not be as stable as previous studies suggest. We also find evidence for exact representativeness (ER), whereby people tend to exaggerate the likelihood that samples will (nearly) exactly mirror the underlying odds, as an additional bias beyond LSN. Our within-subject design of asking many different questions about the same data lets us disentangle the biases from possible rational alternative interpretations by showing that the biases lead to inconsistency in answers. Our design also centers on identifying and controlling for bin effects, whereby the probability assigned to outcomes systematically depends on the categories used to elicit beliefs in a way predicted by support theory. The bin effects are large and systematic and affect some results, but we find LSN, NBLLN, and ER even after controlling for them.

download in pdf format
   (2135 K)

email paper

The NBER Bulletin on Aging and Health provides summaries of publications like this.  You can sign up to receive the NBER Bulletin on Aging and Health by email.

Supplementary materials for this paper:

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w23927

 
Publications
Activities
Meetings
NBER Videos
Themes
Data
People
About

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email: info@nber.org

Contact Us