Identifying Confirmatory Bias in the Field: Evidence from a Poll of Experts

Rodney J. Andrews, Trevon D. Logan, Michael J. Sinkey

NBER Working Paper No. 18064
Issued in May 2012
NBER Program(s):Development of the American Economy

Laboratory experiments have established the existence of cognitive biases, but their explanatory power in real-world economic settings has been difficult to measure. We estimate the extent of a cognitive bias, confirmatory bias, among experts in a real-world environment. In the Associated Press Top 25 College Football Poll expert pollsters are tasked with assessing team quality, and their beliefs are treated week-to-week with game results that serve as signals about an individual team's quality. We exploit the variation provided by actual game results relative to market expectations to develop a novel regression-discontinuity approach to identify confirmatory bias in this real-world setting. We construct a unique personally-assembled dataset that matches more than twenty years of individual game characteristics to poll results and betting market information, and show that teams that slightly exceed and barely miss market expectations are exchangeable. The likelihood of winning the game, the average number of points scored by teams and their opponents, and even the average week of the season are no different between teams that slightly exceed and barely miss market expectations. Pollsters, however, significantly upgrade their beliefs about a team's quality when a team slightly exceeds market expectations. The effects are sizeable-- nearly half of the voters in the poll rank a team one slot higher when they slightly exceed market expectations; one-fifth of the standard deviation in poll points in a given week can be attributed to confirmatory bias. This type of updating suggests that even when informed agents make repeated decisions they may act in a manner which is consistent with confirmatory bias.

download in pdf format
   (1331 K)

email paper

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w18064

Published: Rodney J. Andrews & Trevon D. Logan & Michael J. Sinkey, 2018. "Identifying Confirmatory Bias in the Field," Journal of Sports Economics, vol 19(1), pages 50-81.

Users who downloaded this paper also downloaded* these:
Andrews, Li, and Lovenheim w18068 Quantile Treatment Effects of College Quality on Earnings: Evidence from Administrative Data in Texas
Cook, Logan, and Parman w18802 Distinctively Black Names in the American Past
Andrews, Jargowsky, and Kuhne w18598 The Effects of Texas's Targeted Pre-Kindergarten Program on Academic Performance
Carrell, Sacerdote, and West w16865 From Natural Variation to Optimal Policy? The Lucas Critique Meets Peer Effects
Barsky and Sims w15312 News Shocks
NBER Videos

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email:

Contact Us