Measuring Polarization in High-Dimensional Data: Method and Application to Congressional Speech

Matthew Gentzkow, Jesse M. Shapiro, Matt Taddy

NBER Working Paper No. 22423
Issued in July 2016, Revised in May 2017
NBER Program(s):Political Economy

We study trends in the partisanship of congressional speech from 1873 to 2016. We define partisanship to be the ease with which an observer could infer a congressperson’s party from a fixed amount of speech, and we estimate it using a structural choice model and methods from machine learning. Our method corrects a severe finite-sample bias that we show arises with standard estimators. The results reveal that partisanship is far greater in recent years than in the past, and that it increased sharply in the early 1990s after remaining low and relatively constant over the preceding century. Our method is applicable to the study of high-dimensional choices in many domains, and we illustrate its broader utility with an application to residential segregation.

You may purchase this paper on-line in .pdf format from ($5) for electronic delivery.

Access to NBER Papers

You are eligible for a free download if you are a subscriber, a corporate associate of the NBER, a journalist, an employee of the U.S. federal government with a ".GOV" domain name, or a resident of nearly any developing country or transition economy.

If you usually get free papers at work/university but do not at home, you can either connect to your work VPN or proxy (if any) or elect to have a link to the paper emailed to your work email address below. The email address must be connected to a subscribing college, university, or other subscribing institution. Gmail and other free email addresses will not have access.


Supplementary materials for this paper:

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w22423

Users who downloaded this paper also downloaded* these:
Woodford w22285 Quantitative Easing and Financial Stability
Levitt w22487 Heads or Tails: The Impact of a Coin Toss on Major Life Decisions and Subsequent Happiness
Edmans, Li, and Zhang w20300 Employee Satisfaction, Labor Market Flexibility, and Stock Returns Around The World
Piazzesi and Schneider w22354 Housing and Macroeconomics
Cochrane w22485 Macro-Finance
NBER Videos

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email:

Contact Us