Measuring Polarization in High-Dimensional Data: Method and Application to Congressional Speech
NBER Working Paper No. 22423
We study trends in the partisanship of congressional speech from 1873 to 2016. We define partisanship to be the ease with which an observer could infer a congressperson’s party from a fixed amount of speech, and we estimate it using a structural choice model and methods from machine learning. Our method corrects a severe finite-sample bias that we show arises with standard estimators. The results reveal that partisanship is far greater in recent years than in the past, and that it increased sharply in the early 1990s after remaining low and relatively constant over the preceding century. Our method is applicable to the study of high-dimensional choices in many domains, and we illustrate its broader utility with an application to residential segregation.
You may purchase this paper on-line in .pdf format from SSRN.com ($5) for electronic delivery.
Supplementary materials for this paper:
This paper was revised on May 25, 2017
Document Object Identifier (DOI): 10.3386/w22423