Measuring Polarization in High-Dimensional Data: Method and Application to Congressional Speech
NBER Working Paper No. 22423
We study trends in the partisanship of Congressional speech from 1873 to 2009. We define partisanship to be the ease with which an observer could infer a congressperson’s party from a fixed amount of speech, and we estimate it using a structural choice model and methods from machine learning. The estimates reveal that partisanship is far greater today than at any point in the past. Partisanship was low and roughly constant from 1873 to the early 1990s, then increased dramatically in subsequent years. Evidence suggests innovation in political persuasion beginning with the Contract with America, possibly reinforced by changes in the media environment, as a likely cause. Naive estimates of partisanship are subject to a severe finite-sample bias and imply substantially different conclusions.
You may purchase this paper on-line in .pdf format from SSRN.com ($5) for electronic delivery.
Supplementary materials for this paper:
Document Object Identifier (DOI): 10.3386/w22423