Reasonable Doubt: Experimental Detection of Job-Level Employment Discrimination
This paper develops methods for detecting discrimination by individual employers using correspondence experiments that send fictitious resumes to real job openings. We establish identification of higher moments of the distribution of job-level callback rates as a function of the number of resumes sent to each job and propose shape-constrained estimators of these moments. Applying our methods to three experimental datasets, we find striking job-level heterogeneity in the extent to which callback probabilities differ by race or sex. Estimates of higher moments reveal that while most jobs barely discriminate, a few discriminate heavily. These moment estimates are then used to bound the share of jobs that discriminate and the posterior probability that each individual job is engaged in discrimination. In a recent experiment manipulating racially distinctive names, we find that at least 85% of jobs that contact both of two white applications and neither of two black applications are engaged in discrimination. To assess the potential value of our methods for regulators, we consider the accuracy of decision rules for investigating suspicious callback behavior in various experimental designs under a simple two-type model that rationalizes the experimental data. Though we estimate that only 17% of employers discriminate on the basis of race, we find that an experiment sending 10 applications to each job would enable detection of 7-10% of discriminatory jobs while yielding Type I error rates below 0.2%. A minimax decision rule acknowledging partial identification of the distribution of callback rates yields only slightly fewer investigations than a Bayes decision rule based on the two-type model. These findings suggest illegal labor market discrimination can be reliably monitored with relatively small modifications to existing correspondence designs.
This paper previously circulated under the title “Audits as Evidence: Experiments, Ensembles, and Enforcement.” We thank Isaiah Andrews, Tim Armstrong, Kerwin Charles, Sendhil Mullainathan, and Andres Santos for helpful conversations related to this project, and Eva Arceo-Gomez, Ray Campos-Vasquez, and John Nunley for providing data. We also thank participants at the UC Berkeley labor and econometrics seminars, the Y-Rise External Validity conference, the 2019 NBER Summer Institute, University of British Columbia, UC Irvine, Harris School of Public Policy, the Tinbergen Institute, the Stockholm School of Economics, the University of Oslo, the 2019 All California Economics Conference, the University of Michigan, Stanford University, Harvard University, and Clemson University for useful feedback. Evan Rose and Benjamin Scuderi provided outstanding research assistance. This project was supported by a Russell Sage Foundation Presidential grant. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Patrick Kline & Christopher Walters, 2021. "Reasonable Doubt: Experimental Detection of Job‐Level Employment Discrimination," Econometrica, Econometric Society, vol. 89(2), pages 765-792, March. citation courtesy of