Impact Evaluation in Matching Markets with General Tie-Breaking
Many centralized matching schemes incorporate a mix of random lottery and non-lottery tie-breaking. A leading example is the New York City public school district, which uses criteria like test scores and interviews to generate applicant rankings for some schools, combined with lottery tie-breaking at other schools. We develop methods that identify causal effects of assignment in such settings. Our approach generalizes the standard regression discontinuity design to allow for many running variables and treatments, some of which are randomly assigned. We show that lottery variation generates assignment risk at non-lottery programs for applicants away from non-lottery cutoffs, while non-lottery variation randomizes applicants near cutoffs regardless of lottery risk. These methods are applied to evaluate New York City’s school progress assessments, which give schools letter grades as a summary measure of quality. Our estimates reveal that although Grade A schools boost achievement, these gains emerge only for students who attend lottery schools. Attendance at a coveted Grade A screened school, including some of the highest performing in the district, generates no measurable effects. Evaluation methods that fail to take advantage of both lottery and non-lottery variation miss this difference in impact.
We thank Nadiya Chadha, Sonali Murarka, Lianna Wright, and the staff of the New York City Department of Education for answering our questions and facilitating access to data. Tim Armstrong, Eduardo Azevedo and Whitney Newey provided helpful feedback. We’re especially indebted to Adrian Blattner and Ignacio Rodriguez for expert research assistance and to MIT SEII program manager Eryn Heying for invaluable administrative support. We gratefully acknowledge funding from the Laura and John Arnold Foundation, the National Science Foundation (under awards SES-1056325 and SES-1426541), and the WT Grant Foundation. Abdulkadiroglu and Pathak are Scientific Advisory Board members of the Institute for Innovation in Public School Choice. Angrist’s daughter teaches at a Boston charter school. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.