Crowdsourcing City Government: Using Tournaments to Improve Inspection Accuracy
Can open tournaments improve the quality of city services? The proliferation of big data makes it possible to use predictive analytics to better target services like hygiene inspections, but city governments rarely have the in-house talent needed for developing prediction algorithms. Cities could hire consultants, but a cheaper alternative is to crowdsource competence by making data public and offering a reward for the best algorithm. This paper provides a simple model suggesting that open tournaments dominate consulting contracts when cities have a reasonable tolerance for risk and when there is enough labor with low opportunity costs of time. We also illustrate how tournaments can be successful, by reporting on a Boston-based restaurant hygiene prediction tournament that we helped coordinate. The Boston tournament yielded algorithms—at low cost—that proved reasonably accurate when tested “out-of-sample” on hygiene inspections occurring after the algorithms were submitted. We draw upon our experience in working with Boston to provide practical suggestions for governments and other organizations seeking to run prediction tournaments in the future.
The authors are deeply grateful for the collaboration of the City of Boston (especially Ben Batorsky, Matthew Mayrl, and Commissioner William Christopher), Yelp (Artem Avdacev, Luther Lowe, and Aaron Schur), and DrivenData (Peter Bull and Greg Lipstein), without whom the project described herein would not have been possible. Additionally, the authors gratefully acknowledge the helpful comments of Benjamin Edelman, Anthony Goldbloom, Mitchell Weiss, and especially Susan Athey, as well as the support of Yelp, the National Science Foundation (grants CCF-1216095, DGE-1144152, and SES-1459912), the Harvard Milton Fund, the Taubman Center for State and Local Government, the Rappaport Institute for Greater Boston, and the Wu Fund for Big Data Analysis. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Edward L. Glaeser
I have received speaking fees from organizations that organize members that invest in real estate markets, including the National Association of Real Estate Investment Managers and the Pension Real Estate Association.
Edward L. Glaeser & Andrew Hillis & Scott Duke Kominers & Michael Luca, 2016. "Crowdsourcing City Government: Using Tournaments to Improve Inspection Accuracy," American Economic Review, vol 106(5), pages 114-118. citation courtesy of