Household Surveys in Crisis
Household surveys, one of the main innovations in social science research of the last century, are threatened by declining accuracy due to reduced cooperation of respondents. While many indicators of survey quality have steadily declined in recent decades, the literature has largely emphasized rising nonresponse rates rather than other potentially more important dimensions to the problem. We divide the problem into rising rates of nonresponse, imputation, and measurement error, documenting the rise in each of these threats to survey quality over the past three decades. A fundamental problem in assessing biases due to these problems in surveys is the lack of a benchmark or measure of truth, leading us to focus on the accuracy of the reporting of government transfers. We provide evidence from aggregate measures of transfer reporting as well as linked microdata. We discuss the relative importance of misreporting of program receipt and conditional amounts of benefits received, as well as some of the conjectured reasons for declining cooperation and survey errors. We end by discussing ways to reduce the impact of the problem including the increased use of administrative data and the possibilities for combining administrative and survey data.
We are grateful for the assistance of many New York Office of Temporary and Disability Assistance (OTDA) and Census Bureau employees including George Falco, Dave Dlugolecki, Graton Gathright, Joey Morales, Amy O’Hara, and Frank Limehouse. The micro-data analysis was conducted at the Chicago Census Research Data Center by researchers with Special Sworn Status and the results were reviewed to prevent the disclosure of confidential information. We would like to thank Pablo Celhay for excellent research assistance. We also thank Dan Black, Constance Citro, Michael Davern, Nikolas Mittag and participants at seminars at the American Enterprise Institute and the Federal Deposit Insurance Corporation for their helpful comments. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research, the New York OTDA or the U.S. Census Bureau.
Bruce D. Meyer & Wallace K. C. Mok & James X. Sullivan, 2015. "Household Surveys in Crisis," Journal of Economic Perspectives, American Economic Association, vol. 29(4), pages 199-226, Fall. citation courtesy of