Errors in Survey Reporting and Imputation and their Effects on Estimates of Food Stamp Program Participation
Accurately measuring government benefit receipt in household surveys is necessary when studying disadvantaged populations and the programs that serve them. The Food Stamp Program is especially important given its size and recent growth. To validate survey reports, we use administrative data on participation in two states linked to the American Community Survey (ACS), the Current Population Survey (CPS), and the Survey of Income and Program Participation (SIPP). We find that 23 percent of true food stamp recipient households do not report receipt in the SIPP, 35 percent in the ACS, and fully 50 percent in the CPS. A substantial number of true non-recipients are also recorded as recipients, especially in the SIPP. We examine reasons for these errors including imputation, an important source of error. Both false negative and false positive reports vary with household characteristics, implying complicated biases in multivariate analyses, such as regressions. We then directly examine biases in common survey-based estimates of program receipt by comparing them to estimates from our combined administrative and survey data. We find that the survey estimates understate participation among single parents, non-whites, and low-income households, and also lead to errors in multiple program receipt, and time and age patterns of receipt.
This research was supported by the Economic Research Service of the USDA, the Russell Sage, Alfred P. Sloan and Charles Koch Foundations, the Czech Science Foundation (through grant no. 16-07603Y) and the Czech Academy of Sciences (through institutional support RVO 67985998). Any opinions and conclusions expressed herein are those of the authors and do not necessarily represent the views of the USDA or the U.S. Census Bureau. We have greatly benefitted from the comments of David Johnson, John Kirlin, Gayatri Koolwal, Alan Krueger, Cathleen Li, Daniel Schroeder, James Spletzer, Jane Stavely, Shelly Ver Ploeg, Derek Wu and audiences at the American Economic Association Meetings, American Statistical Association Meetings, Baylor University, European Congress of Methodology, ITSEW, Harvard University, USDA, Yale University and ZEW. We are grateful for the assistance of many current and former Census Bureau employees including David Johnson, Amy O’Hara, Lynn Riggs and Frank Limehouse. The data analysis was conducted at the Chicago RDC and was screened to avoid revealing confidential data. Lucy Bilaver, Kerry Franzetta and Janna Johnson provided excellent research assistance. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Robert M. Goerge
The research was supported by the U.S. Department of Health and Human Services Administration for Children and Families.