Measuring the Measurement Error: A Method to Qualitatively Validate Survey Data

Christopher Blattman, Julian C. Jamison, Tricia Koroknay-Palicz, Katherine Rodrigues, Margaret Sheridan

NBER Working Paper No. 21447
Issued in August 2015
NBER Program(s):Development Economics

Field experiments rely heavily on self-reported data, but subjects may misreport behaviors, especially sensitive ones such as crime. If treatment influences survey responses, it biases experimental estimates. We develop a validation technique that uses intensive qualitative work to assess survey measurement error. Subjects were assigned to receive cash, therapy, both, or neither. According to survey responses, receiving both treatments dramatically reduced crime and other sensitive behaviors. Local researchers spent several days with a random subsample of subjects following their endline surveys, building trust and seeking verbal confirmation of six behaviors: theft, drug use, homelessness, gambling, and two expenditures. This validation suggests that subjects in the control and cash only groups underreported sensitive behaviors and expenditures in the survey relative to the other treatment arms. We bound survey-based treatment effects estimates, and find the impacts of cash and therapy on crime may be larger than suggested by surveys alone.

download in pdf format
   (610 K)

email paper

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w21447

Published: Blattman, Christopher & Jamison, Julian & Koroknay-Palicz, Tricia & Rodrigues, Katherine & Sheridan, Margaret, 2016. "Measuring the measurement error: A method to qualitatively validate survey data," Journal of Development Economics, Elsevier, vol. 120(C), pages 99-112. citation courtesy of

Users who downloaded this paper also downloaded* these:
Blattman, Jamison, and Sheridan w21204 Reducing crime and violence: Experimental evidence from cognitive behavioral therapy in Liberia
Conti, Heckman, and Pinto w21454 The Effects of Two Influential Early Childhood Interventions on Health and Healthy Behaviors
Fryer, Levitt, and List w21477 Parental Incentives and Early Childhood Achievement: A Field Experiment in Chicago Heights
Getmansky, Lee, and Lo w21449 Hedge Funds: A Dynamic Industry In Transition
Gillen, Snowberg, and Yariv w21517 Experimenting with Measurement Error: Techniques with Applications to the Caltech Cohort Study
NBER Videos

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email:

Contact Us