Measuring the Measurement Error: A Method to Qualitatively Validate Survey Data

Christopher Blattman, Julian C. Jamison, Tricia Koroknay-Palicz, Katherine Rodrigues, Margaret Sheridan

NBER Working Paper No. 21447
Issued in August 2015
NBER Program(s):Development Economics

Field experiments rely heavily on self-reported data, but subjects may misreport behaviors, especially sensitive ones such as crime. If treatment influences survey responses, it biases experimental estimates. We develop a validation technique that uses intensive qualitative work to assess survey measurement error. Subjects were assigned to receive cash, therapy, both, or neither. According to survey responses, receiving both treatments dramatically reduced crime and other sensitive behaviors. Local researchers spent several days with a random subsample of subjects following their endline surveys, building trust and seeking verbal confirmation of six behaviors: theft, drug use, homelessness, gambling, and two expenditures. This validation suggests that subjects in the control and cash only groups underreported sensitive behaviors and expenditures in the survey relative to the other treatment arms. We bound survey-based treatment effects estimates, and find the impacts of cash and therapy on crime may be larger than suggested by surveys alone.

You may purchase this paper on-line in .pdf format from ($5) for electronic delivery.

Access to NBER Papers

You are eligible for a free download if you are a subscriber, a corporate associate of the NBER, a journalist, an employee of the U.S. federal government with a ".GOV" domain name, or a resident of nearly any developing country or transition economy.

If you usually get free papers at work/university but do not at home, you can either connect to your work VPN or proxy (if any) or elect to have a link to the paper emailed to your work email address below. The email address must be connected to a subscribing college, university, or other subscribing institution. Gmail and other free email addresses will not have access.


Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w21447

Published: Blattman, Christopher & Jamison, Julian & Koroknay-Palicz, Tricia & Rodrigues, Katherine & Sheridan, Margaret, 2016. "Measuring the measurement error: A method to qualitatively validate survey data," Journal of Development Economics, Elsevier, vol. 120(C), pages 99-112. citation courtesy of

Users who downloaded this paper also downloaded* these:
Conti, Heckman, and Pinto w21454 The Effects of Two Influential Early Childhood Interventions on Health and Healthy Behaviors
Blattman, Jamison, and Sheridan w21204 Reducing crime and violence: Experimental evidence from cognitive behavioral therapy in Liberia
Fryer, Levitt, and List w21477 Parental Incentives and Early Childhood Achievement: A Field Experiment in Chicago Heights
Getmansky, Lee, and Lo w21449 Hedge Funds: A Dynamic Industry In Transition
Feng, Hu, and Moffitt w21460 Long Run Trends in Unemployment and Labor Force Participation in China
NBER Videos

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email:

Contact Us