NATIONAL BUREAU OF ECONOMIC RESEARCH
NATIONAL BUREAU OF ECONOMIC RESEARCH

Measuring the Measurement Error: A Method to Qualitatively Validate Survey Data

Christopher Blattman, Julian C. Jamison, Tricia Koroknay-Palicz, Katherine Rodrigues, Margaret Sheridan

NBER Working Paper No. 21447
Issued in August 2015
NBER Program(s):   DEV

Field experiments rely heavily on self-reported data, but subjects may misreport behaviors, especially sensitive ones such as crime. If treatment influences survey responses, it biases experimental estimates. We develop a validation technique that uses intensive qualitative work to assess survey measurement error. Subjects were assigned to receive cash, therapy, both, or neither. According to survey responses, receiving both treatments dramatically reduced crime and other sensitive behaviors. Local researchers spent several days with a random subsample of subjects following their endline surveys, building trust and seeking verbal confirmation of six behaviors: theft, drug use, homelessness, gambling, and two expenditures. This validation suggests that subjects in the control and cash only groups underreported sensitive behaviors and expenditures in the survey relative to the other treatment arms. We bound survey-based treatment effects estimates, and find the impacts of cash and therapy on crime may be larger than suggested by surveys alone.

You may purchase this paper on-line in .pdf format from SSRN.com ($5) for electronic delivery.

Access to NBER Papers

You are eligible for a free download if you are a subscriber, a corporate associate of the NBER, a journalist, an employee of the U.S. federal government with a ".GOV" domain name, or a resident of nearly any developing country or transition economy.

If you usually get free papers at work/university but do not at home, you can either connect to your work VPN or proxy (if any) or elect to have a link to the paper emailed to your work email address below. The email address must be connected to a subscribing college, university, or other subscribing institution. Gmail and other free email addresses will not have access.

E-mail:

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w21447

Published: Blattman, Christopher & Jamison, Julian & Koroknay-Palicz, Tricia & Rodrigues, Katherine & Sheridan, Margaret, 2016. "Measuring the measurement error: A method to qualitatively validate survey data," Journal of Development Economics, Elsevier, vol. 120(C), pages 99-112. citation courtesy of

Users who downloaded this paper also downloaded* these:
Conti, Heckman, and Pinto w21454 The Effects of Two Influential Early Childhood Interventions on Health and Healthy Behaviors
Fryer, Levitt, and List w21477 Parental Incentives and Early Childhood Achievement: A Field Experiment in Chicago Heights
Getmansky, Lee, and Lo w21449 Hedge Funds: A Dynamic Industry In Transition
Feng, Hu, and Moffitt w21460 Long Run Trends in Unemployment and Labor Force Participation in China
Gillen, Snowberg, and Yariv w21517 Experimenting with Measurement Error: Techniques with Applications to the Caltech Cohort Study
 
Publications
Activities
Meetings
NBER Videos
Themes
Data
People
About

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email: info@nber.org

Contact Us