Characterizing Selection Bias Using Experimental Data

James Heckman, Hidehiko Ichimura, Jeffrey Smith, Petra Todd

NBER Working Paper No. 6699
Issued in August 1998
NBER Program(s):   LS   PE

This paper develops and applies semiparametric econometric methods to estimate the form of selection bias that arises from using nonexperimental comparison groups to evaluate social programs and to test the identifying assumptions that justify three widely-used classes of estimators and our extensions of them: (a) the method of matching; (b) the classical econometric selection model which represents the bias solely as a function of the probability of participation; and (c) the method of difference-in-differences. Using data from an experiment on a prototypical social program combined with unusually rich data from a nonexperimental comparison group, we reject the assumptions justifying matching and our extensions of that method but find evidence in support of the index-sufficient selection bias model and the assumptions that justify application of a conditional semiparametric version of the method of difference-in-difference. Fa comparable people and to appropriately weight participants and nonparticipants a sources of selection bias as conveniently measured. We present a rigorous defin bias and find that in our data it is a small component of conventially meausred it is still substantial when compared with experimentally-estimated program impa matching participants to comparison group members in the same labor market, givi same questionnaire, and making sure they have comparable characteristics substan the performance of any econometric program evaluation estimator. We show how t analysis to estimate the impact of treatment on the treated using ordinary obser

download in pdf format
   (3335 K)

email paper

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w6699

Published: Econometrica, Vol. 66, no. 5 (1999): 1017-1098. citation courtesy of

Users who downloaded this paper also downloaded these:
Heckman and Smith w6105 The Sensitivity of Experimental Impact Estimates: Evidence from the National JTPA Study
Heckman w2861 Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training
Heckman w0172 Sample Selection Bias As a Specification Error (with an Application to the Estimation of Labor Supply Functions)
Heckman The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models
Cameron and Heckman w6385 Life Cycle Schooling and Dynamic Selection Bias: Models and Evidence for Five Cohorts
NBER Videos

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email:

Contact Us