TY - JOUR AU - Heckman,James AU - Ichimura,Hidehiko AU - Smith,Jeffrey AU - Todd,Petra TI - Characterizing Selection Bias Using Experimental Data JF - National Bureau of Economic Research Working Paper Series VL - No. 6699 PY - 1998 Y2 - August 1998 UR - http://www.nber.org/papers/w6699 L1 - http://www.nber.org/papers/w6699.pdf N1 - Author contact info: James J. Heckman Department of Economics The University of Chicago 1126 E. 59th Street Chicago, IL 60637 Tel: 773/702-0634 Fax: 773/702-8490 E-Mail: jjh@uchicago.edu Hidehiko Ichimura Graduate School of Economics University of Tokyo Hongo 7-3-1 Tokyo 113-0033 Japan E-Mail: ichimura@e.u-tokyo.ac.jp Jeffrey Smith Department of Economics University of Michigan 238 Lorch Hall 611 Tappan Street Ann Arbor, MI 48109-1220 Tel: 734/764-5359 E-Mail: econjeff@umich.edu Petra E. Todd Department of Economics University of Pennsylvania 3718 Locust Walk Philadelphia, PA 19104 Tel: 215/898-4084 Fax: 215/573-2057 E-Mail: ptodd@econ.upenn.edu AB - This paper develops and applies semiparametric econometric methods to estimate the form of selection bias that arises from using nonexperimental comparison groups to evaluate social programs and to test the identifying assumptions that justify three widely-used classes of estimators and our extensions of them: (a) the method of matching; (b) the classical econometric selection model which represents the bias solely as a function of the probability of participation; and (c) the method of difference-in-differences. Using data from an experiment on a prototypical social program combined with unusually rich data from a nonexperimental comparison group, we reject the assumptions justifying matching and our extensions of that method but find evidence in support of the index-sufficient selection bias model and the assumptions that justify application of a conditional semiparametric version of the method of difference-in-difference. Fa comparable people and to appropriately weight participants and nonparticipants a sources of selection bias as conveniently measured. We present a rigorous defin bias and find that in our data it is a small component of conventially meausred it is still substantial when compared with experimentally-estimated program impa matching participants to comparison group members in the same labor market, givi same questionnaire, and making sure they have comparable characteristics substan the performance of any econometric program evaluation estimator. We show how t analysis to estimate the impact of treatment on the treated using ordinary obser ER -