Knowledge, Tests, and Fadeout in Educational Interventions
Educational interventions are often evaluated and compared on the basis of their impacts on test scores. Decades of research have produced two empirical regularities: interventions in later grades tend to have smaller effects than the same interventions in earlier grades, and the test score impacts of early educational interventions almost universally "fade out" over time. This paper explores whether these empirical regularities are an artifact of the common practice of rescaling test scores in terms of a student's position in a widening distribution of knowledge. If a standard deviation in test scores in later grades translates into a larger difference in knowledge, an intervention's effect on normalized test scores may fall even as its effect on knowledge does not. We evaluate this hypothesis by fitting a model of education production to correlations in test scores across grades and with college-going using both administrative and survey data. Our results imply that the variance in knowledge does indeed rise as children progress through school, but not enough for test score normalization to fully explain these empirical regularities.
We thank Kevin Lang and Thomas Lemieux for the conversations that motivated this research, and Raj Chetty, Caroline Hoxby, Kevin Lang, Larry Katz, Jesse Rothstein, and seminar participants at the NBER Education Program meetings for helpful comments. We are also grateful for the outstanding research assistance of Sarah Cohodes. This work was supported by the Institute of Education Sciences, U.S. Department of Education grant [R305C090023] to the President and Fellows of Harvard College. Cascio also gratefully acknowledges funding from the National Academy of Education and the National Academy of Education/Spencer Postdoctoral Fellowship Program and a Rockefeller-Haney Fellowship from Dartmouth College. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.