Can Online Off-The-Shelf Lessons Improve Student Outcomes? Evidence from A Field Experiment
There has been a proliferation of websites that warehouse instructional materials designed to be taught by teachers in a traditional classroom. While this new technology has revolutionized how most teachers plan their lessons, the potential benefits of this innovation are unknown. To present evidence on this, we analyze an experiment in which middle-school math teachers were randomly given access to high-quality “off-the-shelf” lessons. Only providing teachers with online access to the lessons increased students’ math achievement by 0.06 of a standard deviation, but providing teachers with online access to the lessons along with supports to promote their use increased students’ math achievement by 0.09 of a standard deviation. Benefits were much larger for weaker teachers, suggesting that weaker teachers compensated for skill deficiencies by substituting the lessons for their own efforts. Survey evidence suggests that these effects were mediated by both improvements in lesson quality and teachers having more time to engage in other tasks. We rationalize these results with a multitask model of teaching. The intervention is more scalable and cost effective than most policies aimed at improving teacher quality, suggesting a real benefit to making high-quality instructional materials available to teachers on the internet.
This paper was previously circulated under the title “Simplifying Teaching: A Field Experiment with Online ‘Off-the-Shelf’ Lessons.” This paper was made possible by a grant from the Carnegie Corporation of New York through 100Kin10. We’re extremely grateful to Ginny Stuckey and Kate Novak at Mathalicious, and Sarah Emmons of the University of Chicago Education Lab, and Tracy Dell’Angela at the University of Chicago Urban Education Institute. We also thank math coordinators and the data management persons in Hanover, Henrico, and Chesterfield school districts. We thank Amy Wagner, Jenni Heissel, Hao Hu, and Mathew Steinberg for excellent research assistance. This paper benefited from comments from Jon Guryan, Eric Mbakop, Irma Perez-Johnson, Sergey V. Popov, Egor Starkov, and participants of APPAM 2015. The statements made and views expressed are solely the responsibility of the authors. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Kirabo Jackson & Alexey Makarin, 2018. "Can Online Off-the-Shelf Lessons Improve Student Outcomes? Evidence from a Field Experiment," American Economic Journal: Economic Policy, vol 10(3), pages 226-254. citation courtesy of