Some Children Left Behind: Variation in the Effects of an Educational Intervention
We document substantial variation in the effects of a highly-effective literacy pro-gram in northern Uganda. The program increases test scores by 1.40 SDs on average, but standard statistical bounds show that the impact standard deviation exceeds 1.0SD. This implies that the variation in effects across our students is wider than the spread of mean effects across all randomized evaluations of developing country education interventions in the literature. This very effective program does indeed leave some students behind. At the same time, we do not learn much from our analyses that attempt to determine which students benefit more or less from the program. We reject rank preservation, and the weaker assumption of stochastic increasingness leaves wide bounds on quantile-specific average treatment effects. Neither conventional nor machine-learning approaches to estimating systematic heterogeneity capture more than a small fraction of the variation in impacts given our available candidate moderators.
We thank participants at the Heckman 75th Birthday Conference, seminar audiences at Aarhus, CESifo, Copenhagen Business School, and RISE, as well as Natalie Bau, Jishnu Das, Paul Glewwe, Lois Miller, PaulNiehaus, Lant Pritchett and three anonymous referees for helpful comments, Brigham Frandsen for assistance in implementing the Frandsen-Lefgren bounds, and Joseph Cummins for sharing his rank similarity test code. Deborah Amuka, Victoria Brown,and Katie Pollman of Ichuli Institute were indispensable to the data collection for this study. This project would not have been possible without the efforts of Anne Alum, Patrick Engola, Craig Esbeck, Jimmy Mwoci, James Odongo, JB Opeto, and the rest of the Mango Tree Uganda staff, who developed and carried out the NULP intervention. We also thank the students, parents, and teachers from our study schools in northern Uganda. We are grateful for funding from DFID/ESRC Raising Learning Outcomes Grant ES/M004996/2, Wellspring, and the International Growth Centre. The original data collection for this project is registered with the AEA RCT Registry under registration number AEARCTR-0000021. The usual disclaimer applies. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Julie Buhl-Wiggers & Jason T. Kerwin & Juan Muñoz-Morales & Jeffrey Smith & Rebecca Thornton, 2022. "Some children left behind: Variation in the effects of an educational intervention," Journal of Econometrics, .