Centro de Investigacion Economica
Institutional Affiliation: ITAM
NBER Working Papers and Publications
|December 2019||Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments|
with Karthik Muralidharan, Kaspar Wüthrich: w26562
Factorial designs are widely used for studying multiple treatments in one experiment. While “long” model t-tests provide valid inferences, t-tests using the “short” model (ignoring interactions) yield higher power if interactions are zero, but incorrect inferences otherwise. Of 27 factorial experiments published in top-5 journals (2007--2017), 19 use the short model. After including all interactions, over half their results lose significance. Modest local power improvements over the long model are possible, but with lower power for most values of the interaction. If interactions are not of interest, leaving the interaction cells empty yields valid inferences and global power improvements.
|May 2019||Designing Effective Teacher Performance Pay Programs: Experimental Evidence from Tanzania|
with Isaac Mbiti, Youdi Schipper: w25903
We use a field experiment in Tanzania to compare the effectiveness on learning of two teacher performance pay systems. The first is a Pay for Percentile system (a rank-order tournament). The second rewards teachers based on multiple proficiency thresholds. Pay for Percentile can (under certain conditions) induce optimal effort among teachers, but our threshold system is easier to implement and provides teachers with clearer goals and targets. Both systems improved student test scores. However, the multiple-thresholds system was more effective in boosting student learning and is less costly.
|July 2018||Inputs, Incentives, and Complementarities in Education: Experimental Evidence from Tanzania|
with Isaac Mbiti, Karthik Muralidharan, Youdi Schipper, Constantine Manda, Rakesh Rajani: w24876
We present results from a large-scale randomized experiment across 350 schools in Tanzania that studied the impact of providing schools with (a) unconditional grants, (b) teacher incentives based on student performance, and (c) both of the above. After two years, we find (a) no impact on student test scores from providing school grants, (b) some evidence of positive effects from teacher incentives, and (c) significant positive effects from providing both programs. Most importantly, we find strong evidence of complementarities between the two programs, with the effect of joint provision being significantly greater than the sum of the individual effects. Our results suggest that combining spending on school inputs (which is the default policy) with improved teacher incentives could substanti...
Published: Isaac Mbiti & Karthik Muralidharan & Mauricio Romero & Youdi Schipper & Constantine Manda & Rakesh Rajani, 2019. "Inputs, Incentives, and Complementarities in Education: Experimental Evidence from Tanzania*," The Quarterly Journal of Economics, vol 134(3), pages 1627-1673. citation courtesy of
|July 2017||Incentives for Replication in Economics|
with Sebastian Galiani, Paul Gertler: w23576
Replication is a critical component of scientific credibility as it increases our confidence in the reliability of the knowledge generated by original research. Yet, replication is the exception rather than the rule in economics. In this paper, we examine why replication is so rare and propose changes to the incentives to replicate. Our study focuses on software code replication, which seeks to replicate the results in the original paper using the same data as the original study and verifying that the analysis code is correct. We analyse the effectiveness of the current model for code replication in the context of three desirable characteristics: unbiasedness, fairness and efficiency. We find substantial evidence of “overturn bias” that likely leads to many false positives in terms of “fin...
Published: How to make replication the norm, Nature, Volume 554, 2018, pages 417-419