Education Program Meeting

November 11 and 12, 2010 -
Caroline M. Hoxby of Stanford University and NBER, Organizer

Sheetal Sekhri, University of Virginia
Affirmative Action and Peer Effects: Evidence from Caste-Based Reservation in General Education Colleges in India

Proponents of affirmative action policies in higher education argue that the beneificiaries of affirmative action could gain academically from positive peer effects, whereas critics argue that they could fall behind because of competition with better prepared peers. Sekhri examines this hypothesis in the context of caste-based affirmative action in college admissions in India. Admission to general education public colleges in India is strictly based on the results of the Senior Secondary School examinations. This rule generates quasi-random variation in the peer quality of the students admitted. Sekhri estimates peer effects using a unique dataset that links admission data with the educational outcomes on college exit exams, and finds that better average quality of the high-caste students has a negative effect on the performance of the low-caste students. The peer quality of the low-caste students also negatively affects high-caste students, with more pronounced effects on high achievers. These findings suggest that integrated college environments may not necessarily help the intended social beneficiaries of affirmative action.


Karthik Muralidharan, University of California, San Diego and NBER; Jishnu Das and Venkatesh Sundararaman, The World Bank; Stefan Dercon, Oxford University; James Habyarimana, Georgetown University; and Pramila Krishnan, University of Cambridge
When Can School Inputs Improve Test Scores?

While the relationship between school inputs and learning outcomes is critical for education policy, existing empirical studies of this relationship typically do not account for the fact that households will respond to changes in school inputs. Muralidharan, Das, Dercon, Habyarimana, Krishnan, and Sundararaman present a dynamic household-optimization model relating cognitive achievement to school and household inputs, and test its predictions in two very different developing country settings: Zambia and India. A key contribution of this paper is the ability to measure household spending changes and student test-score gains in response to both unanticipated and anticipated changes in school funding. Consistent with the optimization model, the researchers find in both settings that households offset anticipated grants more than unanticipated grants, and that unanticipated school grants lead to significant improvements in student test scores, while anticipated grants have no impact on cognitive achievement. These results suggest that 1) school grant programs in developing countries are likely to have high rates of pass through to households, and 2) naive estimates of public education spending on learning outcomes that do not account for optimal household responses are likely to be considerably biased if used to estimate parameters of an education production function.


Steven F. Lehrer, Queen's University and NBER, and Weili Ding, Queen's University
Estimating Context-Independent Treatment Effects in Education Experiments

Lehrer and Ding first document that the magnitude of the estimated treatment effect in Project STAR is substantially larger in schools where fewer students are assigned to small classes. These differences in student performance that exist in multiple subject areas across schools cannot be explained by failure in randomization, other observed school-level characteristics, or differences in selective test taking. Furthermore, these achievement gains are driven exclusively by students in small classes from schools where fewer students are in small classes. These results suggest that there was a proportionate change in motivation or effort by teachers who teach small classes, but not by those in regular classes. The researchers then introduce an empirical strategy for experimental studies that aims at disentangling the pure educational effect of a specific treatment from what is attributable to the interaction between the treatment and the social context of the experiment. They disentangle the estimated treatment effect into components that are context-specific and context-independent and show that between 50 and 70 percent of the estimated treatment effect in Project STAR is context-specific.


Jane Cooley and Jeffrey Traczynski, University of Wisconsin
Spare the Rod? The Effect of Sanctions on Schools

Under No Child Left Behind, schools face sanctions if they repeatedly fail to meet proficiency targets. Cooley and Traczynski explore the effect of sanctions on low-performing schools. Using data on North Carolina public elementary schools, they use both a student-level difference-in-difference and a school-level regression discontinuity approach to identify the effect of sanctions. They find evidence that school sanctions have a positive effect on student performance. The distributional effects appear to vary with the level and type of sanctions. They also find that schools respond to pressure to improve the performance of failing subgroups of students.


Stephanie Riegg Cellini, George Washington University, and Latika Chaudhary, Scripps College
The Labor Market Returns to a Private Two-Year College Education

Cellini and Chaudhary offer the first estimates of the earnings gains to private two-year and less-than-two-year colleges, the vast majority of which are for-profit institutions. Using data from both the 1997 and 1979 panels of the National Longitudinal Survey of Youth (NLSY) to control for time-invariant unobservable characteristics of students, they find that students who complete associates degrees in private two-year colleges generate earnings gains of at least 13-14 percent, or 7 percent per year of education, which is similar to the gains experienced by public community college students in the sample. The estimates suggest that private-sector students may earn an additional 3 percentage points per year over students in the public sector, although these effects are not statistically significant. Further, students who complete associates degrees in either sector are about 10 percent more likely to be employed. Among a broader set of students who enroll in, but may or may not complete an associate's degree, there is a small decline in earnings but a higher likelihood of employment for students attending both types of institutions.


Patrick Wolf and Brian Kisida, University of Arkansas; Babette Gutmann and Lou Rizzo, Westat; Michael Puma, Chesapeake Research Associates; and Nada Eissa, Georgetown University and NBER
Evaluation of the DC Opportunity Scholarship Program: Experimental Impacts after at Least Four Years

The District of Columbia School Choice Incentive Act of 2003, passed by the Congress in January 2004, established the first federally funded, private school voucher program in the United States. The purpose of the scholarship program was to provide low-income parents, particularly those whose children attend schools identified for improvement or corrective action under the Elementary and Secondary Education Act, with "expanded opportunities to attend higher performing schools in the District of Columbia." Wolf and his co-authors compare the outcomes of 2,300 eligible applicants randomly assigned to receive an offer (treatment group) or not receive an offer (control group) of an OSP scholarship through a series of lotteries. Data on outcomes-test scores, high school graduation, perceptions of school safety, and satisfaction-were collected annually over four or five years. Based on analysis of the final spring 2009 data, the researchers find that the Program significantly improved students' chances of graduating from high school. The offer of an OSP scholarship raised students' probability of completing high school by 12 percentage points overall. On average, after at least four years students who were offered (or used) scholarships had reading scores that were statistically higher than those who were not offered scholarships, while math scores were statistically similar to those who were not offered scholarships. Additionally, the OSP raised parents', but not students', ratings of school safety and satisfaction. Parents were more satisfied and felt school was safer if their child was offered or used an OSP scholarship. The Program had no effect on students' reports on school conditions.

Cristian Pop-Eleches and Miguel Urquiola, Columbia University and NBER
Going to a Better School: Effects and Behavioral Responses

Pop-Eleches and Urquiola estimates the effect of going to a better school on students' academic achievement and explore whether this intervention induces behavioral responses on the part of children, their parents, and the school system. They first exploit almost 2,000 regression-discontinuity quasi-experiments observed in the context of Romania's high school educational system. Then they use data from a specialized survey of children, parents, teachers, and principals that they implemented in 59 Romanian towns. They find that students do benefit from access to higher achieving schools and tracks within schools. Further, the stratication of schools by quality in general, and the opportunity to attend a better school in particular, result in significant behavioral responses on the part of teachers, parents, and students. These results have a number of implications for evaluation, particularly because some of them change over time, and some would seem to be relevant only once interventions reach a certain scale.


C. Kirabo Jackson, Northwestern University and NBER
Peer Quality or Input Quality? Evidence from Trinidad and Tobago

Using exogenous secondary school assignments to remove self-selection bias to schools and peers, Jackson is able to obtain credible estimates of: 1) the effect of attending schools with higher-achieving peers; and 2) the direct effect of peer quality improvements within schools, on the same population. Students at schools with higher achieving peers have better academic achievement overall. But within-school increases in peer achievement only improve outcomes at high-achievement schools. Peer quality explains about one tenth of school value-added on average, but over one-third of value-added among the top quartile of schools. The results also vary widelyby gender.


Jeffrey Groen, Bureau of Labor Statistics
Time to the Doctorate and the Labor Market for New PhD Recipients

Training time for a doctorate varies widely across students within a given field. Groen considers how the labor market for new Ph.D. recipients may influence the time to the doctorate. For graduates in a given field, the demand-side of the labor market varies from year to year. This variation affects the opportunity cost of remaining a student, and thus the incentive for students to complete their degrees. Groen uses micro data from the Survey of Earned Doctorates, together with annual counts of job listings from 1975 to 2005 in seven fields in the humanities and social sciences, and finds a positive and statistically significant effect of the job market on the probability of completion (in a given year). His estimates imply that permanently increasing the number of job listings in a field by 10 percent reduces expected time to degree by 0.26 years and increases the cumulative probability of completing the degree within eight years by 3 percentage points. Simulations using the estimates reveal that the observed time-series variation in job listings explains 72 percent of the variation over time in average time to degree within fields.


Pablo Pena, University of Chicago
Randomness and the Measurement of Intergenerational Mobility

Using a standard model of the inter-generational transmission of outcomes, Pena develops a methodology for estimating the inter-generational transmission of ability, taking into account the presence of random shocks to outcomes. His methodology uses outcomes of grandparents as instrumental variables in the regressions of children's outcomes on parents' outcomes. Using survey data on educational attainment for three generations of adults in three sample -- the Health and Retirement Study, the Wisconsin Longitudinal Study, and the Mexican Health and Aging Study -- he finds consistent results: without accounting for random shocks, the estimates of the inter-generational transmission of ability are biased downward and therefore overstate inter-generational mobility.


Pamela Jakiela, Washington University; Edward Miguel, University of California, Berkeley and NBER; and Vera L. te Velde, University of California, Berkeley
You've Earned It: Combining Field and Lab Experiments to Estimate the Impact of Human Capital on Social Preferences

Jakiela, Miguel, and te Velde combine data from a field experiment and a laboratory experiment to measure the causal impact of human capital on respect for earned property rights, a component of social preferences with important implications for economic growth and development. The researchers find that higher academic achievement reduces the willingness of young Kenyan women to appropriate others' labor income, and shifts players toward a 50-50 split norm in the dictator game. This study demonstrates that education may have long-run impacts on social preferences, norms, and institutions beyond the human capital directly produced. It also shows that randomized field experiments can be successfully combined with laboratory experiment data to measure causal impacts on individual values, norms, and preferences which cannot be readily captured in survey data.


Eric S. Taylor, Stanford University, and John H. Tyler, Brown University and NBER
The Effect of Evaluation on Teacher Performance: Evidence from Longitudinal Student Achievement Data of Mid-career Teachers

Measuring and increasing teacher effectiveness have become dominant themes in American education reform movements. The magnitude and measured variation of teacher effectivenes -- variation that is as large within as across schools -- have produced a flurry of policy proposals. But how do we identify more and less effective teachers? Value-added modeling, using student performance data and classroom observation evaluations, are the two methods currently at our disposal. Each has its advocates, and each has demonstrable advantages and disadvantages. Advocates of classroom-observation evaluation suggest that such practice-based evaluation can provide feedback and give professional development direction that can help teachers to become better. To date, however, the effect of evaluation itself on teacher effectiveness is an unanswered empirical question. Taylor and Tyler address this question, asking: do experienced teachers who undergo evaluation in a well developed and rigorous practice-based evaluation system become better teachers as a result of going through the process? Using data from the Cincinnati Public School system, they find that going through Cincinnati's Teacher Evaluation System does increase a teacher's ability to promote student achievement growth as measured by test scores. This effect appears to be non-transitory and is stronger for the weakest teachers and for teachers whose evaluation scores improved the most across the four classroom observations that took place during the year.