Education Program Meeting, Fall 2019

Recent studies document that, in many cases, sought after schools do not improve student test scores. Three explanations are that (a) existing studies identify local average treatment effects that do not generalize to the average student, (b) parents cannot discern schools’ causal impacts, or (c) parents value schools that improve outcomes not well measured by test scores. To shed light on this, Jackson and Beuermann employ administrative and survey data from Barbados. Using discrete choice models, they document that most parents have strong preferences for the same schools. Using a regression-discontinuity design, the researchers estimate the causal impact of attending a preferred school on a broad array of outcomes. As found in other settings, preferred schools have better peers, but do not improve short-run test scores. The researchers implement a new statistical test and find that this null effect is not due to school impacts being different for marginal students than for the average student. Looking at longer-run outcomes, for females, preferred schools reduce teen motherhood, increase educational attainment, increase earnings, and improve health. In contrast, for males, the effects are mixed. The pattern for females is consistent with parents valuing school impacts on outcomes not well measured by test scores, while the pattern for males is consistent with parents being unable to identify schools’ causal impacts. The results indicate that test-scores impacts may be an incomplete measure of school quality.

Many US workers receive a large portion of their lifetime compensation in the form of retirement pensions. How do changes in pensions vis à vis salaries affect labor supply and retirement? Biasi examines the retirement responses to a reform that changed salaries and pensions of public school teachers in a staggered fashion. On one hand, the reform lowered older teachers' gross salaries and, in turn, their future pension benefits, on the other it increased employees' contributions to the pension fund, lowering net salaries but leaving pensions unchanged. Biasi uses the staggered timing of implementation of these two provisions to estimate bounds to the income and substitution effects of salaries and pensions. These estimates suggest large substitution effects and more moderate income effects. They also indicate that workers are more responsive to changes in salaries than to equally sized changes in pensions. Biasi finds support for three possible explanations for this finding: a) a lack of salience/information on pensions, and b) credit constraints. Biasi use the estimated elasticities to evaluate the effect of an alternative budget-saving policy that reduces pensions instead of net salaries. This alternative policy would lead to fewer, older, and lower-quality teachers retiring compared with the actual reform.

Oreopoulos and Petronijevic present results from a five-year effort to design promising online and text-message interventions to improve college achievement through several distinct channels. From a sample of nearly 25,000 students across three different campuses, the researchers find some improvement from coaching-based interventions on mental health and study time, but none of the interventions evaluated significantly influences academic outcomes (even for those students more at risk of dropping out). The researchers interpret the results with their survey data and a model of student effort. Students study about five to eight hours fewer each week than they plan to, though the interventions do not alter this tendency. The coaching interventions make some students realize that more effort is needed to attain good grades but, rather than working harder, they settle by adjusting grade expectations downwards. The study time impacts are not large enough for translating into significant academic benefits. More comprehensive but expensive programs appear more promising for helping college students outside the classroom.

Do students respond to sticker prices or actual prices when applying to college? These costs differ for students eligible for financial aid. Students who do not understand this may not apply to some colleges because of the perceived high cost. We test for this form of “sticker shock” using College Board data on SAT scores sent, as a proxy for applications, to state flagship institutions for students entering college in 2006–2013. Some public flagships guarantee financial aid will meet full financial need. Sticker price increases at those schools would not affect the actual cost after factoring in financial aid and should not affect decisions for those eligible for aid. We exploit the large and variable increases in sticker prices at public flagships during the financial crisis generated from state budget shortfalls. We also control for local labor market conditions to abstract from the recession’s impact on individual educational decisions. We find evidence of sticker shock — students unaffected by sticker price increases still apply less often. Using data from the National Student Clearinghouse, we also find that price increases at public flagships reduce enrollment of high achieving students, regardless of financial aid status, who often choose private colleges instead.

Brunner, Dougherty, and Ross examine the effect of admission to 16 stand-alone technical high schools within the Connecticut Technical High School System (CTHSS) on student educational and labor market outcomes. To identify the causal effect of admission on student outcomes, the researchers exploit the fact that CTHSS utilizes a score-based admissions system and identify the effect of admission using a regression discontinuity approach. They find that male students attending one of the technical high schools are approximately 10 percentage points more likely to graduate from high school and 8 percentage points less likely to attend college, although there is some evidence that the negative effects on college attendance fade over time. The researchers also find that male students attending a technical high school have quarterly earnings that are approximately 31% higher. Analyses of potential mechanisms behind these results reveal that male students that attend a technical high school have higher 9th grade attendance rates and higher 10th grade test scores. The researchers find little evidence that attending a technical high school affects the educational or labor outcomes of women. These effects appear relatively broad based across different types of students in that the researchers find little evidence of heterogeneity in these effects over student attributes like race and ethnicity, free lunch eligibility or residence in a poor, central city school district. However, when distinguishing between students based on the Career and Technical Education (CTE) offerings of the high school that these students likely would have attended, the researchers find that the effects of admission to a CTHSS school are noticeably larger when the counterfactual high school has less CTE offerings.

Financing college expenses through an income share agreement (ISA) is an arrangement where the student agrees to pay a fixed percentage of his or her future earned income for a fixed period of time. Using Purdue University administrative and survey data for 442 ISA participants and 418 non-participants who applied and were offered an ISA but chose an alternative source of college funding, Mumford estimates the adverse selection into the ISA and the moral hazard caused by the ISA. The identification of selection into the ISA is based on being able to observe the full set of interested and eligible students due to the way ISA terms were provided. The evidence suggests that there is no adverse selection by student ability, demographics, and a variety of other student characteristics. The evidence that selection into the ISA is not correlated with the observables motivates the identification strategy for estimating the effect of ISA participation on grades earned and starting salary after graduation. The results show no effect on grades earned but $3,000 to $5,000 lower starting salary.

Using transcript data from a large 4-year public university, Li and Zafar show that male students are 18.6 percent more likely than female students to receive favorable grade changes initiated by instructors. These gender differences cannot be explained by observable characteristics of the students, instructors, and the classes. To understand the mechanisms underlying these gendered outcomes, the researchers conduct surveys of students and teachers, which reveal that not only are regrade requests prevalent but that male students are also more likely than female students to ask for regrades on the intensive margin. Finally, the researchers corroborate the gender differences in regrade requests in an incentivized controlled experiment where participants receive noisy signals of their performance, and where they can ask for regrades: the researchers find that males have a higher willingness to pay (WTP) to ask, both in cases where it makes economic sense to do so and where it does not. About a third of the gender difference in the WTP is due to gender differences in under-confidence, uncertainty in prior beliefs about ability, and the Big Five personality traits.

Recent research in many fields of social science makes extensive use of administrative data, such as from US states, counties, or school districts. Recent work on the labor market consequences of post-secondary education, in particular, have used administrative data from institutions matched to in-state earnings data. However, few of these papers have the ability to follow workers outside of the state, which could bias measured effects on earnings. Similar problems arise outside the US, when workers migrate across countries. While most researchers acknowledge the issue, they are unable to quantify the effect that non-random attrition has on their results. In addition to these academic papers, a number of states and countries have produced and publicized average earnings of graduates to inform students. Using new data merging college records with both in-state and national earnings from the LEHD, Foote and Stange document how earnings estimates are biased in practice. They also document how this differs by field of study and college selectivity, as well as the extent to which attrition is differential across the earnings distribution. The researchers find that out-of-state migration is particularly problematic for high-earners, flagship graduates, and computer science majors and grows with time since graduation. In the empirical example, the researchers find that the effect of graduating from a flagship university (relative to less selective public 4-year) is 26% higher than one would estimate using in-state earnings exclusively. Various approaches to testing for and bounding this bias are considered.

Do nudge interventions that have generated positive impacts at a local level maintain efficacy when scaled state or nationwide? What specific mechanisms explain the positive impacts of promising smaller-scale nudges? Bird, Castleman, Denning, Goodman, Lamberton, and Ochs Rosinger investigate, through two randomized controlled trials, the impact of a national and state-level campaign to encourage students to apply for financial aid for college. The campaigns collectively reached over 800,000 students, with multiple treatment arms to investigate different potential mechanisms. The researchers find no impacts on financial aid receipt or college enrollment overall or for any student subgroups. They find no evidence that different approaches to message framing, delivery, or timing, or access to one-on-one advising affected campaign efficacy. The researchers discuss why nudge strategies that work locally may be hard to scale effectively.

Bergman, Chan, and Kapor shows that imperfect information about school quality causes low-income families to live in neighborhoods with lower-performing, more segregated schools. They randomized the addition of school quality information onto a nationwide provider of online housing listings for families with housing vouchers. The researchers find that this information causes families to move to areas with higher-performing, more racially disparate schools. To understand the value of this information and its implications for models of neighborhood choice, the researchers develop a dynamic model of households' search and location choices that incorporates subjective beliefs and imperfect information about school quality. The researchers use the model to estimate how much families value school quality and how much they would appear to value it if they ignored information frictions. If these information frictions had been ignored, control group families would appear to value school quality half as much, relative to their commute to downtown, as treatment group families. Moreover, the researchers show that control-group households have biased beliefs that underestimate school quality and mispredict it as a function of other neighborhood characteristics.

Neilson, Gallegos, and Calle study policies that use pre-college academic achievement to restrict or incentivize entry to teacher-colleges. First, they use historical records of college entrance exam scores since 1967 in Chile and administrative data on the population of teachers to document a robust positive and concave relationship between several measures of teacher productivity and teachers’ own pre-college academic achievement. The researchers then study the effectiveness of two policies that used pre-college achievement to recruit or screen out students entering teacher-colleges. Using a regression discontinuity design based on the government’s recruitment efforts, the researchers evaluate the effectiveness of targeted scholarships at shifting career choices of high achieving students as well as the effect on the overall stock of teachers predicted effectiveness. The researchers then study the effects of a recent screening policy that forces teacher colleges to exclude below-average students. The researchers retroactively simulate this policy rule and evaluate its success at screening out low performing teachers. They compare this benchmark policy rule to a series of potential data-driven policy rules and find that even simple screening policies can identify a significant portion of ex-post low performing teachers. In both policies studied, screening low performing students is more effective than targeting recruitment efforts to only very high achieving students. These findings suggest that the combination of better data and flexible prediction methods can be used to implement practical screening policies in some contexts.

A common feature of public sector labor markets is the use of pay scales. Murphy, Burgess, and Greaves examines how the removal of pay scales impacts productivity, by exploiting a reform that compelled all schools in England to replace pay scales with school-designed performance related pay schemes. They find that schools in labor markets with better outside options for teachers saw relatively higher increases in teacher pay. Schools in these areas relatively increase their spending on teachers, have higher teacher retention and larger improvements in student tests scores. These effects are largest in schools with the high proportions of disadvantaged students. The researchers conclude that the pay rigidities in the form of centralized pay schedules result in a misallocation of resources, by preventing such schools from retaining their teachers.