Evaluating Energy Efficiency Policies

One of the great mysteries in environmental and energy economics is what's called the "energy efficiency gap." Since the 1980s, a series of influential analyses has constructed energy efficiency cost curves — engineering estimates of the costs of conserving energy. These engineering analyses consistently find that individuals and firms fail to adopt significant privately profitable energy efficiency investments. For example, a widely publicized study by McKinsey & Company found that the U.S. economy could reduce energy demand by 23 percent through privately profitable investments that have a net present value of $700 billion.¹ These findings are closely related to "takeup problems" in other areas, such as "Why don't more farmers use fertilizer and high-yielding variety seeds?" and "Why don't firms adopt privately profitable management technologies?"

If these results are correct, improving energy efficiency presents a remarkable "win-win opportunity" to both lower energy costs and reduce emissions of carbon dioxide and other pollutants. Policymakers have seized on this argument, and there was a remarkable expansion of energy efficiency policy over the past decade: the Bush and Obama administrations both tightened fuel economy standards and appliance energy efficiency standards, and more than half of states have now passed Energy Efficiency Resource Standards that require utilities to run energy conservation programs.

This argument raises two questions. First, for privately profitable energy efficiency investments to remain unadopted, there must be some market failure(s). What are these market failures, and how large are they? An alternative explanation for low adoption of seemingly profitable investments is that the investments are in fact not profitable, and that the engineering analyses by McKinsey and others overstate private net benefits. A further question is: Are the energy efficiency policies now in place well-designed to address the market failures? In this summary, I describe my research on these and other questions, much of it done with a great group of collaborators and colleagues.²

The 'Consumer Protection' Rationale for Energy Efficiency Policy

In addition to concern about environmental externalities, policymakers often use a "consumer protection," or "paternalistic," rationale for energy efficiency policy, suggesting that imperfect information and "behavioral" mistakes could explain why consumers don't take up privately profitable energy efficiency investments. One example of the argument is from the U.S. government's 2010 Regulatory Impact Analysis for Corporate Average Fuel Economy (CAFE) standards:

In 2007, Ian Parry, Margaret Walls, and Winston Harrington described the state of knowledge on these potential behavioral biases: "Unfortunately, there is little in the way of solid empirical (as opposed to anecdotal) evidence on this hotly contested issue."⁴

Since then, three empirical strategies have been used to measure systematic consumer "mistakes" in purchases of energy-using durables such as cars, air conditioners, and lightbulbs. These strategies have close connections to behavioral economics work in other domains, such as tax salience, health, and retirement savings.⁵

The first strategy builds on the insight that, absent credit constraints, consumers should care only about a good's total user cost, not the share of that cost that comes from purchase price versus energy costs versus other costs. For example, consumers should be indifferent between a $1 increase in purchase price and a $1 increase in present discounted energy costs. A seminal 1979 paper by Jerry Hausman tests this indifference condition using a cross-sectional discrete choice model.⁶ One problem with Hausman's paper and many subsequent analyses is that more expensive or higher fuel economy cars could have different unobserved characteristics, which would bias the comparison of vehicle price and energy cost elasticities. Several papers, including one that I wrote with Nathan Wozny, have made progress on this issue by studying used-vehicle markets.⁷ When gas prices increase, low fuel economy vehicles should lose value relative to high fuel economy vehicles because the present value of future fuel costs increases more. Using estimates of vehicle lifetimes, utilization, and discount rates, we can predict how much the relative price of, say, a three-year-old Honda Civic DX should decrease relative to, say, a five-year-old Honda Civic Hybrid if consumers fully value fuel costs. We tested this prediction using data from 86 million used vehicle transactions from 1999 to 2008. Used vehicle prices were sharply responsive to gasoline prices, but slightly less than our model predicted, suggesting that consumers slightly undervalued fuel costs.

A second empirical strategy is to measure the effect of energy cost information on demand. If an information intervention has significant effects, this suggests that consumers would be imperfectly informed or inattentive in the absence of the intervention. On the other hand, if information has no effect, this suggests that imperfect information and inattention do not systematically affect demand. Dmitry Taubinsky and I formalized a model of consumer misoptimization and implemented two randomized experiments to identify the necessary parameters for welfare analysis.⁸ We found that consumers are at most moderately inattentive or misinformed. In our model, while a $2 to $3 subsidy for energy-efficient lightbulbs increases welfare, a ban on traditional incandescents does not. Christopher Knittel and I extended this approach with two field experiments with new vehicle buyers. In both experiments, we found no effect of fuel economy information on the fuel economy of vehicles purchased, with standard errors tight enough to rule out economically meaningful systematic inattention or misinformation.⁹

A third empirical strategy for measuring "mistakes" is to measure consumers' beliefs directly and compare them to an objective benchmark. To do this, I implemented a large, nationally representative survey that elicited beliefs about gas costs for the vehicles that people currently own and for other vehicles. I combined the elicited beliefs with choice data to estimate a structural demand model, then used the model to predict differences in market outcomes and welfare in the absence of belief errors. In the data, consumers have at most a small systematic bias in their perceptions of fuel cost savings from higher fuel economy vehicles, and welfare losses are thus small.¹⁰

This body of research suggests two conclusions. First, the optimal energy efficiency policies calibrated with the empirical estimates discussed above are not very stringent relative to some policies currently in place. For example, Sendhil Mullainathan, Taubinsky, and I develop a formal model of optimal taxation with misoptimizing consumers along with a simulation model of the auto market. In our model, the optimal fuel economy standards are less stringent than the standards currently in place.¹¹ Knittel and I find similar results in a more stylized model. Second, if consumers have heterogeneous information or bias, it is important to consider the targeting of energy efficiency policy. Knittel, Taubinsky, and I show that adopters of major energy efficiency subsidies tend to be more informed about and attentive to energy costs than non-adopters, implying that better-targeted policies might generate larger welfare gains.¹²

Evaluating Energy Conservation Nudges

In recent years, interest in "behavior-based" energy conservation programs has increased significantly. In this context, "behavior-based" refers to using approaches from applied psychology, such as goal setting and social comparisons, to encourage energy conservation. Interest in such approaches is not limited to energy efficiency: they are also used to encourage smoking cessation, healthy eating, retirement savings, charitable giving, and other choices thought to have individual or social benefits.

Perhaps the most salient example of behavior-based energy conservation is the Home Energy Report, a letter that compares a household's energy use with that of its neighbors and provides energy conservation tips. As a measure of the program's importance, the leading Home Energy Report provider, Opower, works with about 100 utilities, sending Home Energy Reports to 15 million households. In most programs, people receive Home Energy Reports every month or every few months over several years.

Several academic papers, including one that I wrote, evaluate early Home Energy Report programs.¹³ In my first paper on this topic, I studied the first 10 Home Energy Report programs, finding that they were highly cost effective. Relative to traditional conservation programs like weatherization subsidies, they caused more conservation at less cost to the utility.

In subsequent work, I have addressed additional questions about these programs. First, would the program's initial evaluation results generalize to other sites? This extrapolation problem is of course fundamental to empirical work, regardless of the exact setting. In a 2015 paper, I analyze results from the 101 sites that followed the first 10.¹⁴ I show that there had been "site selection bias": early sites were selected from later sites through mechanisms correlated with the treatment effect, some of which could be explained through intuitive observable mechanisms, and some of which reflected selection on unobservables. Just as individuals endogenously select into treatment in the absence of random assignment, these results show how sites endogenously select into evaluations. This paper is of interest in the program evaluation literature because it shows that even many replications may not be enough to make correct policy implementation decisions. In some cases, either we need an evaluation in a fully representative population, or we need to focus on theoretical insights that might be more generalizable than a treatment effect estimate.

Second, to what extent are the results driven by malleable attention? Using hundreds of millions of observations of daily electricity-use data, Todd Rogers and I show that responses to repeated Home Energy Reports are consistent with a "cue theory" or time-varying persuasive advertising model: the reports draw attention to energy conservation for about 10 days, after which the effect decays until the next report is received.¹⁵ Eventually, consumers begin to change their capital stock, and the treatment effects become persistent even after the intervention is discontinued. Rogers and I were not able to definitively measure the extent to which the capital stock changes reflected new physical capital investments versus different utilization habits. A more recent paper shows that the bulk of these changes were in fact physical capital.¹⁶

Third, what are the program's social welfare effects? "Nudges" in many domains are evaluated using cost effectiveness metrics — how much did the program cost to implement, and how much did behavior change? — instead of social welfare assessment. Many economists have questioned whether such interventions are truly welfare enhancing; Edward Glaeser and others have argued that some nudges are "emotional taxes" that guilt individuals into behavior change without the benefit of raising revenue.¹⁷ Home Energy Reports are perhaps the ideal setting to evaluate welfare effects of a "nudge" intervention, because they are a private good that can be sold, allowing us to use experienced recipients' willingness to pay as a measure of consumer welfare effects. In partnership with Opower and a partner utility, Judd Kessler and I sold future Home Energy Reports to thousands of prior recipients using an incentive-compatible multiple price list.¹⁸ We combine willingness to pay with the value of externality reduction in a full social welfare evaluation and find that while the program increased welfare, traditional evaluations substantially overstate welfare gains.

Measuring Welfare Effects

One theme that connects several of the above papers is the selective use of revealed preferences to carry out welfare analyses in situations with potential informational or behavioral market failures. I continue this line of thought in research with Michael Greenstone.¹⁹ This paper empirically quantifies two concerns with energy efficiency cost curve analyses such as the aforementioned McKinsey study. First, we show that the engineering models substantially overstate actual, empirically estimated energy savings in our context. Our work, along with related research by Meredith Fowlie, Greenstone, and Catherine Wolfram, suggests that findings that consumers fail to adopt seemingly profitable energy efficiency investments may be at least partially explained by the investments not being profitable, not by market failures that reduce adoption.²⁰ Second, we use investment takeup data to show that energy efficiency investments entail substantial non-monetary costs and benefits that the engineering analyses ignore. We combine experimental and quasi-experimental data in a simple structural model to measure the welfare effects of a large federally funded energy efficiency program. In the context of our model, the program reduces welfare.