The Human Perils of Scaling Smart Technologies: Evidence from Field Experiments
Smart-home technologies have been heralded as an important way to increase energy conservation. While in vitro engineering estimates provide broad optimism, little has been done to explore whether such estimates scale beyond the lab. We estimate the causal impact of smart thermostats on energy use via two novel framed field experiments in which a random subset of treated households have a smart thermostat installed in their home. Examining 18 months of associated high-frequency data on household energy consumption, yielding more than 16 million hourly electricity and daily natural gas observations, we find little evidence that smart thermostats have a statistically or economically significant effect on energy use. We explore potential mechanisms using almost four million observations of system events including human interactions with their smart thermostat. Results indicate that user behavior dampens energy savings and explains the discrepancy between estimates from engineering models, which assume a perfectly compliant subject, and actual households, who are occupied by users acting in accord with behavioral economists’ conjectures. In this manner, our data document a keen threat to the scalability of new user-based technologies.
We thank Dylan Brewer, Fiona Burlig, Ken Gillingham, Michael Greenstone, Koichiro Ito, David Jimenez-Gomez, Juanna Joensen, David Novgorodsky, Mar Reguant, Florian Rundhammer, CaseyWichman, and seminar participants at the Advances with Field Experiments Conference, Heartland Environmental and Resource Economics Workshop, Federal Communications Commission, EPIC Faculty Workshop, NBER Environmental and Energy Economics Summer Institute, and Harvard Seminar in Environmental Economics and Policy for helpful comments and suggestions. Lillian Bartholomew, Ariel Listo, David Liu, Uditi Karna, and Jack Ogle provided excellent research assistance. We would like to thank Richard Caperton, Augustin Fonts, Ilan Frank, Kevin Hamilton, Nicholas Payton, Jim Kapsis, and many others at Opower for sharing data and offering insights. Opower provided the data analyzed in this paper to the authors under a nondisclosure agreement. The authors and Opower structured the agreement in a way that maintains the authors’ independence. In particular, the agreement stipulates that Opower has the right to review the publication prior to public release solely for factual accuracy. None of the authors were paid by Opower for this research. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.