NATIONAL BUREAU OF ECONOMIC RESEARCH
NATIONAL BUREAU OF ECONOMIC RESEARCH

Site Selection Bias in Program Evaluation

Hunt Allcott

NBER Working Paper No. 18373
Issued in September 2012
NBER Program(s):   EEE   LS

"Site selection bias" can occur when the probability that a program is adopted or evaluated is correlated with its impacts. I test for site selection bias in the context of the Opower energy conservation programs, using 111 randomized control trials involving 8.6 million households across the U.S. Predictions based on rich microdata from the first ten replications substantially overstate efficacy in the next 101 sites. Several mechanisms caused this positive selection. For example, utilities in more environmentalist areas are more likely to adopt the program, and their customers are more responsive to the treatment. Also, because utilities initially target treatment at higher-usage consumer subpopulations, efficacy drops as the program is later expanded. The results illustrate how program evaluations can still give systematically biased out-of-sample predictions, even after many replications.

You may purchase this paper on-line in .pdf format from SSRN.com ($5) for electronic delivery.

Information about Free Papers

You should expect a free download if you are a subscriber, a corporate associate of the NBER, a journalist, an employee of the U.S. federal government with a ".GOV" domain name, or a resident of nearly any developing country or transition economy.

If you usually get free papers at work/university but do not at home, you can either connect to your work VPN or proxy (if any) or elect to have a link to the paper emailed to your work email address below. The email address must be connected to a subscribing college, university, or other subscribing institution. Gmail and other free email addresses will not have access.

E-mail:

This paper was revised on September 15, 2014

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w18373

Users who downloaded this paper also downloaded these:
Chaves, Engerman, and Robinson w19673 Reinventing the Wheel: The Economic Benefits of Wheeled Transportation in Early British Colonial West Africa
Gruber w11377 Religious Market Structure, Religious Participation, and Outcomes: Is Religion Good for You?
Barro and McCleary w9682 Religion and Economic Growth
Duflo and Hanna w11880 Monitoring Works: Getting Teachers to Come to School
Brueckner and Neumark w16797 Beaches, Sunshine, and Public-Sector Pay: Theory and Evidence on Amenities and Rent Extraction by Government Workers
 
Publications
Activities
Meetings
NBER Videos
Data
People
About

Support
National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email: info@nber.org

Contact Us