NATIONAL BUREAU OF ECONOMIC RESEARCH
NATIONAL BUREAU OF ECONOMIC RESEARCH

To Hold Out or Not to Hold Out

Frank Schorfheide, Kenneth I. Wolpin

NBER Working Paper No. 19565
Issued in October 2013
NBER Program(s):   DEV   ED   EFG   HE   LS   ME   PE

A recent literature has developed that combines two prominent empirical approaches to ex ante policy evaluation: randomized controlled trials (RCT) and structural estimation. The RCT provides a "gold-standard'' estimate of a particular treatment, but only of that treatment. Structural estimation provides the capability to extrapolate beyond the experimental treatment, but is based on untestable assumptions and is subject to structural data mining. Combining the approaches by holding out from the structural estimation exercise either the treatment or control sample allows for external validation of the underlying behavioral model. Although intuitively appealing, this holdout methodology is not well grounded. For instance, it is easy to show that it is suboptimal from a Bayesian perspective. Using a stylized representation of a randomized controlled trial, we provide a formal rationale for the use of a holdout sample in an environment in which data mining poses an impediment to the implementation of the ideal Bayesian analysis and a numerical illustration of the potential benefits of holdout samples.

The NBER Bulletin on Aging and Health provides summaries of publications like this.  You can sign up to receive the NBER Bulletin on Aging and Health by email.

You may purchase this paper on-line in .pdf format from SSRN.com ($5) for electronic delivery.

Information about Free Papers

You should expect a free download if you are a subscriber, a corporate associate of the NBER, a journalist, an employee of the U.S. federal government with a ".GOV" domain name, or a resident of nearly any developing country or transition economy.

If you usually get free papers at work/university but do not at home, you can either connect to your work VPN or proxy (if any) or elect to have a link to the paper emailed to your work email address below. The email address must be connected to a subscribing college, university, or other subscribing institution. Gmail and other free email addresses will not have access.

E-mail:

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w19565

Users who downloaded this paper also downloaded these:
Scott and Varian w19567 Bayesian Variable Selection for Nowcasting Economic Time Series
Estrada, Gali, and López-Salido w19561 Patterns of Convergence and Divergence in the Euro Area
Chakrabati, Lee, van der Klaauw, and Zafar Household Debt and Saving during the 2007 Recession
Clark, Morrill, and Vanderweide w19566 The Effects of Retiree Health Insurance Plan Characteristics on Retirees’ Choice and Employers’ Costs
de Macedo, Brites Pereira, Martins, and Jalles w19575 Globalization, Democracy and Development
 
Publications
Activities
Meetings
NBER Videos
Data
People
About

Support
National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email: info@nber.org

Contact Us