NATIONAL BUREAU OF ECONOMIC RESEARCH
NATIONAL BUREAU OF ECONOMIC RESEARCH
loading...

Backtesting Strategies Based on Multiple Signals

Robert Novy-Marx

NBER Working Paper No. 21329
Issued in July 2015
NBER Program(s):Asset Pricing

Strategies selected by combining multiple signals suffer severe overfitting biases, because underlying signals are typically signed such that each predicts positive in-sample returns. “Highly significant” backtested performance is easy to generate by selecting stocks on the basis of combinations of randomly generated signals, which by construction have no true power. This paper analyzes t-statistic distributions for multi-signal strategies, both empirically and theoretically, to determine appropriate critical values, which can be several times standard levels. Overfitting bias also severely exacerbates the multiple testing bias that arises when investigators consider more results than they present. Combining the best k out of n candidate signals yields a bias almost as large as those obtained by selecting the single best of nk candidate signals.

download in pdf format
   (960 K)

email paper

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w21329

Users who downloaded this paper also downloaded* these:
D'Erasmo, Mendoza, and Zhang w21574 What is a Sustainable Public Debt?
Romer and Romer w21021 New Evidence on the Impact of Financial Crises in Advanced Countries
Novy-Marx w20984 Fundamentally, Momentum is Fundamental Momentum
Novy-Marx w20591 Understanding Defensive Equity
Bems and Johnson w21070 Demand for Value Added and Value-Added Exchange Rates
 
Publications
Activities
Meetings
NBER Videos
Themes
Data
People
About

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email: info@nber.org

Contact Us