The 'CAPS' Prediction System and Stock Market Returns

Featured in print Digest

Shorting stocks with a disproportionate number of negative picks ... and buying stocks with a disproportionate number of positive picks produces a return of over 9 percent per annum.

In The 'CAPS' Prediction System and Stock Market Returns (NBER Working Paper No. 17298), co-authors Christopher Avery, Judith Chevalier, and Richard Zeckhauser study the predictive power of approximately 2.5 million stock predictions submitted by individual users to the 'CAPS' website run by the Motley Fool company. The data used in this analysis spans the time period between November 2006 and December 2008, a period with significant swings in stock market performance.

In the past, using different data sets, researchers have found that individuals perform poorly as stock market investors, except when they concentrate their portfolios on stocks for which they have an informational advantage. And, while internet trading and message boards have facilitated trading, there is no evidence that those boards predict performance of the stocks. But the CAPS data differ from internet trading or online prediction markets in three ways: First, participants make precise predictions about future price, rather than simple buy/sell/hold recommendations. Second, the website provides a rating of participants by scoring their reputation. And finally, CAPS synthesizes the history of past picks to produce a rating of each stock - on a 5-star scale.

The authors analyze the informational content of the CAPS picks by tracking the performance of portfolios formed on the basis of positive and negative picks (that is, predictions of increases and decreases in the prices of individual stocks, respectively). A preliminary look at the relationship between individual picks in the CAPS system and subsequent stock market returns shows some interesting facts. For example, on average CAPS participants -- like most stock market analysts -- have been relatively bullish, producing a ratio of about five positive picks per negative pick. Second, the relationship between returns for positive versus negative picks varies very little by market cap. Third, averaging across the whole time period, 5-star stocks outperformed 1-star stocks by 9 percentage points (although removing the height of the financial crisis increases the difference in returns between 5-star and 1-star to 14.6 percentage points).

Most interestingly, these picks prove to be surprisingly informative about future stock prices. Although the return from investing in the positive-pick portfolio would have been negative over the course of the study period, the Motley Fool participants' positive picks systematically outperformed the negative picks. Indeed, a strategy of shorting stocks with a disproportionate number of negative picks on the site and buying stocks with a disproportionate number of positive picks produces a return of over 9 percent per annum over the sample period.

These results are mostly driven by the fact that negative picks on the site strongly predict future stock price declines, while positive picks on the site produce returns that are statistically indistinguishable from the market. The authors posit that it may not be surprising that social investing websites are more successful at predicting abnormally negative future stock performance than they are at predicting abnormally positive future stock performance, because acting on negative information about the prospects for a stock can be more costly and difficult than acting on positive information about the prospects for a stock. But the differences in returns between stocks ranked highly and stocks ranked poorly might be attributable to inherent differences in their characteristics, such as differences in risk, in market cap, or in past performance. Controlling for those factors, the authors find that differences in return are mostly due to stock picking.

--Claire Brunel