A common approach to comparing asset pricing models with traded factors involves a competition between models in pricing test-asset returns. We find that such practice, while seemingly reasonable, cannot be relied on to determine which is the superior model for several widely accepted criteria including statistical likelihood, Sharpe ratios and a modified HJ distance. All that matters for model comparison is the extent to which each model is able to price the factors in the other model. Given this information, test assets are actually irrelevant, whether the models are nested or non-nested.
Thanks to Ken French for comments on an earlier draft. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.