Non est Disputandum de Generalizability? A Glimpse into The External Validity Trial
While empirical economics has made important strides over the past half century, there is a recent attack that threatens the foundations of the empirical approach in economics: external validity. Certain dogmatic arguments are not new, yet in some circles the generalizability question is beyond dispute, rendering empirical work as a passive enterprise based on frivolity. Such arguments serve to caution even the staunchest empirical advocates from even starting an empirical inquiry in a novel setting. In its simplest form, questions of external validity revolve around whether the results of the received study can be generalized to different people, situations, stimuli, and time periods. This study clarifies and places the external validity crisis into perspective by taking a unique glimpse into the grandest of trials: The External Validity Trial. A key outcome of the proceedings is an Author Onus Probandi, which outlines four key areas that every study should report to address external validity. Such an evaluative approach properly rewards empirical advances and justly recognizes inherent empirical limitations.
Some of the people represented in this trial are creations of the mind, as are the phrases and statements that I attribute to the real people in this trial. Specifically, these words do not represent actual statements from the real people, rather these are words completely of my creation. I have done my best to maintain historical accuracy. Thanks to incredible research support from Ariel Listo, Lina Ramirez, Haruka Uchida, and Atom Vayalinkal. Min Sok Lee provided quite helpful information for this study. Working with many coauthors as well as discussions/debates over the years with colleagues helped me discover my voice and whereabouts on this issue. I trust that they will not agree with all of the views expressed in this satirical study. Omar Al-Ubaydli, Alec Brandon, Cody Cook, Aart de Zeeuw, Rebecca Diamond, Seda Ertac, Basil Halperin, Justin Holz, Robert Metcalfe, Magne Mogstad, David Novgorodsky, Michael Price, Matthias Rodemeier, Kerry Smith, and Karen Ye provided quite helpful comments on earlier drafts. The views expressed herein are those of the author and do not necessarily reflect the views of the National Bureau of Economic Research.