On the Informativeness of Descriptive Statistics for Structural Estimates
We propose a way to formalize the relationship between descriptive analysis and structural estimation. A researcher reports an estimate ĉ of a structural quantity of interest c that is exactly or asymptotically unbiased under some base model. The researcher also reports descriptive statistics γ̂ that estimate features γ of the distribution of the data that are related to c under the base model. A reader entertains a less restrictive model that is local to the base model, under which the estimate ĉ may be biased. We study the reduction in worst-case bias from a restriction that requires the reader's model to respect the relationship between c and γ specified by the base model. Our main result shows that the proportional reduction in worst-case bias depends only on a quantity we call the informativeness of γ̂ for ĉ. Informativeness can be easily estimated even for complex models. We recommend that researchers report estimated informativeness alongside their descriptive analyses, and we illustrate with applications to three recent papers.
We acknowledge funding from the National Science Foundation (DGE-1654234), the Brown University Population Studies and Training Center, the Stanford Institute for Economic Policy Research (SIEPR), the Alfred P. Sloan Foundation, and the Silverman (1968) Family Career Development Chair at MIT. We thank Tim Armstrong, Matias Cattaneo, Gary Chamberlain, Liran Einav, Nathan Hendren, Yuichi Kitamura, Adam McCloskey, Costas Meghir, Ariel Pakes, Ashesh Rambachan, Eric Renault, Jon Roth, Susanne Schennach, and participants at the Radcliffe Institute Conference on Statistics When the Model is Wrong, the Fisher-Schultz Lecture, the HBS Conference on Economic Models of Competition and Collusion, the University of Chicago Becker Applied Economics Workshop, the UCL Advances in Econometrics Conference, the Harvard-MIT IO Workshop, the BFI Conference on Robustness in Economics and Econometrics (especially discussant Jinyong Hahn), the Cornell Econometrics-IO Workshop, and the Johns Hopkins Applied Micro Workshop, for their comments and suggestions. We thank Nathan Hendren for assistance in working with his code and data. We thank our dedicated research assistants for their contributions to this project. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Jesse M. Shapiro
Shapiro has, in the past, been a paid visitor at Microsoft Research New England and a paid consultant for FutureOfCapitalism, LLC.
Shapiro's spouse has a disclosure statement posted at https://www.brown.edu/research/projects/oster/sites/brown.edu.research.projects.oster/files/uploads/COI.txt.