Adjusting Imperfect Data: Overview and Case Studies
Research users of large administrative have to adjust their data for quirks, problems, and issues that are inevitable when working with these kinds of datasets. Not all solutions to these problems are identical, and how they differ may affect how the data is to be interpreted. Some elements of the data, such as the unit of observation, remain fundamentally different, and it is important to keep that in mind when comparing data across countries. In this paper (written for Lazear and Shaw, 2007), we focus on the differences in the underlying data for a selection of country datasets. We describe two data elements that remain fundamentally different across countries -- the sampling or data collection methodology, and the basic unit of analysis (establishment or firm) -- and the extent to which they differ. We then proceed to document some of the problems that affect longitudinally linked administrative data in general, and we describe some of the solutions analysts and statistical agencies have implemented, and explore, through a select set of case studies, how each adjustment or absence thereof might affect the data.
I am indebted to all the authors of the country-specific chapters for having provided me with detailed data descriptions, allowing me to write this chapter. Juhana Vartiainen, Lia Pacelli, Roberto Leombruni, Claudio Villosio, and Bruno Contini provided valuable contributions beyond their data descriptions. I am thankful to all of the above, John Abowd, and Julia Lane for comments on drafts of this text. All errors, of course, remain mine. The views expressed herein are those of the author(s) and do not necessarily reflect the views of the National Bureau of Economic Research.
Lazear, Edward P. and Kathryn L. Shaw (eds.) The Structure of Wages: An International Comparison, National Bureau of Economic Research Conference Report. Chicago and London: University of Chicago Press, 2009.
Adjusting Imperfect Data: Overview and Case Studies, Lars Vilhuber. in The Structure of Wages: An International Comparison, Lazear and Shaw. 2008