Panel Data and Experimental Design
How should researchers design panel data experiments? We analytically derive the variance of panel estimators, informing power calculations in panel data settings. We generalize Frison and Pocock (1992) to fully arbitrary error structures, thereby extending McKenzie (2012) to allow for non-constant serial correlation. Using Monte Carlo simulations and real world panel data, we demonstrate that failing to account for arbitrary serial correlation ex ante yields experiments that are incorrectly powered under proper inference. By contrast, our “serial-correlation-robust” power calculations achieve correctly powered experiments in both simulated and real data. We discuss the implications of these results, and introduce a new software package to facilitate proper power calculations in practice.
We thank Andrew Foster, several anonymous referees, Michael Anderson, Maximilian Auffhammer, Patrick Baylis, Joshua Blonz, Severin Borenstein, Hai-Anh Dang, Aureo de Paula, Meredith Fowlie, James Gillan, Erin Kelley, Jeremy Magruder, Aprajit Mahajan, Edward Miguel, Tavneet Suri, Catherine Wolfram, and seminar participants at UC Berkeley, BITSS, NEUDC, and EMEE for valuable comments. Molly Van Dop provided excellent research assistance. Funding for this research was provided by the Berkeley Initiative for Transparency in the Social Sciences, a program of the Center for Effective Global Action (CEGA), with support from the Laura and John Arnold Foundation. Burlig was generously supported by the National Science Foundation’s Graduate Research Fellowship Program under grant DGE-1106400, and Preonas was generously supported by Resources for the Future’s Joseph L. Fisher Dissertation Fellowship. All remaining errors are our own. Our accompanying Stata package, pcpanel, is available from ssc. We have no financial relationships that relate to this research. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Fiona Burlig & Louis Preonas & Matt Woerman, 2020. "Panel data and experimental design," Journal of Development Economics, .