How Different are Different Differences-in-Differences? A Large-Scale Meta-Analysis
I use AI-assisted replication to assess the empirical importance of negative weights in two-way fixed effects (TWFE) estimation, analyzing 1,430 staggered-adoption specifications from 116 economics papers. TWFE estimates are highly correlated with modern difference-in-differences estimates, with sign reversals in 12.0–14.9% of comparisons and greater precision for TWFE. Hierarchical models accounting for estimation noise imply estimand correlations of 91–97% and a 0.7–2.4% probability of a statistically significant TWFE estimate opposing the modern estimand’s sign. These findings suggest negative weights may be a less serious problem in practice and illustrate how AI can support systematic, large-scale empirical assessments of new econometric methods.
-
-
Copy CitationPeter Hull, "How Different are Different Differences-in-Differences? A Large-Scale Meta-Analysis," NBER Working Paper 35974 (2026), https://doi.org/10.3386/w35974.Download Citation
-