Aggregating Partial Rankings from Neighbors: Methodology and Empirical Evidence
Many decisions require ordering alternatives: for example, the selection of top candidates for a competitive academic program or the selection of the poorest individuals for a cash transfer program. One common approach consists in aggregating orderings reported by different observers (e.g., committee or community members), but those orderings are typically partial: not all observers rank all applicants. We introduce a novel type of approach, based on pairwise rankings, to (i) aggregate partial orderings reported by multiple observers and (ii) construct confidence intervals for the resulting aggregate ordering. We identify, both theoretically and using simulations, the conditions under which a pairwise approach dominates rank averaging: when reporting error is low, reported orderings are partial, and observers rank alternatives that are close to each other in their true latent ordering. We introduce improvements to rank averaging and pairwise methods and illustrate them using several datasets. We find that, with partial reported orderings, Borda counts (i.e., simple rank averages) are dominated by the averaging of normalized ranks and should never be used in practice.
-
-
Copy CitationPascaline Dupas, Marcel Fafchamps, and Deivy Houeix, "Aggregating Partial Rankings from Neighbors: Methodology and Empirical Evidence," NBER Working Paper 29911 (2022), https://doi.org/10.3386/w29911.Download Citation
-
Related
More from the NBER
In addition to working papers, the NBER disseminates affiliates’ latest findings through a range of free periodicals — the NBER Reporter, the NBER Digest, the Bulletin on Health, and the Bulletin on Entrepreneurship — as well as online conference reports, video lectures, and interviews.
