Experiments with Kemeny ranking: What works when?

This paper performs a comparison of several methods for Kemeny rank aggregation (104 algorithms and combinations thereof in total) originating in social choice theory, machine learning, and theoretical computer science, with the goal of establishing the best trade-offs between search time and performance. We find that, for this theoretically NP-hard task, in practice the problems span three regimes: strong consensus, weak consensus, and no consensus. We make specific recommendations for each, and propose a computationally fast test to distinguish between the regimes.In spite of the great variety of algorithms, there are few classes that are consistently Pareto optimal. In the most interesting regime, the integer program exact formulation, local search algorithms and the approximate version of a theoretically exact branch and bound algorithm arise as strong contenders.

► We compare 104 algorithms for Kemeny rank aggregation across several datasets. ► We find that the problem instances span 3 regimes: strong, weak, and no consensus. ► We make specific algorithmic recommendations for each regime. ► We also propose a computationally fast test to distinguish between the 3 regimes. ► The integer program, local search, and branch and bound algorithms perform well.