Aggregation and decision making using ranked data

Nonparametric procedures are frequently used to rank order alternatives. Often, information from several data sets must be aggregated to derive an overall ranking. When using nonparametric procedures, Simpson-like paradoxes can occur in which the conclusion drawn from the aggregate ranked data set seems contradictory to the conclusions drawn from the individual data sets. Extending previous results found in the literature for the Kruskal–Wallis test, this paper presents a strict condition that ranked data must satisfy in order to avoid this type of inconsistency when using nonparametric pairwise procedures or Bhapkar’s V procedure to extract an overall ranking. Aggregating ranked data poses further difficulties because there exist numerous ways to combine ranked data sets. This paper illustrates these difficulties and derives an upper bound for the number of possible ways that two ranked data sets can be combined.