Checking the consistency of the solution in ordinal semi-democratic decision-making problems

•The paper focuses on “ordinal semi-democratic” decision-making problems.•A new tool for assessing the consistency of the consensus ordering is presented.•The proposed tool (i.e., the p indicators) is simple, intuitive and versatile.•The p indicators can be also used for comparing different aggregation techniques.

An interesting decision-making problem is that aggregating multi-agent preference orderings into a consensus ordering, in the case the agents׳ importance is expressed in the form of a rank-ordering. Due to the specificity of the problem, the scientific literature encompasses a relatively small number of aggregation techniques. For the aggregation to be effective, it is important that the consensus ordering well reflects the input data, i.e., the agents׳ preference orderings and importance rank-ordering.The aim of this paper is introducing a new quantitative tool – represented by the so-called p indicators – which allows to check the degree of consistency between consensus ordering and input data, from several perspectives. This tool is independent from the aggregation technique in use and applicable to a wide variety of practical contexts, e.g., problems in which preference orderings include omissions and/or incomparabilities between some alternatives. Also, the p indicators are simple, intuitive and practical for comparing the results obtained from different techniques. The description is supported by various application examples.