A novel entropy-based decision support framework for uncertainty resolution in the initial subjective evaluations of experts: The NATO enlargement problem

• Experts provide evaluations about alternative weighting opportunities and threats.
• Evaluations differ among experts due to different criteria and subjective weights.
• Decision makers (DMs) observe biases and variations in the entropy among evaluations.
• DMs extrapolate additional evaluations and modify the initial ranking proposed.
• A numerical example based on the NATO membership enlargement process is presented.

We introduce a novel decision support framework that allows decision makers (DMs) to assess the informativeness of a ranking of alternatives provided by different experts and to extrapolate additional evaluations based on the distributional bias and entropy inherent to those received from the experts. In the proposed framework, expert analysts rank several alternatives within opportunities versus threats space of potential evaluations. The resulting rankings of alternatives vary among experts due to differences in the set of evaluation criteria chosen and to the subjective weights assigned by the experts to the criteria considered. As a result, DMs observe biases in the evaluations provided by the experts and variations in the degree of informativeness among alternatives. Thus, DMs must use the information available to them to assess the reliability of the ranking obtained from the experts' evaluations. In this regard, the distributional bias generated by the evaluations received will be used to define the dynamic structure of an algorithm that allows DMs to extrapolate additional expected evaluations and modify the initial ranking proposed by the experts accordingly. At the same time, the entropy generated by the evaluations will be used to validate the reliability of the resulting rankings and to determine the stopping rule for the data generating algorithm. A numerical example based on the North Atlantic Treaty Organization (NATO) membership enlargement problem is presented, where several teams of experts provide different evaluations on a set of applicant countries. A battery of Monte Carlo simulations has been performed, and alternative biased approaches have been followed. The rankings obtained have been compared with those resulting from our framework.