On using the hypervolume indicator to compare Pareto fronts: Applications to multi-criteria optimal experimental design

• The Pareto approach to optimal experimental design is utilized for finding experimental designs that are simultaneously optimal over multiple design criteria. The Pareto approach requires that one adequately populate the underlying Pareto front.
• Population of the Pareto front is computationally burdensome and thus the development of efficient algorithms for the population of the Pareto front is an active area of research. Fundamental to algorithm comparison is a measure that can be utilized to compare approximated Pareto fronts.
• We propose a hypervolume indicator for comparing competing Pareto fronts.
• Hypervolume indicators that are in the literature traditionally suffer from the following issues: lack of Pareto compliance, non-monotonically increases as a front evolves, use of different and arbitrary reference points, and not utilizing the contribution of all Pareto front points in the computation of the hypervolume.
• The proposed measure in this manuscript: 1. Is Pareto compliant, 2. Monotonically increases as a front evolves, 3. Utilizes sound theoretical principles for choice of the reference point, and 4. Allows all points on the Pareto front to contribute to the hypervolume.
• The proposed measure is demonstrated to be effective in comparing existing fronts from competing algorithms (the ‘offline’ application).
• The proposed measure is demonstrated to be effective in determining a point of convergence for an algorithm that is searching for a Pareto front (the ‘online’ application).
• Theoretical principles are developed for determining the reference point.

The Pareto approach to optimal experimental design simultaneously considers multiple objectives by constructing a set of Pareto optimal designs while explicitly considering trade-offs between opposing criteria. Various algorithms have been proposed to populate Pareto fronts of designs, and evaluating and comparing these fronts–and by extension the algorithms that produce them–is crucial. In this paper, we first propose a framework for comparing algorithm-generated Pareto fronts based on a refined hypervolume indicator. We then theoretically address how the choice of the reference point affects comparisons of Pareto fronts, and demonstrate that our approach is Pareto compliant. Based on our theoretical investigation, we provide rules for choosing reference points when two-dimensional Pareto fronts are compared. Because theoretical results for three-dimensional fronts are difficult to obtain, we propose an empirical rule for the three-dimensional case by making an analogy to the rules for two dimensions. We also consider the use of our procedure in evaluating the progress of a front-constructing algorithm, and illustrate our work with two examples from the literature.