Investigating the correlation between indicators of predictive diagnostic optimisation and search result quality

Combinatorially complex problems are often optimised with heuristic solvers which generally provide acceptable results but no indication as to how the quality achieved compares to the best possible. In previous work we have introduced Predictive Diagnostic Optimisation (PDO), a heuristic based on local search that provides information about the search space structure through a set of indicators whilst searching for the optimal solution. PDO can collect useful information about the search process, such as the variation in the number of steps needed to locally optimise a random solution and the error between the expected and actual qualities of the local optimum, known as the prediction error. Given previous experimental results on the quadratic assignment problem, it appears that a high prediction error coincides with lower search quality and vice versa. This work confirms this assumption with the help of two additional problems but also shows that the reliability of the prediction error is challenged by structural properties that lead to a homogeneity of the optima basins. Conversely, a high variation in the number of steps that lead to the local optima increases the reliability of the prediction error as an indicator of search quality.