A hybrid breakout local search and reinforcement learning approach to the vertex separator problem

•New perturbation strategy for Breakout Local Search (BLS).•Parameter control mechanism based on ideas from reinforcement learning theory.•Experiments on a large set of instances from different VSP applications.

The Vertex Separator Problem (VSP) is an NP-hard problem which emerges from a variety of important domains and applications. In this paper, we present an improved Breakout Local Search for VSP (named BLS-RLE). The distinguishing feature of this approach is a new parameter control mechanism that draws upon ideas from reinforcement learning theory to reach an interdependent decision on the number and on the type of perturbation moves. The mechanism complies with the principle of first carrying out intensification and then employing minimal diversification only if needed, it uses a dedicated sampling strategy for a rapid convergence towards a limited set of parameter values that appear to be the most convenient for the given state of search. Extensive experimental evaluations and statistical comparisons on a wide range of benchmark instances show significant improvement in performance of the proposed algorithm over the existing BLS algorithm for VSP. Indeed, out of the 422 tested instances, BLS-RLE was able to attain the best-known solution in 93.8% of the cases, which is around 20% higher compared to the existing BLS. In addition, we provide detailed analyses to evaluate the importance of the key elements of the proposed method and to justify the degree of diversification introduced during perturbation.