A comparison of simulated annealing algorithms for variable selection in principal component analysis and discriminant analysis

Variable selection is a venerable problem in multivariate statistics. Simulated annealing is one of a variety of metaheuristics that can be gainfully employed for variable selection; however, its effectiveness is influenced by algorithm design features such as the construction of the initial subset, the maximum and minimum temperatures, the cooling scheme, and the process for generating trial subsets in the neighborhood of the incumbent subset. These design features were manipulated to produce 24 versions of a simulated annealing algorithm for the problem of selecting exactly p out of m candidate variables. The versions were then compared within the contexts of principal component analysis and discriminant analysis. The results suggest some complex and interesting interactions among the design features, yet some robust versions across the two studies were established.