Complete large margin linear discriminant analysis using mathematical programming approach

In this paper, we develop a novel dimensionality reduction (DR) framework coined complete large margin linear discriminant analysis (CLMLDA). Inspired by several recently proposed DR methods, CLMLDA constructs two mathematical programming models by maximizing the minimum distance between each class center and the total class center respectively in the null space of within-class scatter matrix and its orthogonal complementary space. In this way, CLMLDA not only makes full use of the discriminative information contained in the whole feature space but also overcome the weakness of linear discriminant analysis (LDA) in dealing with the class separation problem. The solutions of CLMLDA follow from solving two nonconvex optimization problems, each of which is transformed to a series of convex quadratic programming problems by using the constrained concave–convex procedure first, and then solved by off-the-shelf optimization toolbox. Experiments on both toy and several publicly available databases demonstrate its feasibility and effectiveness.

► A framework based on large margin idea is proposed for discriminant analysis. ► Two mathematical programming based models are constructed respectively. ► The models are used to extract irregular and regular discriminant information. ► The nonconvex optimization problems are converted to convex ones via CCCP.