A general soft label based Linear Discriminant Analysis for semi-supervised dimensionality reduction

• A label propagation procedure is proposed to handle the outliers and dataset with multi-density distribution.
• A soft label based LDA is proposed to handle the out-of-sample problem of label propagation.
• A fast solution for solving SL-LDA is presented based on a weighted and regularized least square.
• A flexible version of SL-LDA is proposed for better cope with the data sampled from a nonlinear manifold.
• Simulation results show the efficiency and effectiveness of the proposed methods.

Dealing with high-dimensional data has always been a major problem in research of pattern recognition and machine learning, and Linear Discriminant Analysis (LDA) is one of the most popular methods for dimension reduction. However, it only uses labeled samples while neglecting unlabeled samples, which are abundant and can be easily obtained in the real world. In this paper, we propose a new dimension reduction method, called “SL-LDA”, by using unlabeled samples to enhance the performance of LDA. The new method first propagates label information from the labeled set to the unlabeled set via a label propagation process, where the predicted labels of unlabeled samples, called “soft labels”, can be obtained. It then incorporates the soft labels into the construction of scatter matrixes to find a transformed matrix for dimension reduction. In this way, the proposed method can preserve more discriminative information, which is preferable when solving the classification problem. We further propose an efficient approach for solving SL-LDA under a least squares framework, and a flexible method of SL-LDA (FSL-LDA) to better cope with datasets sampled from a nonlinear manifold. Extensive simulations are carried out on several datasets, and the results show the effectiveness of the proposed method.