Orthogonal vs. uncorrelated least squares discriminant analysis for feature extraction

In this paper, a new discriminant analysis for feature extraction is derived from the perspective of least squares regression. To obtain great discriminative power between classes, all the data points in each class are expected to be regressed to a single vector, and the basic task is to find a transformation matrix such that the squared regression error is minimized. To this end, two least squares discriminant analysis methods are developed under the orthogonal or the uncorrelated constraint. We show that the orthogonal least squares discriminant analysis is an extension to the null space linear discriminant analysis, and the uncorrelated least squares discriminant analysis is exactly equivalent to the traditional linear discriminant analysis. Comparative experiments show that the orthogonal one is more preferable for real world applications.

► A new discriminant analysis is derived from the view of least squares regression.
► Two methods are developed under the orthogonal or the uncorrelated constraint.
► We show that the orthogonal one is an extension to the null space LDA.
► We show that the uncorrelated one is exactly equivalent to the traditional LDA.
► Comparative experiments show that the orthogonal one is more preferable.