An empirical study of two typical locality preserving linear discriminant analysis methods

Laplacian linear discriminant analysis (LapLDA) and semi-supervised discriminant analysis (SDA) are two recently proposed LDA methods. They are developed independently with the aim to improve LDA by introducing a locality preserving regularization term, and they have proved their effectiveness experimentally on some benchmark datasets. However, both algorithms ignored comparison with much simpler methods such as regularized discriminant analysis (RDA). In this paper, we make an empirical and supplementary study on LapLDA and SDA, and obtain somewhat counterintuitive results: (1) although LapLDA can generally improve the classical LDA via resorting to a complex regularization term, it does not outperform RDA, which is only based on the simplest Tikhonov regularizer; (2) to reevaluate the performance of SDA, we develop purposely a new and much simpler semi-supervised algorithm called globality preserving discriminant analysis (GPDA) and make a comparison with SDA. Surprisingly, we find that GPDA tends to achieve better performance. These two points drive us to reconsider whether one should use or how to use locality preserving strategy in practice. Finally, we discuss the reasons that lead to the possible failure of the locality preserving criterion and provide alternative strategies and suggestions to address these problems.