Flexible orthogonal semisupervised learning for dimension reduction with image classification

• FODR performs better than the other semisupervised DR algorithms.
• FODR produce a set of orthogonal projection vectors.
• The regression residue ||F−TXW||2||F−XTW||2 is introduced.
• There are more than C features can be extracted.
• FODR has no convergence condition in TR-FSDA.

In this paper, we propose a novel orthogonal manifold learning algorithm for semisupervised dimension reduction, referred to as Flexible Orthogonal Semisupervised Dimension Reduction (FODR). Our algorithm is based on the recently-developed algorithm, called Trace Ratio Based Flexible Semisupervised Discriminant Analysis (TR-FSDA). TR-FSDA introduces an orthogonality constraint and a flexible regularizer to relax such a hard linear constraint in Semisupervised Discriminant Analysis (SDA) that the low-dimensional representation is constrained to lie within the linear subspace spanned by the data, whose solution follows from solving a trace ratio problem iteratively. However, it is not guaranteed that TR-FSDA always converges. Instead of finding the orthogonal projection vectors once, our algorithm produces the orthogonal projection vectors, step by step. In each time of iterations, an orthogonal projection vector and a one-dimensional data representation are produced by solving a standard Rayleigh Quotient problem, and more importantly, the determination of a new orthogonal projection vector does not involve the knowledge of the specific statistical property for the previously-obtained orthogonal projection vectors. Therefore, it is not necessary for our FODR algorithm to guarantee the convergence. The experiments are tried out on COIL20, UMIST, ORL, YALE, MPEG-7, FERET, and Handwritten DIGIT databases, and show the effectiveness of the proposed algorithm.