Multi-manifold Sparse Graph Embedding for Multi‐modal Image Classification

Due to the drastic variations in illumination, occlusion or orientation, images of one object may vary widely. If we regard photos of one object with different appearances as different modalities, multi-modal images are common in reality. The multi-modal image classification can be considered as a learning problem about multi-manifold with possible intersections. General multi-manifold learning methods work on multiple independent subspaces, so it is difficult for them to balance the intra-class local manifold information and global inter-class discriminative structure. On the other hand, traditional supervised graph embedding algorithms neglect both the multi-modal multi-manifold structure and the importance of the feature selection in multi-modal image environment. In this paper, we propose a Multi-manifold Sparse Graph Embedding (MSGE) algorithm, which can explicitly capture multi-modal multi-manifold structure while considering both intra-class compactness and inter-class separability, and then learn an integral subspace model. Furthermore, a sparse embedding is achieved by using L2,1-norm, which makes the transformation matrix row-sparse, so MSGE can select relevant features and learn subspace transformation simultaneously. Experiments on several datasets demonstrate that MSGE can achieve better performance than the state-of-the-art reported in recent literature.