A novel multi-view dimensionality reduction and recognition framework with applications to face recognition

Multi-view data with each view corresponding to a type of feature generally provides more comprehensive information. Learning from multi-view data is a challenging research topic in pattern recognition. For recognition task, most multi-view learning methods separately learn multi-view dimensionality reduction (MvDR) and classification models. Thus, the connection between the two models has not been well studied. In this paper, we propose a novel multi-view dimensionality reduction and recognition framework, which can establish the connection between MvDR and classification. Specifically, a multi-view dimensionality reduction method, termed as sparse representation regularized multiset canonical correlation analysis (SR2MCC) is first proposed. SR2MCC considers both correlation and sparse discrimination among multiple views. In accord with SR2MCC, a classifier, called multi-view sparse representation based classifier (MvSRC) is further developed. MvSRC performs classification by comparing the reconstruction residuals of different classes among all views. An efficient iterative algorithm is proposed to solve the proposed model. Extensive experiments on the AR, CMU PIE, FERET, and FRGC datasets demonstrate that the proposed framework can achieve superior recognition performance than several state-of-the-art methods.