Graph regularized multiset canonical correlations with applications to joint feature extraction

•We propose a novel algorithm called GrMCC for joint feature extraction.•GrMCC considers both discriminative and intrinsic geometrical structure in multi-representation data.•The extracted features by GrMCC have strong discriminant power for recognition.•Experimental results show GrMCC can provide encouraging recognition results in contrast to the state-of-the-art algorithms.

Multiset canonical correlation analysis (MCCA) is a powerful technique for analyzing linear correlations among multiple representation data. However, it usually fails to discover the intrinsic geometrical and discriminating structure of multiple data spaces in real-world applications. In this paper, we thus propose a novel algorithm, called graph regularized multiset canonical correlations (GrMCCs), which explicitly considers both discriminative and intrinsic geometrical structure in multiple representation data. GrMCC not only maximizes between-set cumulative correlations, but also minimizes local intraclass scatter and simultaneously maximizes local interclass separability by using the nearest neighbor graphs on within-set data. Thus, it can leverage the power of both MCCA and discriminative graph Laplacian regularization. Extensive experimental results on the AR, CMU PIE, Yale-B, AT&T, and ETH-80 datasets show that GrMCC has more discriminating power and can provide encouraging recognition results in contrast with the state-of-the-art algorithms.