An orthogonal regularized CCA learning algorithm for feature fusion

•Extract discriminative features via orthogonal constraints and regularization.•The projection matrices are formulated by two general eigen-equation decompositions.•Insensitive to the regularization parameters for handwritten numerals classification.•Get better performances than CCA on both numeral and face recognition experiments.

Canonical correlation analysis (CCA) aims at extracting statistically uncorrelated features via conjugate orthonormalization constraints of the projection directions. However, the formulated directions under conjugate orthonormalization are not reliable when the training samples are few and the covariance matrix has not been exactly estimated. Additionally, this widely pursued property is focused on data representation rather than task discrimination. It is not suitable for classification problems when the samples that belong to different classes do not share the same distribution type. In this paper, an orthogonal regularized CCA (ORCCA) is proposed to avoid the above questions and extract more discriminative features via orthogonal constraints and regularized parameters. Experimental results on both handwritten numerals and face databases demonstrate that our proposed method significantly improves the recognition performance.