Statistical adaptive metric learning in visual action feature set recognition *

•A statistical adaptive metric learning (SAML) is proposed to classify action features.•SAML explores multiple statistic combinations for feature sets in different scales.•Discriminative statistic subspace is learned by a unified metric learning framework.•High competitive performances are achieved by SAML on five benchmark databases.

Great variances in visual features often present significant challenges in human action recognitions. To address this common problem, this paper proposes a statistical adaptive metric learning (SAML) method by exploring various selections and combinations of multiple statistics in a unified metric learning framework. Most statistics have certain advantages in specific controlled environments, and systematic selections and combinations can adapt them to more realistic “in the wild” scenarios. In the proposed method, multiple statistics, include means, covariance matrices and Gaussian distributions, are explicitly mapped or generated in the Riemannian manifolds. Typically, d-dimensional mean vectors in Rd are mapped to a Rd ×d space of symmetric positive definite (SPD) matrices Symd+. Subsequently, by embedding the heterogeneous manifolds in their tangent Hilbert space, subspace combination with minimal deviation is selected from multiple statistics. Then Mahalanobis metrics are introduced to map them back into the Euclidean space. Unified optimizations are finally performed based on the Euclidean distances. In the proposed method, subspaces with smaller deviations are selected before metric learning. Therefore, by exploring different metric combinations, the final learning is more representative and effective than exhaustively learning from all the hybrid metrics. Experimental evaluations are conducted on human action recognitions in both static and dynamic scenarios. Promising results demonstrate that the proposed method performs effectively for human action recognitions in the wild.

Graphical abstractFigure optionsDownload full-size imageDownload high-quality image (166 K)Download as PowerPoint slide