Cross-correlation conditional restricted Boltzmann machines for modeling motion style

Temporal dependency plays a fundamental role in nonlinear generative models for capturing temporal features. One such model, the conditional restricted Boltzmann machine (CRBM), learns these temporal dependencies by considering the visible variables in the previous time slice as additional fixed inputs so that static and temporal features can be captured simultaneously to generate new human motions. However, the temporal dependencies in the CRBM fail to describe various common equilibrium postures in human motion. In this paper, we present cross-correlation as a new representation for modeling temporal dependencies by introducing the Pearson correlation coefficient. We also propose an approach to enhance the discrimination of the CRBM for various human motions by incorporating cross-correlation and temporal dependency features. The experimental results on benchmark databases demonstrate that the proposed method not only retains all the merits of the CRBM, such as exact inference and efficient learning, but also greatly improves the model's ability to blend motion styles and achieve smooth transitions between various motion segments.