SCAPE-based human performance reconstruction

•We proposed a framework to reconstruct human performance from low quality Kinect data with SCAPE model.•We devised a method to enlarge the SCAPE pose training database.•We introduced a hierarchical method with articulated ICP to robustly track the performer's motion.

This paper presents an automatic approach to reconstruct human motion using noisy depth data from multiple views. Although multi-view constraints are provided by this setup, it still exhibits great challenges to robustly reconstruct dynamic human performances due to inherent complexity and self-occlusion of human motion. In the insight that the semantics of human motion will supply strong prior in motion reconstruction, we therefore propose a SCAPE-based motion reconstruction algorithm. As the building blocks of this main algorithm, we (1) re-train a SCAPE model based on an expanded human pose database containing human poses collected from different databases to enlarge the tracking space, (2) develop a correspondence estimation method based on articulated ICP to improve the robustness of SCAPE tracking. We conduct experiments to demonstrate the effectiveness of our method, and show that our system is able to capture and reconstruct accurate human motion.

Graphical abstractReconstruction of human performance from Kinect data. Figure optionsDownload full-size imageDownload high-quality image (346 K)Download as PowerPoint slide