Real time non-rigid 3D surface tracking using particle filter

•We recover simultaneously the non-rigid 3D shape and the corresponding camera pose from a single image in real time.•We use an efficient particle filter that performs an intelligent search of a database of deformations.•The surface is parameterized in different types of deformation clusters.•We present an exhaustive Design of Experiments to obtain the optimal parameterization of the particle filter.

Recovering a deformable 3D surface from a single image is an ill-posed problem because of the depth ambiguities. The resolution to this ambiguity normally requires prior knowledge about the most probable deformations that the surface can support. Many methods that address this problem have been proposed in the literature. Some of them rely on physical properties, while others learn the principal deformations of the object or are based on a reference textured image. However, they present some limitations such as high computational cost or the lack of the possibility of recovering the 3D shape. As an alternative to existing solutions, this paper provides a novel approach that simultaneously recovers the non-rigid 3D shape and the camera pose in real time from a single image. This proposal relies on an efficient particle filter that performs an intelligent search of a database of deformations. We present an exhaustive Design of Experiments to obtain the optimal parametrization of the particle filter, as well as a set of results to demonstrate the visual quality and the performance of our approach.