Recursive non-rigid structure from motion with online learned shape prior

• A novel approach is proposed for recursive non-rigid structure from motion.
• An adaptive algorithm is employed for online update of shape prior.
• Data storage requirements are reduced by removing original data once the shape model is recursively updated.
• The algorithm is able to handle the measurements with missing data.
• The method is not overly sensitive with respect to missing data, missing frames and measurement noise.

Most existing approaches in structure from motion for deformable objects focus on non-incremental solutions utilizing batch type algorithms. All data is collected before shape and motion reconstruction take place. This methodology is inherently unsuitable for applications that require real-time learning. Ideally the online system is capable of incrementally learning and building accurate shapes using current measurement data and past reconstructed shapes. Estimation of 3D structure and camera position is done online. To rely only on the measurements up until that moment is still a challenging problem.In this paper, a novel approach is proposed for recursive recovery of non-rigid structures from image sequences captured by a single camera. The main novelty in the proposed method is an adaptive algorithm for construction of shape constraints imposing stability on the online reconstructed shapes. The proposed, adaptively learned constraints have two aspects: constraints imposed on the basis shapes, the basic “building blocks” from which shapes are reconstructed; as well as constraints imposed on the mixing coefficients in the form of their probability distribution. Constraints are updated when the current model no longer adequately represents new shapes. This is achieved by means of Incremental Principal Component Analysis (IPCA). The proposed technique is also capable to handle missing data. Results are presented for motion capture based data of articulated face and simple human full-body movement.