Tracking elastic deformable objects with an RGB-D sensor for a pizza chef robot

This paper presents a method for tracking a 3D textureless object which undergoes elastic deformations, using the point cloud data provided by an RGB-D sensor and in real-time. This solution is expected to be useful for enhanced manipulation of humanoid robotic systems, especially in the case of pizza dough to be ideally manipulated by a pizza chef robot. Our tracking framework relies on a prior visual segmentation of the object in the image. The segmented point cloud is registered first in a rigid manner and then by non-rigidly fitting the mesh, based on the Finite Element Method to model elasticity, and on geometrical point-to-point correspondences to compute external forces exerted on the mesh. The system has been evaluated on synthetic and real data, and by integrating it into manipulation experiments on the RoDyMan1humanoid robotic platform.