Visual Servoing for Object Manipulation with a Multifingered Robot Hand*

This paper deals with the problem of controlling the 3D Cartesian position of an object grasped by a multifingered robot hand using the visual servoing approach. We consider that the geometry of the object is known and the contact constraints are always satisfied during the grasping motion. An RGB camera and a depth sensor are simultaneously used to provide visual positioning and depth information of the grasped object. The control design is based on the kinematic control approach and two visual servo controllers are developed in the image space, ensuring the zero-convergence of the output tracking errors and high disturbance-rejection capability. Practical tests for visual tracking tasks of a grasped object are performed with a three-fingered robot hand and a natural interface device. Experimental results are shown to illustrate the performance and feasibility of the proposed methodology.