Markerless human–robot interface for dual robot manipulators using Kinect sensor

• Allows a human operator to communicate his motion to the dual robot manipulators by performing his two double hand–arms movements.
• Propose an innovative algorithm of over damping to solve the problem of error extracting and dithering.
• With making full use of the human hand–arms motion, the operator would feel kind of immersive.
• The object manipulation task done in collaboration by dual robots could be carried out flexibly.

Remote teleoperation of robot manipulators is often necessary in unstructured, dynamic, and dangerous environments. However, the existing mechanical and other contacting interfaces require unnatural, or hinder natural, human motions. At present, the contacting interfaces used in teleoperation for multiple robot manipulators often require multiple operators. Previous vision-based approaches have only been used in the remote teleoperation for one robot manipulator as well as require the special quantity of illumination and visual angle that limit the field of application. This paper presents a noncontacting Kinect-based method that allows a human operator to communicate his motions to the dual robot manipulators by performing double hand–arm movements that would naturally carry out an object manipulation task. This paper also proposes an innovative algorithm of over damping to solve the problem of error extracting and dithering due to the noncontact measure. By making full use of the human hand–arm motion, the operator would feel immersive. This human–robot interface allows the flexible implementation of the object manipulation task done in collaboration by dual robots through the double hand–arm motion by one operator.