Adaptive robotic tool use under variable grasps

Successful robotic manipulation of human tools will greatly advance robotic collaboration with humans in manufacturing and robotic assistance in human environments, e.g., in hospitals, offices, and homes. In these settings, the robot needs to grasp a tool (e.g., a drill) before using it, rather than rely on the tool being firmly attached to the robot's end effector. Thus, when using the tool, the robot has to account for the uncertainty in the hand-tool interface, since the grasp will vary between trials. To address this challenge, we propose a new framework in which control-relevant parameters are extracted about the uncertain interface between palm and tool tip. Our approach allows a robot to control position and force at the tool tip using either visual or tactile feedback. In addition, the proposed framework allows a robot to move the tip of a tool along a surface, despite uncertainty about how the tool is held in the hand and uncertainty about the structure of the surface. We demonstrated the feasibility of our new approach on two robotic platforms: the DARPA ARM robot operating a hand-held drill and an ST Robotics R17 robot drawing with a pencil.