3D monocular robotic ball catching

•The robotic 3D ball catching problem is solved by using a monocular visual system.•Comparison with the ground-truth provided by an OptiTrack motion-capture system.•An industrial robot manipulator endowed of a camera mounted inside the gripper.•Continuous refinement of the interception point with nonlinear estimation algorithm.•Initial starting solution is provided by a fast linear estimating process.

A new method to catch a thrown ball with a robot endowed with an eye-in-hand monocular visual system integrated into a gripper is proposed. As soon as the thrown ball is recognized by the visual system, the camera carried by the robot end-effector is forced to follow a baseline in the space so as to acquire an initial dataset of visual measurements from several points of view, providing a first estimate of the catching point through a linear estimation algorithm. Hereafter, additional measurements are acquired to constantly refine the previous estimate by exploiting a nonlinear estimation algorithm. During the robot trajectory, the translational components of the camera are controlled in such a way as to follow the planned path to intercept the ball, while the rotational components are forced to keep the ball into the field of view. Experimental results performed on a common industrial robotic system prove the effectiveness of the presented solution.