3D monocular robotic ball catching

• The robotic 3D ball catching problem is solved by using a monocular visual system.
• Comparison with the ground-truth provided by an OptiTrack motion-capture system.
• An industrial robot manipulator endowed of a camera mounted inside the gripper.
• Continuous refinement of the interception point with nonlinear estimation algorithm.
• Initial starting solution is provided by a fast linear estimating process.

A new method to catch a thrown ball with a robot endowed with an eye-in-hand monocular visual system integrated into a gripper is proposed. As soon as the thrown ball is recognized by the visual system, the camera carried by the robot end-effector is forced to follow a baseline in the space so as to acquire an initial dataset of visual measurements from several points of view, providing a first estimate of the catching point through a linear estimation algorithm. Hereafter, additional measurements are acquired to constantly refine the previous estimate by exploiting a nonlinear estimation algorithm. During the robot trajectory, the translational components of the camera are controlled in such a way as to follow the planned path to intercept the ball, while the rotational components are forced to keep the ball into the field of view. Experimental results performed on a common industrial robotic system prove the effectiveness of the presented solution.