Autonomous Leaf Picking Using Deep Learning and Visual-Servoing

Grasping unstructured objects with a robotic manipulator is a difficult task. The challenge lies in both determining which objects are desirable to grasp and where those objects are relative to the manipulator. However, a robotic arm with an eye-in-hand camera can be used to effectively grab and hold desired objects, even if the exact geometry and position of the objects are unknown before the experiment. Combining Convolutional Neural Networks in image processing and Monoscopic Depth Analysis for visual-servoing, it was successfully demonstrated that a robotic manipulator could be used to accurately locate and grasp a leaf from a plant while only using a single camera for identification. This task of finding and manipulating a leaf shows the potential of using robotic manipulators in increasingly unstructured environments.