Robust visual servo control in the presence of fruit motion for robotic citrus harvesting

• Robust visual servo controller is formulated to compensate for unknown fruit motion.
• Lyapunov-based stability guarantees uniformly ultimately bounded regulation of the end-effector.
• By selecting appropriate control gains, the desired pick cycle time can be achieved.
• The developed controller is computationally inexpensive and easy to implement.
• The developed controller can easily be extended to non-citrus fruits with general ellipsoid geometry.

Unknown fruit motion due to exogenous disturbances such as wind gust, canopy unloading, and particularly, fruit detachment forces can reduce overall harvesting efficiency in robotic fruit harvesting. Existing approaches relying on high-gain controllers to compensate for fruit motion are inherently susceptible to measurement noise and can lead to instability. The contribution of this paper is in the development of a robust image-based visual servo controller to regulate a robotic manipulator to a target fruit in the presence of unknown fruit motion. The robust feedback elements included in the control structure compensate for non-vanishing nonlinear disturbances without the need for high-gain feedback. Lyapunov-based stability analysis guarantees uniformly ultimately bounded regulation of the robot end-effector to a target fruit. In addition, finite-time convergence analysis is presented to show that the controller gains can be chosen to achieve the desired fruit removal rate, or cycle time. Numerical simulations with varying fruit displacement of {35,70,105,140,175,210}{35,70,105,140,175,210} mm verify the feasibility of the developed controller while the performance is evaluated on a seven degrees-of-freedom kinematically redundant manipulator using an artificial citrus fruit moving with 120 mm displacement. The developed robust controller demonstrates stable closed-loop operation of the system. Further, the effect of uncertainties in field conditions such as illumination variations and partial fruit occlusion, overlapping fruit, and obstacles on the developed controller are discussed.