Recognition of clustered tomatoes based on binocular stereo vision

• Clustered tomatoes were classified into two types based on depth differences.
• Two recognition methods were used for two types of clustered tomatoes.
• An eight-neighbor mode denoising method was applied for depth map.
• An iterative OTSU method was used for depth map segmentation.
• The predict model of tomato size in image was applied to set parameters automatically.

To improve the applicability of the recognition method for clustered tomatoes, an algorithm based on binocular stereo vision was presented. First, a depth map of clustered tomatoes was acquired using a combination stereo matching method. Second, the noises in the depth map were removed using an eight-neighbor mode denoising method. Third, the clustered regions were classified into two types (i.e., overlapping and adhering regions) based on the depth difference between the front and back regions in a clustered region using an iterative Otsu method. Finally, different recognition methods were used for different types of clustered tomatoes. For adhering tomatoes, a recognition method based on edge curvature analysis was used for the edges in color image. For overlapping tomatoes, the same method was applied for the edges in color image, which were segmented into several parts by the edges in depth map after segmentation. A total of 189 pairs of stereo images were tested, and the recognition accuracy rate of clustered tomatoes was 87.9% when the leaf or branch occlusion rate was less than 25%. The acquisition distance and average execution time of this method were 300–500 mm and approximately 0.5 s, respectively. In conclusion, this method can realize the recognition of the clustered types and different types of clustered tomatoes, despite the serious occlusion of other tomatoes. Moreover, the headmost tomato in clustered tomatoes can be recognized based on depth information. This method can also realize the recognition of clustered tomatoes based on the images taken at different distances. However, the success rate of clustered tomatoes was not satisfactory when the occlusion was serious. Further research should focus on the improvement of the accuracy of stereo matching and the recognition of tomatoes occluded by leaves.