Stereo vision with Equal Baseline Multiple Camera Set (EBMCS) for obtaining depth maps of plants

- The multi-camera set is applied for obtaining distances to plants.
- The method for merging disparity maps from different cameras is proposed.
- The proposed vision system is over 20% more accurate than a stereo camera.
- The system is designed for autonomous harvesting robots.

This paper presents a method of improving the estimation of distances between an autonomous harvesting robot and plants with ripe fruits by using the vision system based on five cameras. The system is called Equal Baseline Multiple Camera Set (EBMCS). EBMCS has some features of a camera matrix and a camera array. EBMCS is regarded as a set of stereo cameras for estimating distances by obtaining disparity maps and depth maps. This paper introduces Exceptions Excluding Merging Method (EEMM) which makes it possible to improve the quality of disparity maps by integrating maps acquired from individual stereo cameras included in EBMCS. The method was tested with eight different stereo matching algorithms including Efficient Large-scale Stereo Matching (ELAS), algorithms implemented in the OpenCV library and algorithms available in Middlebury Stereo Vision Page. Experiments were performed on input data sets which contained images of strawberry plants, cherry trees and redcurrant plants. The bad matching pixels (BMP) metric was used for measuring the error rate in disparity maps used in the distance estimation. The results of experiments showed that, on average, the EEMM merging method used with EBMCS consisting of five cameras reduces the error rate of the distance estimation by 26.55% in comparison to results obtained from stereoscopy based on a single stereo camera. The best results were acquired by using with five cameras a stereo matching algorithm based on a graph cut.