Automatic fruit recognition and counting from multiple images

•Image analysis automatically counted fruits of complex shape and varying colour.•Datasets each consisted of 28,000 colour images of 1000 3 m high pepper plants.•Correlation between manual and automatic fruit counts was 74%.

In our post-genomic world, where we are deluged with genetic information, the bottleneck to scientific progress is often phenotyping, i.e. measuring the observable characteristics of living organisms, such as counting the number of fruits on a plant. Image analysis is one route to automation. In this paper we present a method for recognising and counting fruits from images in cluttered greenhouses. The plants are 3-m high peppers with fruits of complex shapes and varying colours similar to the plant canopy. Our calibration and validation datasets each consist of over 28,000 colour images of over 1000 experimental plants. We describe a new two-step method to locate and count pepper fruits: the first step is to find fruits in a single image using a bag-of-words model, and the second is to aggregate estimates from multiple images using a novel statistical approach to cluster repeated, incomplete observations. We demonstrate that image analysis can potentially yield a good correlation with manual measurement (94.6%) and our proposed method achieves a correlation of 74.2% without any linear adjustment for a large dataset.