Region-based colour modelling for joint crop and maize tassel segmentation

•A region-based colour modelling method is developed for joint segmentation.•We proposed to use region proposals for region-smoothing and edge-preserving.•Ensemble neural networks robust to illumination are learnt for colour modelling.•We have established a joint segmentation dataset regarding crop and maize tassel.

Crop segmentation is a frequently occurring problem for computer vision applications in agriculture. Meanwhile, the fine-grained shape extraction of maize tassels is also an essential step in the detasselling and field-based phenotyping research. However, existing methods are usually dependent on category, which is hard to transfer to other cultivars with different colours. To address this, the goal of this study is to develop a general method that can process different colours simultaneously and that has good flexibility and expandability. Targeting maize, we proposed to segment jointly the crop and maize tassel. In particular, a novel joint segmentation dataset regarding crop and maize tassel (323 images with corresponding manually-annotated ground-truth images) is constructed, hoping that it can serve as a benchmark to facilitate related studies. Technically, a region-based approach that leverages the efficient graph-based segmentation algorithm and simple linear iterative clustering (SLIC) is developed to generate region proposals. Also, we proposed to model colours with ensemble neural networks specific to each intensity, aiming to achieve robustness to illumination. In addition, two simple but effective strategies are devised to accelerate the colour statistics extraction and ensemble model prediction. The effectiveness and efficiency of our method are demonstrated on the two segmentation tasks, respectively. Results show that our method significantly outperforms other state-of-the-art approaches on tassel segmentation, with average precision of 74.3%, and achieves comparable performance of 77.8% on the traditional crop segmentation even with the naivest colour feature. The dataset and source code are made available online.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slide