Stacking denoising auto-encoders in a deep network to segment the brainstem on MRI in brain cancer patients: A clinical study

•A deep learning-based network to segment organs at risk in brain cancer is proposed.•Our approach is compared with a state-of-the-art machine learning classifier, SVM.•The use of additional information in the classifier is suggested and evaluated.•Results of our approach lie on the variability of experts’ segmentations.•Segmentation time is largely reduced by the proposed scheme.

Delineation of organs at risk (OARs) is a crucial step in surgical and treatment planning in brain cancer, where precise OARs volume delineation is required. However, this task is still often manually performed, which is time-consuming and prone to observer variability. To tackle these issues a deep learning approach based on stacking denoising auto-encoders has been proposed to segment the brainstem on magnetic resonance images in brain cancer context. Additionally to classical features used in machine learning to segment brain structures, two new features are suggested. Four experts participated in this study by segmenting the brainstem on 9 patients who underwent radiosurgery. Analysis of variance on shape and volume similarity metrics indicated that there were significant differences (p < 0.05) between the groups of manual annotations and automatic segmentations. Experimental evaluation also showed an overlapping higher than 90% with respect to the ground truth. These results are comparable, and often higher, to those of the state of the art segmentation methods but with a considerably reduction of the segmentation time.