Segmentation of neuronal nuclei based on clump splitting and a two-step binarization of images

•We propose a automatic segmentation algorithm for confocal microscope images.•Algorithm based on two-steps binarization and extended clump splitting algorithm.•Compared with state-of-the-art algorithms (Watershed, Level Set Methods).•Validation on a set of representative images coming from the rat neocortex.•Accurate segmentations (91–98% of properly identified cells’ nuclei).

In this paper we present an algorithm to segment the nuclei of neuronal cells in confocal microscopy images, a key technical problem in many experimental studies in the field of neuroscience. We describe the whole procedure, from the original images to the segmented individual nuclei, paying particular attention to the binarization of the images, which is not straightforward due to the technical difficulties related to the visualization of nuclei as individual objects and incomplete and irregular staining. We have focused on the division of clusters of nuclei that appear frequently in these images. Thus we have developed a clump-splitting algorithm to separate touching or overlapping nuclei allowing us to accurate account for both the number and size of the nuclei. The results presented in the paper show that the proposed algorithm performs well on different sets of images from different layers of the cerebral cortex.