Texture based segmentation of epithelial layer from oral histological images

The objective of this paper is to provide a texture based segmentation algorithm for better delineation of the epithelial layer from histological images in discriminating normal and oral sub-mucous fibrosis (OSF). As per literature and oral clinicians, it is established that the OSF initially originates and propagates in the epithelial layer. So, more accurate segmentation of this layer is extremely important for a clinician to make a diagnostic decision. In doing this, Gabor based texture gradient is computed in gray scale images, followed by preprocessing of the microscopic images of oral histological sections. On the other hand, the color gradients of these images are obtained in the transformed Lab color space. Finally, the watershed segmentation is extended to segment the layer based on the combination of texture and color gradients. The segmented images are compared with the ground truth images provided by the oral experts. The segmentation results depict the superiority of the texture based segmentation in comparison to the Otsu's based segmentation in terms of misclassification error. Results are shown and discussed.

► In this study we provide a texture based epithelial layer segmentation algorithm.
► Accurate segmentation is important for clinician to make a diagnostic decision.
► The watershed is used to segment the layer using texture and color gradients.
► The results depict the superiority of the segmentation compared with Otsu's method.