Selective invocation of shape priors for deformable segmentation and morphologic classification of prostate cancer tissue microarrays

• An optimized level set segmentation scheme, which selectively leverages shape prior.
• A CAD system to perform automated Gleason grading on large histopathology images.

Shape based active contours have emerged as a natural solution to overlap resolution. However, most of these shape-based methods are computationally expensive. There are instances in an image where no overlapping objects are present and applying these schemes results in significant computational overhead without any accompanying, additional benefit. In this paper we present a novel adaptive active contour scheme (AdACM) that combines boundary and region based energy terms with a shape prior in a multi level set formulation. To reduce the computational overhead, the shape prior term in the variational formulation is only invoked for those instances in the image where overlaps between objects are identified; these overlaps being identified via a contour concavity detection scheme. By not having to invoke all three terms (shape, boundary, region) for segmenting every object in the scene, the computational expense of the integrated active contour model is dramatically reduced, a particularly relevant consideration when multiple objects have to be segmented on very large histopathological images. The AdACM was employed for the task of segmenting nuclei on 80 prostate cancer tissue microarray images from 40 patient studies. Nuclear shape based, architectural and textural features extracted from these segmentations were extracted and found to able to discriminate different Gleason grade patterns with a classification accuracy of 86% via a quadratic discriminant analysis (QDA) classifier. On average the AdACM model provided 60% savings in computational times compared to a non-optimized hybrid active contour model involving a shape prior.