FISH image analysis using a modified radial basis function network

Fluorescent in situ hybridization (FISH) is an exceptionally useful method in determining HER-2/neu gene status in breast carcinoma samples, which is a valuable cancer prognostic indicator. Its visual evaluation is a difficult task, which involves manual counting of red/green dots in multiple microscopy images, a procedure which is both time consuming and prone to human errors. A number of algorithms have recently been developed dealing with the (semi)-automated analysis of FISH images. Their performance is quite promising, but further improvement is required in their diagnostic accuracy. In addition, they have to be evaluated on large FISH image data sets. Here, we present a novel method for analyzing FISH images based on cell nuclei and red/green spot modelling by radial basis functions (RBFs). Our method was compared to one of the most prominent methods reported in the literature on a large data set, comprised of 246 breast cancer cases (in total 3412 FISH images) and showed statistically significant diagnostic accuracy improvement, especially on HER-2/neu positive cases. The overall diagnostic accuracy of the proposed method is 95.93% over this data set.

► In this paper we present a novel method for analyzing FISH images based on cell nuclei and red/green spot modelling by radial basis functions (RBFs). ► FISH image analysis comprises two major tasks, namely nuclei segmentation and spot detection. In this paper we describe a novel radial basis function (RBF) approach for both these steps. ► Our method was compared to one of the most prominent methods reported in the literature on a large data set, comprised of 246 breast cancer cases (in total 3412 FISH images) and showed statistically significant diagnostic accuracy improvement, especially on HER-2/neu positive cases.