Breast mass classification via deeply integrating the contextual information from multi-view data

Automatic differentiation of benign and malignant mammography images is a challenging task. Recently, Convolutional Neural Networks (CNNs) have been proposed to address this task based on raw pixel input. However, these CNN-based methods are unable to exploit information from multiple sources, e.g., multi-view image and clinical data. A hybrid deep network framework is presented in this paper, aiming to efficiently integrate and exploit information from multi-view data for breast mass classification. Starting from a generic CNN for feature extraction and assuming a multi-view setup, an attention-based network is used to automatically select the informative features of breast mass. The attention mechanism attempts to make CNN focus on the semantic-related regions for a more interpretable classification result. Then, mass features from multi-view data are effectively aggregated by a Recurrent Neural Network (RNN). In contrast to previous works, the proposed framework learns the attention-driven features of CNN as well as the semantic label dependency among different views. We justify the proposed framework through extensive experiments on the BCDR data set and quantitative comparisons against other methods. We achieve a good performance in terms of ACC (0.85) and AUC (0.89).