Natural scene text detection with multi-layer segmentation and higher order conditional random field based analysis

• The contrasts in RGB channels are integrated to segment image into multi layers.
• The multi-layer segmentation is implemented with a graph cuts based model.
• A higher order CRF based connected component analysis is used.

Text detection in natural scene images is a hot and challenging problem in pattern recognition and computer vision. Considering the complex situations in natural scene images, we propose a robust two-steps method in this paper based on multi-layer segmentation and higher order conditional random field (CRF). Given an input image, the method separates text from its background by using multi-layer segmentation, which decomposes the input image into nine layers. Then, the connected components (CCs) in these different layers are obtained as candidate text. These candidate text CCs are verified by higher order CRF based analysis. Inspired from the multistage information integration mechanism of visual brains, features from three different levels, including separate CCs, CC pairs and CC strings, are integrated by a higher order CRF model to distinguish text from non-text. The remaining CCs are then grouped into words for easy evaluation. Experiments on the ICDAR datasets and street view dataset show that the proposed method achieves the state-of-art in natural scene text detection.