Scene text recognition using a Hough forest implicit shape model and semi-Markov conditional random fields

•Part-based character models are built in a Hough forest.•The Hough forest robustly detects characters even if characters are highly degraded.•Checking text line properties effectively remove false detections.•Utilizing character models throughout the process raises end-to-end text recognition accuracy.

Most of the scene text recognition methods utilize character models only in the character recognition phase, the last stage of the process. In former phases such as text detection, only abstracted features of text regions are used, which might cause loss of information. In this paper, we propose a novel scene text recognition method which fully utilizes model of target characters throughout the process. Each of the target character set is modeled with a part-based object model called implicit shape model (ISM) to achieve robustness for the partial degradation of characters. Towards this end, we trained a Hough forest which localizes and aggregates character parts to detect character candidates in the image. The detected character candidates are verified by organizing the most plausible text lines in a semi-Markov conditional random field (semi-CRF) framework. As concrete character models are utilized throughout the process, even extremely deformed texts are detected and recognized.

Graphical abstractFigure optionsDownload full-size imageDownload high-quality image (325 K)Download as PowerPoint slide