A new multi-modal approach to bib number/text detection and recognition in Marathon images

Bib number/text detection and recognition in Marathon natural images is challenging because of unconstrained poses created by background and bib number font variations. This paper presents a new multi-modal approach to integrate torso and text detection approaches in a novel way to achieve good results in contrast to the conventional methods that rely on the characteristics of texts but not numerals of arbitrary orientations. In the first stage, we explore HOG features along with an SVM classifier for upper body detection from the input image. Then the well-known Grab Cut method is adapted for foreground segmentation from the upper body image. Next, the pictorial structural model has been used for torso detection with the help of conditional random field (CRF). In the second stage, we propose to use a text detection method for locating texts from torso results, which are used for recognition further. We conduct experiments on a standard database as well as our database for validating torso detection in real life scenarios. To show the effectiveness of the proposed multi-modal approach, we also conduct extensive experiments on text detection and recognition before and after torso detection.