Multi-order co-occurrence activations encoded with Fisher Vector for scene character recognition

Scene character recognition remains a challenging task due to many interference factors. Considering that characters are composed of a series of parts arranged in certain structures, in this paper, we propose a novel representation termed multi-order co-occurrence activations (MCA) encoded with Fisher Vector (FV), namely MCA-FV. It implicitly models the co-occurrence information of discriminative character parts at different orders to boost the recognition performance. We first extract convolutional activations as local descriptors of character parts from convolutional neural networks (CNNs). Then, we introduce MCA features to capture the multi-order co-occurrence cues among different discriminative character parts. Finally, we apply FV to encode co-occurrence features of each order and obtain a global representation of MCA-FV. The proposed method is evaluated on four scene character datasets including English and Chinese datasets. Experiment results demonstrate the effectiveness of the proposed method for scene character recognition.