Convolutional pyramid of bidirectional character sequences for the recognition of handwritten words

Handwritten word recognition is a challenging task due to the large intra-class variability of handwritten shapes and the complexity of modeling and segmenting sequences of overlapping characters. This work proposes a novel approach based on deep convolutional neural networks (CNNs), which does not require the explicit segmentation of characters and can learn a suitable representation for handwritten data in an automated way. The proposed approach uses a CNN to learn the mapping from word images to a robust representation, called pyramid of bidirectional character sequences. This novel representation encodes sub-sequences of characters in a hierarchical manner, considering both forward and backward directions. An efficient inference technique is then employed to find the most likely word in a lexicon, based on the CNN output probabilities. By implicitly modeling the distribution of character sub-sequences in the data, our approach can transfer knowledge across words containing the same sub-sequences. The proposed approach achieves a word error rate of 8.83% on the IAM database and 6.22% on the RIEMS database, outperforming recent methods for this task.