Neural network language models for off-line handwriting recognition

• We study the use of neural network language models for two state-of-the-art recognizers for unconstrained off-line HTR.
• We found consistent improvement when using this language model, combined or not with standard N-grams language models.
• The neural network language model scales well with different dictionary sizes for the IAM-DB task.
• By combining the two recognition systems, unprecedented accuracy for the IAM database is reported.

Unconstrained off-line continuous handwritten text recognition is a very challenging task which has been recently addressed by different promising techniques. This work presents our latest contribution to this task, integrating neural network language models in the decoding process of three state-of-the-art systems: one based on bidirectional recurrent neural networks, another based on hybrid hidden Markov models and, finally, a combination of both. Experimental results obtained on the IAM off-line database demonstrate that consistent word error rate reductions can be achieved with neural network language models when compared with statistical N-gram language models on the three tested systems. The best word error rate, 16.1%, reported with ROVER combination of systems using neural network language models significantly outperforms current benchmark results for the IAM database.