Recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models

This paper describes a formal model for the recognition of on-line handwritten mathematical expressions using 2D stochastic context-free grammars and hidden Markov models. Hidden Markov models are used to recognize mathematical symbols, and a stochastic context-free grammar is used to model the relation between these symbols. This formal model makes possible to use classic algorithms for parsing and stochastic estimation. In this way, first, the model is able to capture many of variability phenomena that appear in on-line handwritten mathematical expressions during the training process. And second, the parsing process can make decisions taking into account only stochastic information, and avoiding heuristic decisions. The proposed model participated in a contest of mathematical expression recognition and it obtained the best results at different levels.

► We developed an on-line handwritten mathematical expressions recognition system. ► Symbol recognition was performed combining on-line and off-line information with HMM. ► We model spatial relations using geometric features and SVM. ► We described a formal framework based on 2D stochastic context-free grammars. ► This model properly recognizes expressions and it avoids making heuristic decisions.