A Dempster–Shafer Theory based combination of handwriting recognition systems with multiple rejection strategies

• A two level handwritten word recognition system is proposed.
• Combining Hidden Markov models classifier based on Dempster–Shafer Theory.
• A post-processing module based on different DST based acceptance/rejection strategies.
• We improve the accuracy rate and the reliability of a Handwriting recognition system.

Dempster–Shafer Theory (DST) is particularly efficient in combining multiple information sources providing incomplete, imprecise, biased, and conflictive knowledge. In this work, we focused on the improvement of the accuracy rate and the reliability of a HMM based handwriting recognition system, by the use of Dempster–Shafer Theory (DST). The system proceeds in two steps: First, an evidential combination method is proposed to finely combine the probabilistic outputs of the HMM classifiers. Second, a global post-processing module is proposed to improve the reliability of the system thanks to a set of acceptance/rejection decision strategies. In the end, an alternative treatment of the rejected samples is proposed using multi-stream HMM to improve the word recognition rate as well as the reliability of the recognition system, while not causing significant delays in the recognition process. Experiments carried out on two publically available word databases (RIMES for Latin script and IFN/ENIT for Arabic script) show the benefit of the proposed strategies.