Towards a taxonomy of error-handling strategies in recognition-based multi-modal human–computer interfaces

In this paper, we survey the different types of error-handling strategies that have been described in the literature on recognition-based human–computer interfaces. A wide range of strategies can be found in spoken human–machine dialogues, handwriting systems, and multi-modal natural interfaces. We then propose a taxonomy for classifying error-handling strategies that has the following three dimensions: the main actor in the error-handling process (machine versus user), the purpose of the strategy (error prevention, discovery, or correction), and the use of different modalities of interaction. The requirements that different error-handling strategies have on different sets of interaction modalities are also discussed. The main aim of this work is to establish a classification that can serve as a tool for understanding how to develop more efficient and more robust multi-modal human–machine interfaces.