A graphical editor for the SMUIML multimodal user interaction description language

• We present a graphical editor for the creation of multimodal interfaces.
• The editor is part of a triad of tools including a language and a framework.
• Multimodal interfaces are modelled through states and transitions.
• Core features of the editor are code generation, layouting and real-time debugging.
• An analysis of the different families of graphical tools is provided.

We present the results of an investigation on software support for the SMUIML multimodal user interaction description language (UIDL). In particular, we introduce a graphical UIDL editor for the creation of SMUIML scripts. The data management as well as the dialogue modelling in our graphical editor is based on the SMUIML language. Due to the event-centred nature of SMUIML, the multimodal dialogue modelling is represented by a state machine in the graphical SMUIML dialogue editor. Our editor further offers a real-time graphical debugging tool. Compared to existing multimodal dialogue editing solutions, the SMUIML graphical editor offers synchronised dual editing in graphical and textual forms as well as a number of operators for the temporal combination of modalities. The presented graphical editor represents the third component of a triad of tools for the development of multimodal user interfaces, consisting of an XML-based modelling language, a framework for the authoring of multimodal interfaces and a graphical editor.