An approach integrating two complementary model-based environments for the construction of multimodal interactive applications

This paper presents a tool suite for the engineering of multimodal Post-WIMP Interactive Systems. The work presented here extends previous work done on design, prototyping, specification and verification of interactive systems and integrates two previously unrelated approaches. The first element of this integration is ICoM (a data-flow model dedicated to low-level input modelling) and its environment ICon which allows for editing and simulating ICoM models. The other element is ICOs (a formal description technique mainly dedicated to dialogue modelling) and its environment PetShop, which allows for editing, simulating and verifying ICOs models. This paper shows how these two approaches have been integrated and that this integration allows for engineering multimodal interactive systems. We show on a Range Slider case study how these tools can be used for prototyping interactive systems in general and multimodal interaction techniques in particular. We also present in details how the changes in the interaction techniques impact the models at various levels of the software architecture.