Toward natural multi-user interaction in advanced collaborative display environments

•The concept of cue-less multi-user interaction.•An approach for building user-aware interactive environment.•Combination of multiple modalities to enable association of input events with users.•Interaction framework for group collaborative environments.•Building multi-sensor environments from commodity hardware.

Large high-resolution displays have become widely spread in research centers, laboratories, and public spaces during the last decade. There have been various research efforts in transformation of these displays from passive screens to interactive environments where multiple users can interact independently with multi-touch surfaces, in-air interaction or pointing devices. Such systems provide basic multi-user interaction, i.e., two or more users can interact concurrently. However, continuous user tracking and association of input events with users, which could considerably improve user experience, is still a largely unexplored topic. In this paper, we present a set of techniques enabling cue-less multi-user interaction in environments where horizontal or vertical high-resolution displays are in the role of central visualization platforms. We define the concept of cue-less multi-user interaction and set of techniques that enable unobtrusive user tracking and their association with input events through combination of a multi-touch surface and a depth sensor. Further, we introduce an open-source framework implementing techniques behind this concept and their evaluation in terms of accuracy of user association and the overall system interactivity when integrated with selected distributed rendering middleware.