Cognitive Augmented Reality

Although the concept of Augmented Reality (AR) has already been proposed more than 20 years ago, most AR-applications are still limited to simple visualization of virtual objects onto spatially limited scenes. Ideas such as the “augmented reality manual”, showing step-by-step instructions to a user wearing a Head-Mounted Display (HMD), have been implemented, but the developed systems did not pass the barrier of demonstration prototypes. One major reason, beside remaining ergonomic and hardware limitations, consists of the large effort required for creating the content of such virtual instructions and for building models allowing accurate tracking. In this paper, we introduce the concept of Cognitive Augmented Reality, which radically revises existing approaches on AR-based assistance systems and proposes a fundamentally new paradigm exploiting prior visual observation and learning of a complete manipulative workflow. More precisely, we present in this paper a complete approach for creating augmented reality content for procedural tasks from video examples, and give then details about the presentation of such content at runtime. We show that this new approach, in spite of its very challenging aspects, is scalable, and valuable from a practicable point of view.