An end-to-end tool chain for Sensory Experience based on MPEG-V

This paper provides an overview of our research conducted in the area of Sensory Experience including our implementations using MPEG-V Part 3 entitled “Sensory Information”. MPEG-V Part 3 introduces Sensory Experience as a tool to increase the Quality of Experience by annotating traditional multimedia data with sensory effects. These sensory effects are rendered on special devices like fans, vibration chairs, ambient lights, scent disposers, water sprayers, or heating/cooling devices stimulating senses beyond the traditional ones. The paper's main focus is on the end-to-end aspects including the generation, transmission, and synchronized rendering of sensory effects with the traditional multimedia data taking movie clips as an example. Therefore, we present in this paper an open source tool chain that provides a complete end-to-end sensory effect generation and consumption framework. Furthermore, we summarize results from various subjective quality assessments conducted in this area. Finally, we point out research challenges that may encourage further research within this emerging domain.

► We give an overview of MPEG-V Part 3: Sensory Information.
► We present an end-to-end tool chain for creating, simulating, delivering, and rendering of Sensory Experience.
► We motivate the research area of Sensory Experience by presenting our efforts and achievements.
► We summarize Quality of Experience evaluation results conducted using our tool chain.