A system for mobile music authoring and active listening

• We propose an end-to-end system allowing authoring and real-time active of music.
• An authoring system enables users to personalize active music content.
• A run-time environment implements the interaction with active music content.
• User experiences active music content with mobile devices and environmental sensors.
• An evaluation study confirmed the suitability of active listening for creative entertainment.

We propose a paradigm and an end-to-end system to support authoring and real-time active listening experience of prerecorded music. Users are prosumers who navigate and express themselves in a shared (physical or virtual) orchestra space, populated by the sections of an orchestra playing a prerecorded music piece. The system consists of two components: an authoring system enabling the user to edit and shape personalized versions of active listening experiences, and a run-time environment implementing the mechanisms of interaction with the active music content. Both components exploit metadata defining the mapping rules of users expressive gesture and context information onto sound and music processing. A user interacts with content by holding her smartphone in her hand and by using environmental sensors and context-aware information to shape the experience, possibly in cooperation with other users.