Paralinguistics in speech and language—State-of-the-art and the challenge

Paralinguistic analysis is increasingly turning into a mainstream topic in speech and language processing. This article aims to provide a broad overview of the constantly growing field by defining the field, introducing typical applications, presenting exemplary resources, and sharing a unified view of the chain of processing. It then presents the first broader Paralinguistic Challenge organised at INTERSPEECH 2010 by the authors including a historical overview of the Challenge tasks of recognising age, gender, and affect, a summary of methods used by the participants, and their results. In addition, we present the new benchmark obtained by fusion of participants’ predictions and conclude by discussing ten recent and emerging trends in the analysis of paralinguistics in speech and language.

► The first broader public Paralinguistic Challenge held at INTERSPEECH 2010 is summarised. ► A new benchmark is reported by fusion of the participants’ individual results. ► A survey on paralinguistic analysis in naturalistic speech is given.