Towards the automatic detection of spontaneous agreement and disagreement based on nonverbal behaviour: A survey of related cues, databases, and tools

While detecting and interpreting temporal patterns of nonverbal behavioural cues in a given context is a natural and often unconscious process for humans, it remains a rather difficult task for computer systems. Nevertheless, it is an important one to achieve if the goal is to realise a naturalistic communication between humans and machines. Machines that are able to sense social attitudes like agreement and disagreement and respond to them in a meaningful way are likely to be welcomed by users due to the more natural, efficient and human‐centred interaction they are bound to experience. This paper surveys the nonverbal behavioural cues that could be present during displays of agreement and disagreement; discusses a number of methods that could be used or adapted to detect these suggested cues; lists some publicly available databases these tools could be trained on for the analysis of spontaneous, audiovisual instances of agreement and disagreement, it examines the few existing attempts at agreement and disagreement classification, and finally discusses the challenges in automatically detecting agreement and disagreement.

Figure optionsDownload high-quality image (156 K)Download as PowerPoint slideHighlights
► We survey the nonverbal cues that are relevant to agreement and disagreement.
► We examine the methods and tools available that could detect these cues.
► We present a list of databases rich in spontaneous (dis)agreement episodes.
► We discuss the progress made so far in automatic detection of (dis)agreement.
► We discuss challenges, including treating (dis)agreement in a dimensional approach.