Perceiving incredulity: The role of intonation and facial gestures

Recently, some studies have revealed that facial gestures can play an important role in teasing out the meaning of interrogative sentence types in a particular language (Srinivasan and Massaro, 2003 and Borràs-Comes and Prieto, 2011; among others). However, less is known about potential cross-linguistic differences. This paper investigates the interaction between facial gestures and intonation in the distinction between information-seeking and incredulity yes/no questions in two languages (i.e., Catalan and Dutch) which use different prosodic strategies to express the distinction between these two types of interrogatives. While Dutch uses two phonologically distinct intonational contours, Catalan uses the same pitch contour with a distinction in pitch range. Twenty listeners of Catalan and twenty listeners of Dutch performed a perception experiment with audio-only, video-only, and audiovisual stimuli in congruent and incongruent intonation and gestural combinations. The results reveal that there is a contrast between Dutch and Catalan listeners in the perceptual processing of these sentences. While Dutch participants rely more on intonational differences, Catalan participants use the facial expression cues to a greater extent. All in all, the results show that both languages express pragmatic contrasts both at the intonation and facial expression levels, and native speakers are highly sensitive to the relative weight of these cues at the perceptual level.

► We analyze the interaction between facial gestures and intonation in the distinction between information-seeking and incredulity yes/no questions in two languages.
► We examine changes in intonation in Catalan and Dutch and the interaction with gestures.
► Identification of both tune meanings needs the visual cues in both languages.