Effects of input modality on speech–gesture integration

Do the gestures that accompany narrations produced after watching videos differ from those that accompany narrations produced after reading texts? Building on previous work by Hostetter and Hopkins, we ask whether participants gesture less after reading texts, whether participants produce different types of gestures after reading texts, whether the same motion event features (path and manner) are present in gesture in both cases, and whether gestural viewpoint (character or observer viewpoint) is impacted by input modality. We find that while participants produce longer narrations (and thus more gestures overall) after watching videos, gesture rate (number of gestures per word), gesture type, motion event feature encoding, and viewpoint are not affected by input modality. We comment on the implications of our findings for embodied theories of language.