Language processing in the visual world: Effects of preview, visual complexity, and prediction

• Processing of utterances containing syntactic ambiguities was investigated.
• Utterances were interpreted against the context of a relevant visual world.
• Garden-path effects occur only when subjects preview displays with few objects.
• Production data suggest listeners may use preview to predict upcoming utterances.
• Results support a flexible and adaptive language processing system.

This study investigates how people interpret spoken sentences in the context of a relevant visual world by focusing on garden-path sentences, such as Put the book on the chair in the bucket, in which the prepositional phrase on the chair is temporarily ambiguous between a goal and modifier interpretation. In three comprehension experiments, listeners heard these types of sentences (along with disambiguated controls) while viewing arrays of objects. These experiments demonstrate that a classic garden-path effect is obtained only when listeners have a preview of the display and when the visual context contains relatively few objects. Results from a production experiment suggest that listeners accrue knowledge that may allow them to have certain expectations of the upcoming utterance based on visual information. Taken together, these findings have theoretical implications for both the role of prediction as an adaptive comprehension strategy, and for how comprehension tendencies change under variable visual and temporal processing demands.