Prediction, events, and the advantage of Agents: The processing of semantic roles in visual narrative

•We utilized short visual narratives to investigate semantic roles.•Agents elicited more predictions about actions than Patients.•Agents were viewed longer than Patients independently of serial order.•Actions were processed more quickly after preparatory Agents than Patient.•Results suggest Agents initiate the building of event structure.

Agents consistently appear prior to Patients in sentences, manual signs, and drawings, and Agents are responded to faster when presented in visual depictions of events. We hypothesized that this “Agent advantage” reflects Agents’ role in event structure. We investigated this question by manipulating the depictions of Agents and Patients in preparatory actions in wordless visual narratives. We found that Agents elicited a greater degree of predictions regarding upcoming events than Patients, that Agents are viewed longer than Patients, independent of serial order, and that visual depictions of actions are processed more quickly following the presentation of an Agent vs. a Patient. Taken together these findings support the notion that Agents initiate the building of event representation. We suggest that Agent First orders facilitate the interpretation of events as they unfold and that the saliency of Agents within visual representations of events is driven by anticipation of upcoming events.