Multisensory perception of action in posterior temporal and parietal cortices

Environmental events produce many sensory cues for identifying the action that evoked the event, the agent that performed the action, and the object targeted by the action. The cues for identifying environmental events are usually distributed across multiple sensory systems. Thus, to understand how environmental events are recognized requires an understanding of the fundamental cognitive and neural processes involved in multisensory object and action recognition. Here, we investigated the neural substrates involved in auditory and visual recognition of object-directed actions. Consistent with previous work on visual recognition of isolated objects, visual recognition of actions, and recognition of environmental sounds, we found evidence for multisensory audiovisual event-selective activation bilaterally at the junction of the posterior middle temporal gyrus and the lateral occipital cortex, the left superior temporal sulcus, and bilaterally in the intraparietal sulcus. The results suggest that recognition of events through convergence of visual and auditory cues is accomplished through a network of brain regions that was previously implicated only in visual recognition of action.

Research highlights▶ Unisensory auditory and visual selectivity was seen in brain regions found in previous studies. ▶ Bi-modal audiovisual “event-selectivity” was found in posterior temporal cortex and posterior parietal cortex. ▶ Multisensory perception of action is controlled by a cortical network that was previously thought to be involved in visual action perception.