Cross-modal priming effect based on short-term experience of ecologically unrelated audio-visual information: An event-related potential study

In the present study, we aimed to identify whether cross-modal priming effect based on short-term experience of ecologically unrelated audio-visual information existed. In the experiment, we employed ecologically unrelated pictures and sounds as visual and auditory stimuli. Participants were required to learn the associations between pictures and sounds in the study phase, and then finished the following experiment adopting a cross-modal priming paradigm. The result showed that the processing of sound was facilitated by a priming picture if an audio-visual pair was presented in the study phase. Enhanced induced gamma-band activity (GBA) was also found under the Match condition, and we suggested that induced GBA reflected the association between auditory and visual information to form supra-modal representation, which is top-down modulated by short-term experience of audio-visual information. Event-related potential (ERP) analysis revealed an N400 effect under the Mismatch condition compared to the Match condition, and source reconstruction of N400 effect showed that the biggest difference of activity between two conditions was localized in middle temporal gyrus (MTG), suggesting that MTG played an important role in the mapping process of auditory information onto a temporal semantic network.

► Cross-modal priming effect based on short-term multisensory experience exists.
► Probe sound elicited N400 effect when picture-sound pair was not learned.
► Enhanced GBA was found when picture-sound pair was temporally learned.
► Induced GBA reflected the association process to form supra-modal representation.
► The source of N400 effect was localized in MTG.