Judging semantic similarity: an event-related fMRI study with auditory word stimuli

Much of mental life consists in thinking about object concepts that are not currently within the scope of perception. The general system that enables multiple representations to be maintained and compared is referred to as “working memory” [Repovš G, Baddeley A (2006) Neuroscience 139:5–21], and involves regions in medial and lateral parietal and frontal cortex [e.g., Smith EE, Jonides J (1999) Science 283:1657–1661]. It has been assumed that the contents of working memory index information in regions of the brain that are critical for processing and storing object knowledge. To study the processes involved in thinking about common object concepts, we used event related fMRI to study BOLD activity while participants made judgments of conceptual similarity over pairs of sequentially presented auditory words. Through a combination of conventional fMRI analysis approaches and multi-voxel pattern analysis (MVPA), we show that the brain responses associated with the second word in a pair carry information about the conceptual similarity between the two members of the pair. This was the case in frontal and parietal regions involved in the working memory and decision components of the task for both analysis approaches. However, in other regions of the brain, including early visual regions, MVPA permitted classification of semantic distance relationships where conventional averaging approaches failed to show a difference. These findings suggest that diffuse and statistically sub-threshold “scattering” of BOLD activity in some regions may carry substantial information about the contents of mental representations.