Identification of task sets within and across stimulus modalities

Previous studies have shown that task sets can be identified from functional magnetic resonance imaging (fMRI) data. However, these results may be partially confounded by differences in stimulus features associated with the different tasks. We disentangle stimulus modality and task features by presenting the same stimulus while varying task, and conversely, presenting different stimuli using the same task. Analyses were conducted on fMRI data previously collected on twenty participants who made either affective or semantic judgements of the same music pieces or the same silent video clips (Kim et al., 2017). Holding stimuli constant, task set was identified from fMRI data across individuals from both task activation data and functional connectivity data. Thus, we were able to identify whether participants made affective or semantic judgments when exposed to identical stimuli based on the task activation and functional connectivity data from other participants. Moreover, task set was successfully identified for cross-modal prediction in which stimuli in the training set bore no resemblance to those in the test set (e.g., using videos data to predict task for music data). Brain regions that were sensitive to tasks irrespective of sensory modality were identified by univariate and searchlight analyses of fMRI data. Consistent with a frontal-parietal network, middle frontal gyrus, inferior parietal gyrus, mid-cingulate cortex, and superior temporal sulcus were found to be key regions distinguishing the two task sets.