Untangling the relatedness among correlations, part I: Nonparametric approaches to inter-subject correlation analysis at the group level

•Inter-subject correlation shows similarities of response to naturalistic conditions.•A patterned matrix is formulated to characterize the relationship among ISC values.•Previous methods of ISC group analysis are problematic in controlling for FPR.•Subject-wise permutation testing is ideal for comparing two groups of ISC.•Bootstrapping is the best nonparametric method to make ISC inferences for one group.

FMRI data acquisition under naturalistic and continuous stimuli (e.g., watching a video or listening to music) has become popular recently due to the fact that it entails less manipulation and more realistic/complex contexts involved in the task, compared to the conventional task-based experimental designs. The synchronization or response similarities among subjects are typically measured through inter-subject correlation (ISC) between any pair of subjects. At the group level, summarizing the collection of ISC values is complicated by their intercorrelations, which necessarily lead to the violation of independence assumed in typical parametric approaches such as Student's t-test. Nonparametric methods, such as bootstrapping and permutation testing, have previously been adopted for testing purposes by resampling the time series of each subject, but the quantitative validity of these specific approaches in terms of controllability of false positive rate (FPR) has never been explored before. Here we survey the methods of ISC group analysis that have been employed in the literature, and discuss the issues involved in those methods. We then propose less computationally intensive nonparametric methods that can be performed at the group level (for both one- and two-sample analyses), as compared to the popular method of circularly shifting the EPI time series at the individual level. As part of the new approaches, subject-wise (SW) resampling is adopted instead of element-wise (EW) resampling, so that exchangeability and independence assumptions are satisfied, and the patterned correlation structure among the ISC values can be more accurately captured. We examine the FPR controllability and power achievement of all the methods through simulations, as well as their performance when applied to a real experimental dataset.