Simultaneous EEG-fMRI: Trial level spatio-temporal fusion for hierarchically reliable information discovery

•Hierarchically reliable information•Mining nonlinearity•Open approach

Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have been pursued in an effort to integrate complementary noninvasive information on brain activity. The primary goal involves better information discovery of the event-related neural activations at a spatial region of the BOLD fluctuation with the temporal resolution of the electrical signal. Many techniques and algorithms have been developed to integrate EEGs and fMRIs; however, the relative reliability of the integrated information is unclear. In this work, we propose a hierarchical framework to ensure the relative reliability of the integrated results and attempt to understand brain activation using this hierarchical ideal. First, spatial Independent Component Analysis (ICA) of fMRI and temporal ICA of EEG were performed to extract features at the trial level. Second, the maximal information coefficient (MIC) was adopted to temporally match them across the modalities for both linear and non-linear associations. Third, fMRI-constrained EEG source imaging was utilized to spatially match components across modalities. The simultaneously occurring events in the above two match steps provided EEG-fMRI spatial-temporal reliable integrated information, resulting in the most reliable components with high spatial and temporal resolution information. The other components discovered in the second or third steps provided second-level complementary information for flexible and cautious explanations. This paper contains two simulations and an example of real data, and the results indicate that the framework is a feasible approach to reveal cognitive processing in the human brain.