Investigating gaze behavior during processing of inconsistent text-picture information: Evidence for text-picture integration

In two experiments, eye tracking was used to investigate whether learners construct a mental representation during learning that integrates information from text and pictures. The experimental groups received inconsistent text-picture information on one or two pages of the learning materials. The control groups received only consistent text-picture information. It was expected that learners of the experimental groups should have difficulties in integrating text-picture information when faced with the inconsistencies. This should be reflected in their gaze behavior. Experiment 1 (N = 51) and Experiment 2 (N = 45) confirmed that assumption for several eye tracking variables. Regarding learning outcomes, only in Experiment 1 worse performance of the experimental group was observed. Furthermore, Experiment 2 revealed that the majority of learners did not remember the inconsistency between text and picture when asked for it after learning. In sum, the results add to our understanding about the cognitive processes underlying multimedia learning.