Scientific Visualizations Based on Integrated Model of Text and Picture Comprehension via Eye-tracking

This study investigates the “color combination effect” and the “sequence of pictures and text effect” in the “Integrated Model of Text and Picture Comprehension” (ITPC) on the formation of the mental model and cognitive load in both desktop and mobile devices. These effects are studied using an experiment including scientific contexts with specific color combinations and different picture and text sequences. The participants of this experiment are graduate students from different fields of science in Universiti Sains Malaysia. Four tasks are assigned for each participant in each phase and every task includes a scientific context accompanied with visualization. After each task the participant is asked to answer a set of questions related to the context. Eye-tracking methods are adopted to record gaze data of the participants while reading each context. Analysis of the gaze data and the percentage of the correct answers to the questions indicate that the sequence of picture and text and use of color in the visualizations results in reduction of the cognitive load of the participants.