Explaining the modality effect in multimedia learning: Is it due to a lack of temporal contiguity with written text and pictures?

The study examined whether the modality effect is caused by either high visuo-spatial load or a lack of temporal contiguity when processing written text and pictures. Students (N = 147) viewed pictures on the development of tornados, which were accompanied by either spoken or written explanations presented simultaneously with, before, or after the pictures. For verbal recall no modality effect was observed, whereas for transfer the influence of modality varied as a function of phonological working memory capacity. For pictorial recall the results showed a modality effect that was limited to simultaneous presentation, apparently in line with the temporal contiguity explanation. However, spoken simultaneous presentations were not superior to spoken sequential presentations, which contradicts the temporal contiguity explanation for the modality effect. Rather, it seems as if learners with simultaneous presentation of written text and picture concentrated more on the text and ignored the pictures, resulting in worse pictorial recall.

► Two different explanations for the modality effect in multimedia learning were examined. ► Spoken or written texts were presented together with, before, or after the pictures. ► For pictorial recall, a modality effect appeared only with simultaneous presentation. ► For verbal recall, no modality effect was observed. ► These results are explained by the fact that with written text pictures are ignored.