Processing multimedia material: Does integration of text and pictures result in a single or two interconnected mental representations?

• We studied text-picture integration based on research on the misinformation effect.
• We contrasted a single-mental-representations with a two-mental-representations view.
• Specific picture information was integrated with general sentence information.
• Sentence recognition relied on single, integrated mental representation.
• There was no evidence for text-picture integration for picture recognition.

We investigated whether individuals construct either two interconnected mental representations or a single mental representation that merges information from two representations when processing multimedia materials. Individuals memorized text-picture stimuli in four different versions that differed in the specificity of information contained in either text or pictures: general pictures/general sentences, general pictures/specific sentences, specific pictures/general sentences, and specific pictures/specific sentences. Afterwards, individuals decided whether they had previously seen the specific or the general version of the sentences and the pictures. Across two experiments, individuals more frequently falsely recognized the specific sentences after having seen general sentences/specific pictures. This indicates that individuals had integrated the specific picture information with the general sentence information into a single mental representation. No such effects were observed for picture recognition. The implications of these results for multimedia learning are discussed.