Strategies to reduce the negative effects of spoken explanatory text on integrated tasks

Two experiments involving 125 grade-10 students learning about commerce investigated strategies to overcome the transient information effect caused by explanatory spoken text. The transient information effect occurs when learning is reduced as a result of information disappearing before the learner has time to adequately process it, or link it with new information. Spoken text, unless recorded or repeated in some fashion, is fleeting in nature and can be a major cause of transiency. The three strategies investigated, all theoretically expected to enhance learning, were: (a) replacing lengthy spoken text with written text (Experiments 1 and 2), (b) replacing lengthy continuous text with segmented text (Experiment 1), and (c) adding a diagram to lengthy spoken text (Experiment 2). In both experiments on tasks that required information to be integrated across segments, written text was found to be superior to spoken text. In Experiment 1 the expected advantage of segmented text in reducing transitory effects was not found. Compared with written continuous text the segmented spoken text strategy was inferior. Experiment 2 found that adding a diagram to spoken text was an advantage compared to spoken text alone consistent with a multimedia effect. Overall, the results suggest that spoken text is a cause of the transient information effect, which can be best avoided by substituting written text for spoken text on tasks that require integration of information.     

Welche Rolle spielt neben Merkmalen des Instruktionsdesigns die fachspezifische und aufgabenspezifische Motivation beim Lernen mit Multimedia im naturwissenschaftlichen Unterricht?

The present paper reports on an empirical study which investigated learning with text and animations in the science classroom. In a 2?×?2 design the presence of multimedia learning material instruction (text only vs. text + animations) as well as the modality of the explanatory text (spoken vs. written) were tested. Prior to learning, students’ motivation to learn was assessed as a continuous factor. Recall and transfer were assessed immediately after learning. The results show better recall of information for learners with multimedia materials, providing the animations were accompanied by spoken text. However, in contrast to the positive effects of domain-specific motivation this multimedia effect was not evident for transfer. The effects of multimedia design were independent of motivation. Implications for future studies are discussed.     

Instructional benefits of spoken words: A review of cognitive load factors

Spoken words have always been an important component of traditional instruction. With the development of modern educational technology tools, spoken text more often replaces or supplements written or on-screen textual representations. However, there could be a cognitive load cost involved in this trend, as spoken words can have both benefits and disadvantages based on essential characteristics of our cognitive architecture. This paper analyzes factors that might moderate the effectiveness of using spoken text in instruction by reviewing relevant studies in multimedia learning and considering cognitive load consequences of the transiency of spoken information. However, in contrast to earlier studies that considered spoken words in the context of a specific cognitive load effect, this paper provides a framework for evaluating potential instructional benefits of spoken text by analyzing various instructional situations depending on whether spoken text is used together with pictures and written text, and taking into account relations between presented sources of information.     

The reverse modality effect: Examining student learning from interactive computer‐based instruction

The purpose of this study was to explore the effects of modality on learning from multimedia instruction. This study utilized a factorial between‐subject design to examine the effects of modality on student learning outcomes, study patterns and mental effort. An interactive computer‐presented diagram was developed to teach the places of articulation in human speech. A total of 151 undergraduate students at a large southwestern university in USA participated in this study. Participants were randomly assigned to one of two modality conditions (ie, written text and spoken text). Data were obtained through surveys, student logs and knowledge tests. Findings revealed a reverse modality effect, wherein participants who studied with written text outperformed those who studied with spoken text.     

Redundancy in foreign language reading comprehension instruction: Concurrent written and spoken presentations

In an example of the redundancy effect, learning is inhibited when written and spoken text containing the same information is presented simultaneously rather than in written or spoken form alone. The current research was designed to investigate whether the redundancy effect applied to reading comprehension in English as a foreign language (EFL) by comparing two instructional formats, written presentation only and written presentation concurrent with verbatim spoken presentation. Participants were in their first year of tertiary education. Examination of translation scores, subjective mental load ratings, and free recall performance indicated that simultaneous presentations rendered text comprehension less effective both at a lexical level and at the level of text comprehension compared with written presentation only.     

A closer look at split visual attention in system- and self-paced instruction in multimedia learning

Two experiments examined visual attention distribution in learning from text and pictures. Participants watched a 16-step multimedia instruction on the formation of lightning. In Experiment 1 (N = 90) the instruction was system-paced (fast, medium, slow pace), while it was self-paced in Experiment 2 (N = 31). In both experiments the text modality was varied (written, spoken). During learning, the participants’ eye movements were recorded. Results from both experiments revealed that learners spent more time studying the visualizations with spoken text than those with written text. In written text conditions learners consistently started reading before alternating between text and visualization; moreover, they spent more time reading the text than inspecting the visualizations. While in Experiment 1 additional time that was made available in conditions with a slow or medium instruction pace was spent inspecting visualizations, in Experiment 2 longer learning times resulted from reading the text more intensively. With respect to learning outcomes (retention, transfer, and visual memory) Experiment 1 revealed an effect of text modality for visual memory only. In Experiment 2 no modality effects were found. Instruction pace was hardly related to learning outcomes. Overall, the results confirm prior findings suggesting that the distribution of visual attention in multimedia learning is largely guided by the text.     

The modality effect in a mobile learning environment: Learning from spoken text and real objects

The finding that under split-attention conditions students learn more from a picture and spoken text than from a picture and written text (ie, the modality effect) has consistently been found in many types of computer-assisted multimedia learning environments. Using 58 fifth-grade and sixth-grade elementary school children as participants, we investigated whether the modality effect can also be found in a mobile learning environment (MLE) on plants' leaf morphology, in which students had to learn by integrating information from text and real plants in the physical environment. A single factor experimental design was used to examine the hypothesis that students in a mixed-mode condition with real plants and spoken text (STP condition) would pay more attention to the real plants, and achieve higher performance on retention, comprehension, and transfer tests than the single-mode condition with real plants and written text (WTP condition). Whereas we found that participants in the STP condition paid more attention to observing the plants, and achieved a higher score on the transfer test than participants in the WTP condition, no differences were found between the conditions for retention and comprehension test performance.     

Eye tracking as a tool to study and enhance multimedia learning

This special issue comprises a set of six papers, in which studies are presented that use eye tracking to analyse multimedia learning processes in detail. Most of the papers focus on the effects on visual attention of animations with different design features such as spoken vs. written text, different kinds of cues, or different presentation speeds. Two contributions concern effects of learner characteristics (prior knowledge) on visual attention when learning with video and complex graphics. In addition, in some papers eye tracking is not only used as a process measure in itself, but also as input for verbal reports (i.e., cued retrospective reporting). In the two commentaries, the contributions are discussed from a multimedia learning perspective and an eye tracking perspective, by prominent researchers in those fields. Together, the contributions to this issue give an overview of the various possibilities eye tracking opens up for research on multimedia learning and instruction.     

When Do Pictures Help Learning from Expository Text? Multimedia and Modality Effects in Primary Schools

Adding pictures to a text is very common in today’s education and might be especially beneficial for elementary school children, whose abilities to read and understand pure text have not yet been fully developed. Our study examined whether adding pictures supports learning of a biology text in fourth grade and whether the text modality (spoken or written) plays a role. Results indicate that overall, pictures enhanced learning but that the text should be spoken rather than written. These results are in line with instructional design principles derived from common multimedia learning theories. In addition, for elementary school children, it might be advisable to read texts out to the children. Reading by themselves and looking at pictures might overload children’s cognitive capacities and especially their visual channel. In this case, text and pictures would not be integrated into one coherent mental model, and effective learning would not take place.     

Focus of attention and choice of text modality in multimedia learning

The term ‘modality effect’ in multimedia learning means that students learn better from pictures combined with spoken rather than written text. The most prominent explanations refer to the split attention between visual text reading and picture observation which could affect transfer of information into working memory, maintenance of information in working memory or the effective size of working memory. The assumption of a continuous need for split attention is questionable, however. Learners can keep pictorial information in working memory, when they have seen the picture before, especially if they have higher prior knowledge. Instead of suffering from a permanent split attention, learners frequently show tendencies to simply ignore pictures. This suggests guiding learners towards picture analysis by picture-related text paragraphs. We assume that these paragraphs are associated with stronger modality effects than content-related paragraphs, especially if the pictures are new to learners. These assumptions were tested in an experiment with 120 students learning about volcanism from illustrated text consisting of segments each including a content-related paragraph followed by a picture-related paragraph describing the accompanying visualization. Content-related and picture-related paragraphs were presented as visual or auditory texts leading to 2x2 conditions of text presentation. Picture novelty was manipulated by presenting a picture throughout the whole segment or only when the picture-related paragraph was read. As expected, picture-related paragraphs were associated with stronger modality effects than content-related paragraphs if picture novelty is high. The distinction between different kinds of paragraphs seems to be important for the prediction of modality effects.     

Observational learning from animated models: Effects of modality and reflection on transfer

Animated models use animations and explanations to teach how a problem is solved and why particular problem-solving methods are chosen. Often spoken explanations are proposed to accompany animations in order to prevent overloading the visual channel (i.e., the modality effect). In this study we adopt the hypothesis that the inferior performance of written text compared to spoken text is due to the fact that written text receives less attention and, consequently, less effortful processing. In a 2 × 2 factorial experiment (N = 96) with the factors modality (written, spoken) and reflection (reflection prompts, no reflection prompts) the hypothesis is tested that prompted reflection requires learners to explicitly attend to written explanations and carefully process them, thus yielding higher transfer performance, whereas for spoken explanations prompted reflection would have no effect on transfer performance. The results indeed showed the hypothesized interaction between modality and reflection prompts. They suggest that the modality effect can be compensated for when learners explicitly attend to the information and effortfully process it. This has implications for learning situations in which spoken explanations are no option, such as education for the hearing-impaired.     

Learning with multiple representations: Extending multimedia learning beyond the lab

The present study extended multimedia learning principles beyond the lab to an ecologically valid setting (homework). Eighteen information cards were used to perform three homework tasks. The control group students learnt from single representation (SR) cards that presented all information as printed text. The multiple representation (MR) group students received the same information, but each card contained either printed text or an informationally equivalent graphic such as a chart or graph. The MR group students performed better than the SR group students on accuracy of their homework answers and on subsequent posttest retention and transfer, and used more information cards while learning. The improved MR group students' performance is discussed in relation to two alternative possible explanations – the nature of the information sources and students' motivation to learn.     

Explaining the modality effect in multimedia learning: Is it due to a lack of temporal contiguity with written text and pictures?

The study examined whether the modality effect is caused by either high visuo-spatial load or a lack of temporal contiguity when processing written text and pictures. Students (N = 147) viewed pictures on the development of tornados, which were accompanied by either spoken or written explanations presented simultaneously with, before, or after the pictures. For verbal recall no modality effect was observed, whereas for transfer the influence of modality varied as a function of phonological working memory capacity. For pictorial recall the results showed a modality effect that was limited to simultaneous presentation, apparently in line with the temporal contiguity explanation. However, spoken simultaneous presentations were not superior to spoken sequential presentations, which contradicts the temporal contiguity explanation for the modality effect. Rather, it seems as if learners with simultaneous presentation of written text and picture concentrated more on the text and ignored the pictures, resulting in worse pictorial recall.     

Investigating the Modality and Redundancy Effects for Learners with Persistent Pain

Two experiments were conducted to investigate how individuals with persistent pain would respond to instructional materials designed to promote the modality and redundancy effects. It was predicted that persistent pain would reduce the positive impact of narrated text due to reduced working memory capacity. One hundred thirty-seven full-time teachers from six schools in Australia participated in the study. Pain and pain-free participants were identified and matched through a demographics questionnaire before being randomly assigned to selected instructional formats. In both experiments, participants viewed system-paced multimedia presentations depicting how lightning works. In Experiment 1, it was found that for pain-free participants, illustrations plus narrated text led to significantly higher learning outcomes than illustrations plus written text (a modality effect). However, for pain participants, no advantage was found for the illustrations plus narrated text format. In Experiment 2, it was found that for pain-free participants, illustrations plus narrated text led to significantly higher learning outcomes than illustrations plus narrated text plus identical written text (a redundancy effect). For pain participants, a reverse effect was found, suggesting that the replicated written text format was not redundant but beneficial for these individuals. Furthermore, despite low scores on standardized pain measures, individuals with persistent pain achieved significantly lower learning outcomes than pain-free participants in both experiments. It was concluded that clinically low levels of pain could significantly interfere with information retention and transference. For individuals with persistent pain, the use of narrated explanatory text could be a disadvantage due to its transitory nature.     

Signals foster multimedia learning by supporting integration of highlighted text and diagram elements

Two experiments investigated how signals foster learning from text and diagrams by examining the relationship between visual attention and learning outcomes. In Experiment 1 (N = 55) students learned about the circulatory heart system from a multimedia lesson either with or without signals highlighting text–diagram correspondences. Results showed that students learning with signals attended to signaled (but not to non-signaled) information more frequently and earlier during learning; these changes in visual attention could explain better performance in answering text–diagram-integration questions. Experiment 2 (N = 78) replicated these findings with respect to early attention on signaled diagram elements and learning outcomes; in addition, a third condition was investigated, where signals highlighted diagram elements that did not match the text. Results showed that mismatched signals guided attention only initially, whereas later on students attended more to information that corresponded to the text. Mismatched signals had no effect on learning outcomes. Taken together, the results suggest that signals aid learning by highlighting specific text–diagram correspondences and not by amplifying diagram processing more generally.     

Learning from Multimedia Materials: The relative impact of individual differences

This paper explores whether the principles of cognitive load and multimedia theory are mediated by cognitive style, gender, and prior knowledge. Participants were 91 children aged 10–11 years (54 boys, 37 girls), each assigned to one of two presentation modes. In Condition 1 children were presented with diagrams supported by printed textual material. In Condition 2 the same diagrams were supported by narrated text. Condition 1 was designed in the conventional manner but Condition 2 was designed to adhere to cognitive load and multimedia theory. Then the children were asked a number of comprehension questions, an outcome measure requiring assimilation of information from both pictures and words. The Cognitive Style Analysis was administered to measure wholist–analytic style and verbal–imagery style. Results from national achievement tests were used as indicators of prior knowledge. Results indicate that outcome is differentiated by style, gender, and prior knowledge, and not just instructional design.     

Effects of student‐generated diagrams versus student‐generated summaries on conceptual understanding of causal and dynamic knowledge in plate tectonics

This research examines the beneficial effects of student‐generated diagrams versus student‐generated summaries on conceptual understanding in the domain of plate tectonics. Fifty‐eight Grade 5 students read a brief expository text about plate tectonics. During their reading of the text, students were asked to either draw diagrams, produce written summaries, or simply read the text (control). Conceptual understanding was measured by the diagrams and summaries which were generated during students' reading of the text, as well as by a posttest which assessed students' understanding of both spatial/static and causal/dynamic knowledge of the domain. Results indicated that the summaries generated during the reading of the text contained more domain‐related information than the diagrams which were generated during the reading of the text. However, on the posttest measures, the diagram group outperformed both the summary and text only groups in terms of understanding both the spatial/static as well as causal/dynamic aspects of the domain. Results are discussed with regard to the differential effects that generating diagrams as compared to generating summaries or simply reading has on both on‐line comprehension during reading and resulting conceptual understanding of the domain. © 1999 John Wiley & Sons, Inc. J Res Sci Teach 36: 39–53, 1999.     

Improving students’ learning by providing a graphic organizer after a multimedia document

The present study was intended to deepen current knowledge about the impact of graphic organizers (GOs) on students’ learning in a computer-based learning environment, by evaluating the effects of successive versus simultaneous presentation of a GO and an explanatory text. Based on the previous studies, we hypothesized that simultaneous display of a GO improves learning performances for the information it highlights (represented elements), but impairs those for nonrepresented elements, owing to a focused processing effect during text reading. We predicted that staggering the display of the document and the GO would avoid this effect and enhance the learning of both represented and nonrepresented elements. We compared three groups: students who studied a multimedia document without a GO (control group); students who viewed the same multimedia document accompanied by a GO that was displayed simultaneously (simultaneous GO group); and students who viewed the document and were then shown the GO (successive GO group). In line with previous research on GOs, results showed that adding a GO to the multimedia document increased students’ memorization of the represented elements and their comprehension. Interestingly, students in the successive GO group exhibited enhanced memorization of the nonrepresented elements and had the highest comprehension scores. These original results indicate that delaying the display of the GO can lead students to process the overall information more deeply and, in turn, improve their learning.     

Computer-based multimedia testing: Effects of static and animated representational pictures and text modality

Static representational pictures (RPs) have been focused in research on the multimedia effect in testing and might be especially important in arithmetical word problems, which require a multi-stage mental processing to segment the task. To further highlight the task segments visually, dynamic visualizations could help. However, conventional animations might not apply to this context and the role of dynamic visualizations with temporal segmentations (i.e., animated RPs) is unexplored. This classroom experiment with 456 students investigated multimedia and modality effects in 24 mathematical word problems. Our 3 × 2 mixed design included three multimedia conditions (static RPs, animated RPs, and text-only) and two modality conditions (written text vs spoken text). We investigated effects on response correctness, metacognitive ratings, item-solving satisfaction and time on task. Both static and animated RPs increased response correctness, item-solving satisfaction, and metacognitive ratings compared to text-only. Time on task was affected in distinctive ways in both RP conditions and also varied depending on text modality. Spoken text barely increased response correctness in animated RP items but not at all in static RP items. Moderator analyses revealed that the effects of static and animated RPs on response correctness were dependent on the text modality but varied across school types and the level of mathematical prior knowledge. For students at non-academic-track schools or with low prior knowledge, static and animated RPs improved response correctness compared to text-only across both modalities. For students at academic-track schools or with high prior knowledge, mainly combinations of static or animated RPs with spoken text were effective.     

PRIOR KNOWLEDGE AND ONLINE INQUIRY-BASED SCIENCE READING: EVIDENCE FROM EYE TRACKING

This study employed eye-tracking technology to examine how students with different levels of prior knowledge process text and data diagrams when reading a web-based scientific report. Students’ visual behaviors were tracked and recorded when they read a report demonstrating the relationship between the greenhouse effect and global climate change in 2 diagrams and 4 textual sections. Based on the pretest scores, 13 participants were categorized into high and low prior knowledge (PK) groups. Eye-tracking measures including the total reading time, total fixation duration, and total regression number on each area of interest of the 2 groups were compared. A heat map was further used to show the overall visual distribution of each group. In addition, the inter-scanning transitions between the textual and graphical information of the 2 groups were compared and further confirmed by the patterns of the scan paths. The results revealed that overall students spent more time reading the textual than the graphical information. The high PK students showed longer fixation durations and more regressions on the graphics than the low PK students. Meanwhile, the high PK students showed more inter-scanning transitions than the low PK students not only between the text and graphics but also between the 2 data diagrams. This suggests that the high PK students were more able to integrate text and graphic information and inspect scientific data which is essential for online inquiry learning. This study provides eye-tracking evidence to show that low PK students have difficulties integrating scientific diagrams with expository texts and inspecting scientific data diagrams that are commonly shown in websites. Suggestions are made for future studies and instructional design for online inquiry-based science learning.