Multimodal learning analytics for game-based learning

A distinctive feature of game-based learning environments is their capacity to create learning experiences that are both effective and engaging. Recent advances in sensor-based technologies such as facial expression analysis and gaze tracking have introduced the opportunity to leverage multimodal data streams for learning analytics. Learning analytics informed by multimodal data captured during students’ interactions with game-based learning environments hold significant promise for developing a deeper understanding of game-based learning, designing game-based learning environments to detect maladaptive behaviors and informing adaptive scaffolding to support individualized learning. This paper introduces a multimodal learning analytics approach that incorporates student gameplay, eye tracking and facial expression data to predict student posttest performance and interest after interacting with a game-based learning environment, Crystal Island . We investigated the degree to which separate and combined modalities (ie, gameplay, facial expressions of emotions and eye gaze) captured from students (n = 65) were predictive of student posttest performance and interest after interacting with Crystal Island . Results indicate that when predicting student posttest performance and interest, models utilizing multimodal data either perform equally well or outperform models utilizing unimodal data. We discuss the synergistic effects of combining modalities for predicting both student interest and posttest performance. The findings suggest that multimodal learning analytics can accurately predict students’ posttest performance and interest during game-based learning and hold significant potential for guiding real-time adaptive scaffolding.     

The power of emotions: Can enjoyment and boredom explain the impact of individual preconditions and teaching methods on interest and performance in mathematics?

We investigated students’ emotions as intervening variables between teaching methods, motivational and performance prerequisites, and outcomes. 144 students from German schools were assigned to two conditions. In one condition, students were prompted to develop multiple solutions for modelling problems that were missing information. In the other condition, students had to find one solution for modelling problems that were not missing information. Students’ interest and performance were measured before and after the 5-lesson teaching unit, and students’ enjoyment and boredom were measured during the teaching unit. The path analyses revealed: (1) Students who developed more solutions enjoyed their mathematics lessons more and were less bored than students in the other condition; (2) Enjoyment affected students’ interest and performance at posttest and mediated the effects of prompting them to find multiple solutions on interest at posttest; (3) Students’ enjoyment during learning mediated the effects of prior interest on interest at posttest.     

The relative responsiveness of concrete operational seventh grade and college students to science instruction

Numerous persons have suggested that instruction should match the developmental level of the learner. Are “concrete operational” college students developmentally the same as “concrete operational” seventh grade students thus in need of identical instruction? Matched concrete operational seventh grade and college students were given identical classroom instruction in probabilistic and correlational reasoning. The college students performed significantly better on posttest measures which appeared to require greater processing of information while significant differences did not exist on less difficult items. Level of cognitive development, field independence, and fluid intelligence correlated moderately with posttest performance for the seventh grade students. Field independence and fluid intelligence correlated moderately with posttest performance for the college students but not pretest knowledge of specific biological concepts and cognitive level. It was concluded that college students are more responsive to instruction due either to (1) greater amount of experience or (2) greater information processing capacity. Implications for science teaching are discussed.     

University student’s goal profiles and metacomprehension accuracy

In this study, undergraduate students provided confidence ratings to predict future performance in answering questions drawn from the text before reading the text, after reading the text and after rereading the text. Self-reports of achievement goal orientations during reading and posttest scores were also collected. Student’s calibration index was the comparison between their predicted posttest performance and actual performance in the posttest. Correlational analyses did not reveal any statistically detectable relationships between self-reported goal orientations and monitoring accuracy, except that bias scores were marginally related to goal orientations. Further cluster analyses and analyses of variance (ANOVA) also showed that student’s multiple goal profiles failed to clearly differentiate the groups in terms of their calibration accuracy, yet performance-approach goals did distinguish overconfident from underconfident students. Plausible reasons for the finding were provided and implications for future research were also discussed.     

Collaborating With a Skilled Peer: The Influence of Achievement Goals and Perceptions of Partners' Competence on the Participation and Learning of Low-Achieving Students

The authors examined whether motivational goals influenced the participation and performance of low-achieving students during collaborative problem solving with a high-achieving partner. Thirty-five pairs of 4th- and 5th-grade students were randomly assigned a set of instructions designed to induce students to adopt a learning goal or a performance goal. The following day, the students were individually given a posttest on problems similar to those worked on collaboratively. The low-achieving students given learning-goal instructions performed better on the posttest problems and perceived their partner's competence as more similar to their own than did the low-achieving students given performance-goal instructions. No differences in overall amount or level of low achievers' participation during collaborative problem solving were observed. Implications of the findings for the use of peer learning in heterogeneous classrooms are discussed.     

Peer Collaboration on a Nonverbal Reasoning Task by Urban,Minority Students

An experimental study of collaborative reasoning in 86 7th-grade minority students from an urban, low-income school was conducted. The students completed items from the Test of Nonverbal Intelligence-Third Edition (TONI-3) and were asked to provide written explanations for their judgments. All students worked individually during the pretest and posttest phases of the study. During the experimental phase, some students worked independently while others worked in small, same-gender or mixed-gender groups. Significant improvements in judgments were evident during the experimental phase for all groups except the single-gender female group. The mixed-gender group reached the ceiling level of performance and outperformed both the students who worked independently and the collaborative, single-gender female group during the experimental phase. Posttest judgment scores declined and were not significantly different from pretest scores for any group. Significant improvements in written explanations for judgments were evident during the experimental phase and were maintained on the posttest by all groups. The students who collaborated had higher percentages of fully correct explanations on the posttest than students who worked alone. Collaborative experiences were beneficial for students' reasoning about unfamiliar, moderately difficult, nonverbal problems.     

The relationship between students' learning strategies and the change in their misunderstandings during a high school chemistry course

The purposes of this study are as follows: (a) to investigate the relationship between high school students' learning approaches, prior knowledge and attitudes toward chemistry, and their performance on a misunderstandings test; and (b) to describe and analyze the differences between the responses of students with different learning approaches on the same test. Forty-nine suburban high school students enrolled in two sections of New York State Regents Chemistry classes participated in the study. The students' performance on a misunderstandings pretest and the students' learning approach both accounted for a statistically significant proportion of the variance on their performance on the misunderstandings posttest. Additionally, the results showed that the relatively meaningful learners performed significantly better than the relatively rote learners on the misunderstandings posttest.     

An interaction of screen colour and lesson task in CAL

Colour is a common feature in computer‐aided learning (CAL), though the instructional effects of screen colour are not well understood. This investigation considers the effects of different CAL study tasks with feedback on posttest performance and on posttest memory of the lesson colour scheme. Graduate students (n=68) completed a computer‐based vocabulary lesson that included either multiple‐choice or constructed‐response study tasks with feedback. Each lesson section used a different colour theme, while the posttest did not use colour. The constructed‐response study task was a little more effective for posttest memory of lesson content but was substantially less effective for posttest memory of the lesson colour scheme. These results show a clear interaction of screen colour and lesson study task, a memory‐context effect. The practical and theoretical implications of these findings are discussed.     

Student conceptions of natural selection and its role in evolution

Pretests and posttests on the topic of evolution by natural selection were administered to students in a college nonmajors' biology course. Analysis of test responses revealed that most students understood evolution as a process in which species respond to environmental conditions by changing gradually over time. Student thinking differed from accepted biological theory in that (a) changes in traits were attributed to a need-driven adaptive process rather than random genetic mutation and sexual recombination, (b) no role was assigned to variation on traits within a population or differences in reproductive success, and (c) traits were seen as gradually changing in all members of a population. Although students had taken an average of 1.9 years of previous biology courses, performance on the pretest was uniformly low. There was no relationship between the amount of previous biology taken and either pretest or posttest performance. Belief in the truthfulness of evolutionary theory was also unrelated to either pretest or posttest performance. Course instruction using specially designed materials was moderately successful in improving students' understanding of the evolutionary process.     

Improvements of self-regulation procedures for fifth graders' reading competence: Analyzing effects on reading comprehension,reading strategy performance,and motivation for reading

In this study, reciprocal teaching (RT) was combined with specific self-regulation procedures to promote the reading comprehension of fifth grade students. Twenty four classes with N = 534 students were assigned to RT plus strategy implementation procedures (RT + SIP), RT plus outcome regulation procedures (RT + ORP), RT plus strategy implementation and outcome regulation procedures (RT + SRL), or RT without explicit instruction in self-regulation. At maintenance students assigned to the three self-regulation conditions outperformed RT students according to a standardized measure of reading comprehension. RT + SIP and RT + SRL students outperformed RT as well as RT + ORP students using a measure of reading strategy performance. However, reading comprehension was only mediated by strategy performance under the RT + SRL condition. Among students assigned to the RT + ORP condition, an improvement in reading motivation between pretest and posttest and between posttest and maintenance was observed.     

Skills‐Based Training and Counseling Self‐Efficacy

The authors studied the effectiveness of the skilled counselor training model (SCTM). Counseling students who completed the SCTM demonstrated greater gains in skills acquisition and counseling self‐efficacy than counseling students who did not receive the training. At pretest, the counseling students in both groups, doing self‐appraisals, rated their performance of counseling skills significantly higher than trained raters assessed the students' performance of counseling skills. At posttest, members of the control group continued to overestimate their skills levels when compared with the ratings given by the trained raters, whereas, in comparison, students who received SCTM training underestimated their performance.     

Improving seventh grade students’ learning of ratio and proportion: The role of schema-based instruction

The present study evaluated the effectiveness of an instructional intervention (schema-based instruction, SBI) that was designed to meet the diverse needs of middle school students by addressing the research literatures from both special education and mathematics education. Specifically, SBI emphasizes the role of the mathematical structure of problems and also provides students with a heuristic to aid and self-monitor problem solving. Further, SBI addresses well-articulated problem solving strategies and supports flexible use of the strategies based on the problem situation. One hundred forty eight seventh-grade students and their teachers participated in a 10-day intervention on learning to solve ratio and proportion word problems, with classrooms randomly assigned to SBI or a control condition. Results suggested that students in SBI treatment classes outperformed students in control classes on a problem solving measure, both at posttest and on a delayed posttest administered 4 months later. However, the two groups’ performance was comparable on a state standardized mathematics achievement test.     

The Role of Instruction for Spelling Performance and Spelling Consciousness

This study examined the role of instruction for spelling performance and spelling consciousness in the Dutch language. Spelling consciousness is the ability to reflect on one's spelling and correct errors. A sample of 115 third-grade spellers was assigned to a strategy-instruction, strategic-monitoring, self-monitoring, or control condition representing different types of metacognitive aspects. The results showed that students in all three training conditions made more progress in both spelling performance and spelling consciousness than students in the control condition. With respect to spelling consciousness, only students in the strategy-instruction condition made significant improvement between pretest and posttest. Students made more progress in spelling performance on regular words than on loan words. Students in all four conditions became more accurate at assessing which words they could spell correctly. Students in the control condition more frequently overestimated their spelling ability.     

Exploring the Value Added of a Guided,Silent Reading Intervention: Effects on Struggling Third-Grade Readers’ Achievement

ABSTRACT

The authors’ purpose was to explore the effects of a supplementary, guided, silent reading intervention with 80 struggling third-grade readers who were retained at grade level as a result of poor performance on the reading portion of a criterion referenced state assessment. The students were distributed in 11 elementary schools in a large, urban school district in the state of Florida. A matched, quasi-experimental design was constructed using propensity scores for this study. Students in the guided, silent reading intervention, Reading Plus, evidenced higher, statistically significant mean scores on the Florida Comprehensive Assessment Test criterion assessment measure of reading at posttest. The effect size, favoring the guided, silent reading intervention group was large, 1 full standard deviation, when comparing the 2 comparison groups’ mean posttest scores. As such, the results indicate a large advantage for providing struggling third-grade readers guided silent reading fluency practice in a computer-based practice environment. No significant difference was found between the treatment and control group on the Stanford Achievement Test–10 (SAT-10) posttest scores, although posttest scores for the treatment group trended higher than the control. After conducting a power analysis, it was determined that the sample size (n = 80) was too small to provide sufficient statistical power to detect a difference in third-grade students’ SAT-10 scores.     

Effects of different sequences of examples and problems on motivation and learning

Recent research has shown that example study only (EE) and example-problem pairs (EP) were more effective (i.e., higher test performance) and efficient (i.e., attained with less effort invested in learning and/or test tasks) than problem-example pairs (PE) and problem solving only (PP). We conducted two experiments to investigate how different example and problem-solving sequences would affect motivational (i.e., self-efficacy, perceived competence, and topic interest) and cognitive (i.e., effectiveness and efficiency) aspects of learning. In Experiment 1, 124 technical students learned a mathematical task with the help of EEEE, EPEP, PEPE, or PPPP and then completed a posttest. Students in the EEEE Condition showed higher posttest performance, self-efficacy, and perceived competence, attained with less effort investment, than students in the EPEP and PPPP Condition. Surprisingly, there were no differences between the EPEP and PEPE Condition on any of the outcome measures. We hypothesized that, because the tasks were relevant for technical students, starting with a problem might not have negatively affected their motivation. Therefore, we replicated the experiment with a different sample of 81 teacher training students. Experiment 2 showed an efficiency benefit of EEEE over EPEP, PEPE, and PPPP. However, only EEEE resulted in greater posttest performance, self-efficacy, and perceived competence than PPPP. We again did not find any differences between the EPEP and PEPE Condition. These results suggest that, at least when short training phases are used, studying examples (only) is more preferable than problem solving only for learning. Moreover, this study showed that example study (only) also enhances motivational aspects of learning whereas problem solving only does not positively affect students’ motivation at all.     

A digital game-based assessment of middle-school and college students’ choices to seek critical feedback and to revise

A major goal of contemporary education is to teach students how to learn on their own. Assessments have largely lagged behind this goal, because they measure what students have learned and not necessarily their learning processes. This research presents Posterlet, an assessment that collects evidence regarding the strategies that students choose while learning on their own. Posterlet is an educational game-based assessment that measures two design thinking choices: students’ choices to seek critical (ie, negative) feedback and to revise their work while they learn graphic design principles through creating posters. This research also presents an examination of students’ choices to seek feedback and to revise, as well as of students’ learning outcomes based on these choices. This game-based assessment approach is empirically validated with three research studies sampling nearly 300 middle-school and college students who played Posterlet and completed a posttest. Results show that the game helps students learn, as students who play the game before completing the posttest learn more graphic design principles than students who only complete the posttest. Moreover, the choices to seek critical feedback and to revise can predict learning and can be used as valid outcome measures for learning. Findings can be used in developing and evaluating models of instruction and assessment that may help students make informed learning choices. A discussion of present and future trends in theory regarding digital feedback environments is also included.     

Exploration of a Blended Learning Approach to Reading Instruction for Low SES Students in Early Elementary Grades

This study investigated the potential benefits of a blended learning approach on the reading skills of low socioeconomic status students in Grades 1 and 2. Treatment students received English language arts instruction that was both teacher-led and technology-based. Comparisons were made with control students who received the same English language arts instruction without the blended learning component. Results showed significantly greater pretest/posttest gains on a standardized reading assessment for the treatment students compared to the control students. The greatest discrepancy occurred in reading comprehension. A sub-analysis of low-performing English language learner students in the treatment group revealed the largest reading gains. At posttest, these students performed at the level of non-English language learner students in the control group. Results indicated a blended learning approach can be effective in enhancing the reading skills of low socioeconomic students.     

Student Learning about Biomolecular Self-Assembly Using Two Different External Representations

Self-assembly is the fundamental but counterintuitive principle that explains how ordered biomolecular complexes form spontaneously in the cell. This study investigated the impact of using two external representations of virus self-assembly, an interactive tangible three-dimensional model and a static two-dimensional image, on student learning about the process of self-assembly in a group exercise. A conceptual analysis of self-assembly into a set of facets was performed to support study design and analysis. Written responses were collected in a pretest/posttest experimental design with 32 Swedish university students. A quantitative analysis of close-ended items indicated that the students improved their scores between pretest and posttest, with no significant difference between the conditions (tangible model/image). A qualitative analysis of an open-ended item indicated students were unfamiliar with self-assembly prior to the study. Students in the tangible model condition used the facets of self-assembly in their open-ended posttest responses more frequently than students in the image condition. In particular, it appears that the dynamic properties of the tangible model may support student understanding of self-assembly in terms of the random and reversible nature of molecular interactions. A tentative difference was observed in response complexity, with more multifaceted responses in the tangible model condition.     

Effects of different student response modes on science learning

Student response systems (SRSs) are wireless answering devices that enable students to provide simple real-time feedback to instructors. This study aims to evaluate the effects of different SRS interaction modes on elementary school students’ science learning. Three interaction modes which include SRS Individual, SRS Collaborative, and Classroom were examined. A pretest–posttest quasi-experimental design was used to compare the effects of the three interaction modes on science posttest scores for higher order thinking (HOT) questions as well as lower order thinking questions. The results show that SRS Collaborative mode has significant positive effects on science posttest scores of HOT questions. This study points to the potential benefits of using SRS, particularly for collaborative tasks, to solve problems that require HOT skills.     

The Effect of a Community Project on Sense of Community among Social Work Students: An Exploratory Evaluative Study

This article evaluates the effect of a community project that was adopted by the students of a school of social work in Israel. A quasi-experimental pretest–posttest design was used. The pretest was administered during the planning stage of the project and the posttest a year later, during the implementation period. The findings show a significant positive effect of the project on sense of community among the students. This effect was moderated by students’ ethnic affiliation (Arab or Jewish) and professional commitment. The implications for using community projects to develop a sense of community in schools of social work are discussed.