The effectiveness of an adaptive digital educational game for the training of early numerical abilities in terms of cognitive,noncognitive and efficiency outcomes

Adaptive educational games provide new opportunities to train early numerical skills. However, empirical evidence for the effectiveness of adaptive educational games is scarce. This study investigated the effectiveness of an adaptive game compared to a nonadaptive game in terms of cognitive, noncognitive and efficiency outcomes. In total, 84 children were randomly assigned to a condition in which children trained early numerical skills with an adaptive version of the Number Sense Game (NSG), or to a condition in which they trained with a nonadaptive version. Early numeracy was evaluated before the training, immediately after the training and 3 weeks after the training. Math anxiety (MA) was assessed before and 3 weeks after the training. The time children practiced with the NSG was used to assess efficiency. Results revealed that children in both conditions improved on early numerical ability, with sustained effects 3 weeks after the training. In both conditions, children’s MA scores were lower after the training. Children in the adaptive condition learned more efficiently compared to the nonadaptive condition, and the interaction between prior knowledge and condition has shown that children with low prior knowledge benefited more from a nonadaptive training while children with high prior knowledge benefited more from an adaptive training in terms of learning efficiency. These results confirm that adaptive educational games can offer solace in terms of the need for differentiation.     

Instructional control of cognitive load in the training of complex cognitive tasks

Limited processing capacity constrains learning and performance in complex cognitive tasks. In traditional instruction, novices' failure to adequately learn cognitive tasks can often be attributed to the inappropriate direction of attention and the related high or excessive load that is imposed on a learner's cognitive system. An instructional design model for the training of complex cognitive tasks should provide instructional strategies that control cognitive load. We propose such a model and recommend research in which the cognitive load of instructional manipulations is systematically investigated and determined with mental-effort based measures.     

Obtaining Diagnostic Classification Model Estimates Using Mplus

Diagnostic classification models (aka cognitive or skills diagnosis models) have shown great promise for evaluating mastery on a multidimensional profile of skills as assessed through examinee responses, but continued development and application of these models has been hindered by a lack of readily available software. In this article we demonstrate how diagnostic classification models may be estimated as confirmatory latent class models using Mplus, thus bridging the gap between the technical presentation of these models and their practical use for assessment in research and applied settings. Using a sample English test of three grammatical skills, we describe how diagnostic classification models can be phrased as latent class models within Mplus and how to obtain the syntax and output needed for estimation and interpretation of the model parameters. We also have written a freely available SAS program that can be used to automatically generate the Mplus syntax. We hope this work will ultimately result in greater access to diagnostic classification models throughout the testing community, from researchers to practitioners.     

儿童在平衡秤任务上的认知规则及评估方法

平衡秤任务是儿童高级认知策略研究非常著名的一种形式,Siegler提出的规则评估技术极大地推动了解题规则的相关研究,但在应用过程中显现了规则评估技术的不稳定性等诸多局限。目前,平衡秤任务的解题规则的研究方法发展迅速,基于潜变量的分析方法已克服了这些不足,在统计方面具有明显优势。因此,我国教育与心理研究者应关注使用潜在类别分析等方法解决高级认知策略研究中的各种问题。     

认知诊断评价及其关键技术

认知诊断测验是近年来兴起的测验形式,受到越来越多研究人员的关注。认知诊断测验与传统的纸笔测验、计算机自适应测验最大的不同之处在于它能够提供被试者在知识上的详细掌握情况,能够让被试者了解自身的长处和不足,可以进行有针对性的补救学习;能够让指导者了解学生在知识上的优点和弱点,可以进行有针对性的补救教学。认知诊断测验是适应现代教育的发展而产生的,认知诊断测验的功能虽然强大,但是要真正大规模的实现,却也不是容易的事情,因为实施认知诊断测验涉及很多关键技术,并且有的技术目前还不成熟。对实施认知诊断测验的关键技术进行系统性的介绍,以使更多的研究人员投身到诊断测验的研究中来,促进认知诊断测验的发展。     

Psychometric stability of nationally normed and experimental decoding and related measures in children with reading disability

Achievement and cognitive tests are used extensively in the diagnosis and educational placement of children with reading disabilities (RD). Moreover, research on scholastic interventions often requires repeat testing and information on practice effects. Little is known, however, about the test-retest and other psychometric properties of many commonly used measures within the beginning reader population, nor are these nationally normed or experimental measures comparatively evaluated. This study examined the test-retest reliability, practice effects, and relations among a number of nationally normed measures of word identification and spelling and experimental measures of achievement and reading-related cognitive processing tests in young children with significant RD. Reliability was adequate for most tests, although lower than might be ideal on a few measures when there was a lengthy test-retest interval or with the reduced behavioral variability that can be seen in groups of beginning readers. Practice effects were minimal. There were strong relations between nationally normed measures of decoding and spelling and their experimental counterparts and with most measures of reading-related cognitive processes. The implications for the use of such tests in treatment studies that focus on beginning readers are discussed.     

Pointing to parallels in ability-related differences in the use of metacognition in academic and psychomotor tasks

This paper offers a brief review of the use of metacognition by proficient and poor performers in academic and psychomotor tasks as well as highlights the parallels and provides directions for future research. Metacognition is knowledge about one's own cognitive processes [Flavell, J.H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906–911.]. To date, the study of the use of metacognition by children with different levels of abilities (from those having a learning disability to those identified as gifted) has been mostly restricted to the cognitive abilities in academic areas such as reading, writing, or mathematics. The structure of knowledge has been more extensively explored in the expertise literature in the performance of both academic and psychomotor tasks. Similarities have been noted in the characteristic differences between experts and novices in both these types of tasks. Studies have begun to explore the use of metacognition in psychomotor tasks such as key strokes, ball throwing–catching, and running. It seems that, as with the structure of knowledge, parallels also exist in the use of metacognition by poor and proficient performers in academic and psychomotor tasks.     

Latent variable models: Applications in education

Most theories in the social sciences involve relationships among constructs which are not directly observable. Behavioral measures exist of all constructs, such as intelligence, creativity, and other cognitive traits, aggressiveness, sociability, and other personality and affective characteristics; but these observed measures are usually assumed to be imperfect indicators of the (presumed) underlying construct. The imperfections exist because of errors of measurement and because the observed behavior may be influenced by other underlying constructs in addition to the one of primary interest. Statistical methods, called latent variable models, have been developed to provide rigorous tests of theories involving unobserved variables. This paper describes the major types of latent variable models, shows how they can be applied in educational research, and gives representative examples of their use from the literature.     

Measurement of Executive Functioning Using a Playful Robot in Kindergarten

Abstract

We explored the potential of a robotics application in education as a measurement tool of child executive functioning skills. Sixty-five kindergarteners received assignments to go through a maze with a programmable robot, the Bee-Bot. Via observation we quantified how they solved these tasks. Their performance was successfully aggregated into a latent variable, which was used to predict the outcomes on standardized tasks that measure executive functioning. The latent variable significantly predicted performance on several tasks that measure problem-solving abilities, memory, visuospatial abilities and attention. It did not significantly predict pencil-and-paper tasks that measured visuospatial ability and nonverbal or design fluency. This study showed that it is possible to use a playful robotics task to obtain information on children’s ability. We recommend more research on using diverse robots in larger samples with different age groups to further explore the possibilities of robots as a test instrument.     

Challenges in measuring wisdom-relevant constructs in young adult undergraduate students

ABSTRACT

University settings present a unique opportunity for young adults to develop characteristics constitutive of wisdom. One challenge for educators working to support this development involves effectively measuring these characteristics. In this article, we present results from a secondary analysis of cognitive interviews to examine challenges that may arise when young adult undergraduate students respond to self-report measures of Meaning, Identification with all Humanity and Dedication to Lifelong Service—three characteristics of personality-focused accounts of wisdom. Although the items had well-fitting exploratory and confirmatory latent variable models, between 12.5% and 87.5% of students (n = 8 for each item) provided unintended interpretations for each item. This mismatch between students’ interpretations and the intended interpretations can threaten item validity and obfuscate findings from applications such as pre-post assessments. Further research is needed to better understand the factors affecting young adults’ responses to measures of wisdom-relevant characteristics.     

Revitalizing Algebra: the effect of the use of a cognitive tutor in a remedial course

This study investigated the effect of instruction with a cognitive tutoring software system in a remedial algebra course. The performance on algebra tasks of students who attended the experimental (cognitive tutor) and a control class was compared. The results indicated that the two groups of students were equally proficient with respect to algebraic manipulation skills. However, students who attended the experimental algebra section performed significantly better in problem solving than students in the control section. This finding suggested that the use of the cognitive tutor (a) improved students' problem‐solving abilities; (b) fostered student development of richer concepts of variable and function; and (c) improved students' procedural abilities in approaching and carrying through mathematical analyses of relatively complex situations.     

Does the Brain Have a Place in Educational Psychology?

Byrnes and Fox (1998) present the case for the relevance of cognitive neuroscience in educational psychology, including both logical and empirical arguments. In this commentary, I begin by briefly reviewing the history of the case for including the brain in educational psychology: Early educational psychology—as reflected in Thorndike's (1926) educational psychology textbook—emphasized the neuronal basis of learning; contemporary educational psychology—as reflected in educational publications—tends to ignore the brain; and future educational psychology will need to overcome the pitfalls encountered in previous misuses of brain research. Next, I examine two logical arguments for Byrnes and Fox's case, namely, that including cognitive neuroscience research makes educational psychology more complete and more plausible. Then, I examine the empirical argument of Byrnes and Fox by focusing on the value of cognitive neuroscience research in attention and memory as well as in reading and arithmetic. Finally, I suggest criteria for evaluating the contributions of cognitive neuroscience research in educational psychology, including the need for research on educationally relevant tasks and issues.     

Effects of cognitive demand on situational interest and running task performances

This study examined the effect of cognitive demand on situational interest and performance using running tasks in physical education. Adolescents (N = 179) participated in a field study involving three different levels of cognitive demand. Running performances and situational interest were measured four times with a testing interval of seven days. Results from multivariate analyses of covariance suggested that cognitive demand had a significant effect on situational interest and running performances. Higher cognitive demand led to higher levels of situational interest. Situational interest was weakly or moderately correlated with running performances (|r| = .17?.40), yet students reported significantly lower performance when the perceived interest was the highest. These findings suggest the importance of designing instructional tasks with moderate cognitive demand to optimally preserve situational interest and performance in physical education. The findings also help further conceptualise the relations among cognitive demand, situational interest and performance.     

Variable‐Length Computerized Adaptive Testing Using the Higher Order DINA Model

Cognitive diagnosis models provide profile information about a set of latent binary attributes, whereas item response models yield a summary report on a latent continuous trait. To utilize the advantages of both models, higher order cognitive diagnosis models were developed in which information about both latent binary attributes and latent continuous traits is available. To facilitate the utility of cognitive diagnosis models, corresponding computerized adaptive testing (CAT) algorithms were developed. Most of them adopt the fixed‐length rule to terminate CAT and are limited to ordinary cognitive diagnosis models. In this study, the higher order deterministic‐input, noisy‐and‐gate (DINA) model was used as an example, and three criteria based on the minimum‐precision termination rule were implemented: one for the latent class, one for the latent trait, and the other for both. The simulation results demonstrated that all of the termination criteria were successful when items were selected according to the Kullback‐Leibler information and the posterior‐weighted Kullback‐Leibler information, and the minimum‐precision rule outperformed the fixed‐length rule with a similar test length in recovering the latent attributes and the latent trait.     

Exploring the Utility of Background and Cognitive Variables in Explaining Latent Differential Item Functioning: An Example of the PISA 2009 Reading Assessment

Differential item functioning (DIF) may be caused by an interaction of multiple manifest grouping variables or unexplored manifest variables, which cannot be detected by conventional DIF detection methods that are based on a single manifest grouping variable. Such DIF may be detected by a latent approach using the mixture item response theory model and subsequently explained by multiple manifest variables. This study facilitates the interpretation of latent DIF with the use of background and cognitive variables. The PISA 2009 reading assessment and student survey are analyzed. Results show that members in manifest groups were not homogenously advantaged or disadvantaged and that a single manifest grouping variable did not suffice to be a proxy of latent DIF. This study also demonstrates that DIF items arising from the interaction of multiple variables can be effectively screened by the latent DIF analysis approach. Background and cognitive variables jointly well predicted latent class membership.     

An analysis of the programme for international student assessment 2000 (PISA 2000) mathematical literacy data for brazilian, japanese and norwegian students

A linear structural model was studied to investigate the factors affecting reading literacy and mathematical literacy skills of 15-year-old students in the Programme for International Student Assessment (PISA) 2000 data across different cultural settings. Brazil, Japan, and Norway were selected for the purpose of comparison on the basis of their rankings in the PISA 2000 study. The factors studied were attitudes towards reading, student-teacher relations, classroom climate, communication with parents, use of technology, attitudes towards mathematics, and reading literacy. The results indicated that the latent independent variable with the strongest effect on mathematical literacy is the use of technology in Brazil, communication with parents in Japan, and attitudes towards reading in Norway. In all the three countries, reading literacy has the strongest effect on mathematical literacy skills; mathematical literacy has a stronger relation to attitudes towards mathematics; attitudes towards reading is negatively related to mathematical literacy measures but positively related to reading literacy measures, and finally, communication with parents has a positive relation with reading literacy skills. A disciplined classroom environment fosters more success in PISA tests in Japan; on the other hand, in Brazil a reversed result is found for this particular variable. The use of technology has a strong influence on reading skills in Brazil; however, no and negative effects of this variable are observed in Norway and Japan respectively. These findings are discussed with reference to cultural context.     

工作记忆:人类高级认知活动的核心

工作记忆是个体在执行认知任务过程中,对信息暂时保持与操作的能力。近年来的诸多研究指出,工作记忆在人类智力、学习、推理、创造力等高级认知活动中起重要的作用。潜变量分析揭示了工作记忆的中央执行系统与高级认知活动之间存在着密切关系;而脑成像研究显示,工作记忆与高级认知活动的共享脑机制可能是两者关系紧密的主要原因;通过对个体工作记忆能力的训练,提升其阅读能力、智力水平等高级认知能力的干预研究,则进一步证明了工作记忆在人类高级认知活动中的核心影响。     

Defining and Evaluating Models of Cognition Used in Educational Measurement to Make Inferences About Examinees' Thinking Processes

The purpose of this paper is to define and evaluate the categories of cognitive models underlying at least three types of educational tests. We argue that while all educational tests may be based—explicitly or implicitly—on a cognitive model, the categories of cognitive models underlying tests often range in their development and in the psychological evidence gathered to support their value. For researchers and practitioners, awareness of different cognitive models may facilitate the evaluation of educational measures for the purpose of generating diagnostic inferences, especially about examinees' thinking processes, including misconceptions, strengths, and/or abilities. We think a discussion of the types of cognitive models underlying educational measures is useful not only for taxonomic ends, but also for becoming increasingly aware of evidentiary claims in educational assessment and for promoting the explicit identification of cognitive models in test development. We begin our discussion by defining the term cognitive model in educational measurement. Next, we review and evaluate three categories of cognitive models that have been identified for educational testing purposes using examples from the literature. Finally, we highlight the practical implications of "blending" models for the purpose of improving educational measures .     

Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions

Traditionally, Cognitive Load Theory (CLT) has focused on instructional methods to decrease extraneous cognitive load so that available cognitive resources can be fully devoted to learning. This article strengthens the cognitive base of CLT by linking cognitive processes to the processes used by biological evolution. The article discusses recent developments in CLT related to the current view in instructional design that real-life tasks should be the driving force for complex learning. First, the complexity, or intrinsic cognitive load, of such tasks is often high so that new methods are needed to manage cognitive load. Second, complex learning is a lengthy process requiring learners motivational states and levels of expertise development to be taken into account. Third, this perspective requires more advanced methods to measure expertise and cognitive load so that instruction can be flexibly adapted to individual learners needs. Experimental studies are reviewed to illustrate these recent developments. Guidelines for future research are provided.