Highly integrated model assessment technology and tools

Effective and efficient measurement of the development of skill and knowledge, especially in domains of human activity that involve complex and challenging problems, is important with regard to workplace and academic performance. However, there has been little progress in the area of practical measurement and assessment, due in part to the lack of automated tools that are appropriate for assessing the acquisition and development of complex cognitive skills. In the last 2 years, an international team of researchers has developed and validated an integrated set of assessment tools called highly integrated model assessment technology and tools (HIMATT) which addresses this deficiency. HIMATT is web-based and has been shown to scale up for practical use in educational and workplace settings, unlike many of the research tools developed solely to study basic issues in human learning and performance. In this paper, we describe the functions of HIMATT and demonstrate several applications for its use. Additionally, we present two studies on the quality and usability of HIMATT. We conclude with research suggestions for the use of HIMATT and for its further development.     

Identifying cross-domain distinguishing features of cognitive structure

Our research aims to identify domain-specific similarities and differences of externalized cognitive structures. Cognitive structure, also known as knowledge structure or structural knowledge, is conceived as the manner in which an individual organizes the relationships of concepts in memory. By diagnosing these structures precisely, even partially, the educator comes closer to influencing them through instructional settings and materials. Our assessment and analysis of cognitive structures is realized within the HIMATT tool, which automatically generates four quantitative indicators for the structural entities of written text or causal maps. In our study, participants worked on the subject domains biology, history, and mathematics. Results clearly indicate different structural and semantic features across the three subject domains. Additionally, we found that written texts and causal maps seem to represent different structure and content across the three subject domains when compared to an expert??s representation. We conclude with a general discussion, instructional implications and suggestions for future research.     

Theoretically grounded guidelines for assessing learning progress: cognitive changes in ill-structured complex problem-solving contexts

It is generally accepted that the cognitive development for a wide range of students can be improved through adaptive instruction-learning environments optimized to suit individual needs (e.g., Cronbach, Am Psychol 12:671?C684, 1957; Lee and Park, in Handbook of research for educational communications and technology, Taylor & Francis Group, Routledge, New York, pp 469?C484, 2007; Shute and Zapata-Rivera, in Handbook of research for educational communications and technology, Taylor & Francis Group, Routledge, New York, pp 227?C294, 2007). It is vital in adaptive instruction to diagnose an individual??s learning progress in terms of cognitive changes in complex problem-solving contexts. This study conceptualized the levels of learning progress, associating the development of expertise in domain learning with the structural features of mental models. The theory of mental models accounts for how people conceptualize problem situations. That is, a mentally represented problem space is a structure including diverse relationships. It is necessary that assessment tools be adapted for the complex, dynamic structure of mental models so that the diagnostic, formative information become more precise. In short, the proposed stage-sequential model of learning progress was theoretically justified as being able to serve as a diagnostic model of learning progress.     

认知诊断理论在数学教育评价中的应用

质性地比较论证了基于认知诊断理论的数学教育评价工具开发的可行性。通过分析实际测试数据,比较几种不同认知诊断模型中的参数估计方法实际应用于分析数学评价测验的可能性。研究发现,认知属性概念可以帮助研究者和实践者分析影响学生解答数学题目背后的认知结构。人工神经网络模型能够充分利用理论设计的认知模型,克服测验题目有限、题型多样、认知属性差异大等不易分析的困难,较好评价学生的数学学业成就,为后续教学提供诊断性信息,达到了诊断性测验的目的。     

Formative assessment and teaching practice: the point of view of Italian teachers

Over the years the debate on the aims, approaches, and impacts of formative assessment has never stopped to grow. Parallel to this growth is the awareness that it is crucial to understand what conceptions teachers have of assessment in order to guarantee more effective teacher education. This paper tries to connect these two fields of research: formative assessment and teacher education through an analysis of teachers’ conceptions: What do teachers think about assessment? What aims do they pursue through it? Do teachers really distinguish between formative and summative assessment? On the backdrop of the formative assessment literature, the article reports on an exploratory research study. Limitations and issues are analysed in order to shed light on teacher education implication and on future educational research paths in the educational assessment field.     

大学英语教学评估理论与实践

教学评估是大学英语教学的一个重要环节。教学评估分为形成性评估和终结性评估。本文根据教学理论和实践,探讨了基于计算机和课堂的英语多媒体教学评估模式。     

数学方法论在高等数学教学中的作用

数学方法论是提高数学教学质量的有效工具,将数学方法论应用于概念教学中,展示概念的形成过程,用类比与联想揭示概念之间的联系、异同,注重概念体系的建立;将数学方法论应用于定理教学中,充分暴露如何用类比、归纳、演绎以及形象思维探索新证法的过程;将数学方法论用于解题教学,在讲解典型例题和习题时向学生传授常用的解题方法。从而达到帮助学生形成正确的数学观念和优秀的数学精神。     

基于网络的形成性考核的有效性和时效性分析

中央广播电视大学的形成性考核,素以单一的作业册的形式向各试点学校发行。由于师生见面时间有限,教师批改作业后很难及时反馈给学生,并且单一的纸笔式练习也有很大的局限性。针对这一情况,中央电大于2005年3月启动了“宏微观经济学”等六门课程的基于网络的课程形成性考核试点项目。基于网络的形成性考核能否提高评价的有效性和时效性,实现信息的及时反馈,改进远程教与学的质量呢?本文应用院校研究的方法,对基于网络的课程形成性考核的评阅率、评阅的及时性、评阅的方式与质量进行了分析。     

A case study of a formative assessment practice and the effects on students’ self-regulated learning

The study investigates one mathematics teacher’s implementation of formative assessment and its effects on students’ self-regulated learning (SRL). A questionnaire administered before and after the eight-month long intervention shows a significant effect, compared to two control classes, on students’ motivational beliefs involved in SRL. Qualitative data shows a notable enhancement of the students’ SRL behavior in the classroom. Analysis of the teacher’s implemented formative assessment shows a practice integrating several aspects of formative assessment, and provides empirical evidence of what formative assessment with large effects on students’ SRL may look like and how it fits with models of SRL development.     

Zwischen Schule und Universität: Argumentation in der Mathematik

Several studies show that university students in Germany still have problems in reasoning mathematically although this already should be fostered at high school since the implementation of standards for school mathematics. Mathematical argumentation is a core competence and highly important, especially in academic mathematics. To foster mathematical argumentation at the beginning of university studies, competence models are needed which give more detailed insights in the skills that are necessary for reasoning. As mathematical argumentation is a complex process, especially at the higher secondary level or at university, many little steps are needed to complete a competence model for argumentation at the secondary–tertiary transition gradually. A possible step can be to initially identify several aspects of mathematical argumentation competence that influence the reasoning quality. The empirical basis for identifying those aspects is a cross-sectional study with 439 engineering students who participate in a transition course in mathematics. We address the following questions: (1) how is the quality of student’s reasoning? (2) Which kind of arguments do students use? (3) What resources do students who reasoned correctly use for solving the problems? (4) Does the content of the tasks play an important role? The results show a great influence of the content on the reasoning quality, especially if the content is abstract or concrete. Argumentation quality of students decreases with an increasing level of abstraction of the content. Furthermore, the results reveal that students often use routines for solving the problems. That indicates that procedural approaches still play an important role in school mathematics. If procedures could be used for solving the tasks, students are more successful. Competence models for mathematical argumentation at the beginning of the tertiary level should, therefore, include these factors.     

Working toward integrated practice: Understanding the interaction among formative assessment strategies

The purpose of this study was to directly observe and investigate the factors that support or hinder the implementation of an integrated approach to formative assessment and to understand how the implementation of formative assessment strategies interact. The authors present findings from a collective case study that included the observation of six teachers (three mathematics and three English language arts) who were purposefully sampled from a large, public, suburban high school in the northeast United States. The analysis of the case studies resulted in the identification of different patterns in teachers’ formative assessment practice. These patterns illustrate how the complex interaction between formative assessment practices can either support or hinder the implementation of an integrated approach to formative assessment and suggests classroom observations as a potential mechanism for identifying challenges and informing just-in-time professional development.     

A systematic review of trends and findings in research employing drawing assessment in science education

ABSTRACT

In this study, we reviewed 76 journal articles on employing drawing assessment as a research tool in science education. Findings from the systematic review suggest four justifications for using drawing as a type of research tool, including assessment via drawing as (a) an alternative method considering young participants’ verbal or writing abilities, and affective or economic reasons, (b) a unique method that can reveal aspects not easily measured by other methods, (c) a major method that reflects characteristics of science subjects, and (d) a formative assessment to diagnose students’ ideas to benefit their learning. Furthermore, five research trends of studies using drawing as assessment tools are identified, including: (a) students’ conceptions of scientists from the Draw-a-Scientist-Test (DAST) and evolving studies, (b) students’ understanding or mental models of science concepts, (c) participants’ conceptions of science learning or teaching, (d) students’ inquiry abilities and modelling skills via drawing, and (e) technology to support drawing. For each trend, we synthesised and commented on the current findings. A framework conceptualising phases and issues when designing research and instruments employing drawing assessments is proposed. The review provides insights into the design and future direction of research employing drawing assessments in science education.     

Summative and Formative Assessments in Mathematics Supporting the Goals of the Common Core Standards

Being proficient in mathematics involves having rich and connected mathematical knowledge, being a strategic and reflective thinker and problem solver, and having productive mathematical beliefs and dispositions. This broad set of mathematics goals is central to the Common Core State Standards for Mathematics.

High-stakes testing often drives instructional practice. In this article, I discuss test specifications and sample assessment items from the two major national testing consortia and the prospects that their assessments will be positive levers for change.

For more than 20 years, the Mathematics Assessment Project has focused on the development of assessments that emphasize productive mathematical practices, most recently creating formative assessment lessons (FALs) designed to help teachers build up student understandings through focusing on student thinking while engaging in rich mathematical tasks. This article describes our recent work.     

Differential Improvement in Student Understanding of Mathematical Principles Following Formative Assessment Intervention

ABSTRACT

The authors describe results from a study of a middle school mathematics formative assessment strategy. They employed a randomized, controlled design to address the following question: Does using our strategy improve student performance on assessments of key mathematical ideas relative to a comparison group? Eighty-five teachers and 4,091 students were included. Students took a pretest and a transfer measure at the end of the year. Treatment students completed formative assessments. Treatment teachers had exposure to professional development and instructional resources. Results indicated students with higher pretest scores benefited more from the treatment compared to students with lower pretest scores. In addition treatment students significantly outperformed control students on distributive property items. This effect was larger as pretest scores increased. Results, limitations, and future directions are discussed.     

The layering of mathematical interpretations through digital media

How might understanding emerge when learners engage mathematical phenomena through digital technologies? This paper considers the ways children’s mathematical thinking was influenced by their interpretations through various pedagogical discourses and how understanding emerged through those various filters. Current research into using digital technologies in mathematics education is predominantly positioned within two theoretical perspectives, semiotic mediation and the instrumental approach. Meanwhile, within a contemporary hermeneutic position, mathematical learning comprises a process of interpretation, where understanding is seen as a formative process as the learner views events from fresh, ever-evolving perspectives. Concepts are then seen in ongoing formation. This paper examines learning through a contemporary hermeneutic frame and the contention is that this opens opportunity for learning to be perceived in alternative ways.     

From Evidence to Action: A Seamless Process in Formative Assessment?

Based on the results of a generalizability study of measures of teacher knowledge for teaching mathematics developed at the National Center for Research on Evaluation, Standards, and Student Testing at the University of California, Los Angeles, this article provides evidence that teachers are better at drawing reasonable inferences about student levels of understanding from assessment information than they are at deciding the next instructional steps. We discuss the implications of the results for effective formative assessment and end with considerations of how teachers can be supported to know what to teach next.     

新课程理念下中学数学学习过程评价的探究

数学学习过程评价，是数学新课程评价的核心内容．它是将评价与教师的教学与学生的学习活动有机地结合起来，把评价纳入学习活动过程之中，并将评价作为学生主动学习的一部分，以评价促进学生的学习和学生的发展．数学学习过程评价的目的是为了及时反馈调节学生数学学习活动，促进学生的发展，而不是为了检查．正确建构数学学习过程评价必须明确过程评价的意义、类型、内容、评价方法和呈现方式．