Problems and Issues in Translating International Educational Achievement Tests

The article reviews research and findings on problems and issues faced when translating international academic achievement tests. The purpose is to draw attention to the problems, to help to develop the procedures followed when translating the tests, and to provide suggestions for further research. The problems concentrate on the following: the unique and demanding purpose of the translation task, the partly contradictory task specifications and translation instructions, the indecision as to whether to produce one or two target versions, the indecision as to whether to use one or two source versions, inadequate revision and verification, deficient translator competences, and a lack of time. To solve the problems, the article suggests the following: ensuring that the translation guidelines provide a right, unequivocal, and balanced picture of the purpose of the translation task; ensuring the equivalence of the two source versions; putting more emphasis on revision, and ensuring that the verification is sufficiently thorough; using only qualified translators, providing them with training in test translation, and including also subject matter and testing specialists in the translation teams; and allotting sufficient time to the translation work. However, the main lesson from the review is that more research in the field is badly needed.     

Alternative Interpretations of Alternative Assessments: Some Validity Issues in Educational Performance Assessments

The use of alternative assessments has led many researchers to reexamine traditional views of test qualities, especially validity. Because alternative assessments generally aim at measuring complex constructs and employ rich assessment tasks, it becomes more difficult to demonstrate (a) the validity of the inferences we make and (b) that these inferences extrapolate to target domains beyond the assessment itself. An approach to addressing these issues from the perspective of language testing is described. It is then argued that in both language testing and educational assessment we must consider the roles of both language and content knowledge, and that our approach to the design and development of performance assessments must be both construct-based and task-based.¹     

Opportunities to Learn and Achievement in Mathematics in a Sample of Sixth Grade Students in Lima, Peru

In recent years, the mathematics achievement of Peruvian students has been evaluated in three national and two international studies. The results in all cases suggest very poor learning. A similar situation is found in many developing countries. In this study, we analyzed the opportunities to learn (OTL) mathematics of sixth grade students from 22 public schools in Lima, Peru. OTL were defined in this study as curriculum coverage, cognitive demand of the tasks posed to the students, percent of mathematical exercises that were correct and quality of feedback. These variables were coded in the workbooks and notebooks of the students, which were gathered at the end of the school year (at the same time, the mathematics achievement test was administered). The results show that: (a) regarding OTL, less than half of the exercises available in the workbooks were solved, teachers overemphasize some topics of the national curriculum ({i.e.} related to Number and Number Sense), they pose tasks that are at very low levels of cognitive demand, and it is common to find mistakes in the students' answers to problems that have no feedback (or even worse, the feedback is wrong); (b) students in relatively poorer, multigrade classrooms have less OTL; (c) OTL, as defined earlier, is positively associated with achievement.A preliminary version of this paper received the Research Medal in the category Knowledge, Education andTechnology at the Annual Meeting of the Global Development Network (GDN) in Cairo, in January 2003.     

Dynamic Assessment of Young Children: Educational and Intervention Perspectives

Dynamic/interactive assessment (DA) has been motivated by the inadequacy of conventional static tests to provide accurate information about the individual's learning ability, specific deficient functions, change processes, and mediational strategies that are responsible for cognitive modifiability. A growing need for DA with young children has emerged recently because decisions about treatment should be made as early as possible. The objectives of this paper are to review the major criticism of the standardized static testing approach, present the theoretical basis of the DA approach, describe current research on DA of young children within educational and intervention perspectives, and suggest some directions for future research. The theoretical foundations of the presented research derive from Vygotsky's sociocultural theory, specifically the zone of proximal development concept, and Feuerstein's mediated learning experience theory. DA has been applied with different clinical and educational groups and was found to be more accurate in reflecting children's learning potential than static tests, especially with minority and learning disabled children. The mediational strategy is also reported as more effective than other intervention approaches (e.g., graduated prompt). The DA approach was found useful in assessing outcome effects of cognitive education programs aimed at enhancing learning how to learn skills. Crucial issues of cost-effectiveness, training, reliability and validity, and generalization are discussed.     

新课程背景下的学业评价:测量理论的价值

当前学业评价中存在的主要问题是:尚未形成一套规范的学业评价体系;缺乏对考试信息的深度挖掘;未能体现考试的诊断和发展功能。这些问题的解决在很大程度上有赖于真正将心理与教育测量理论应用于学业评价之中。在新课程背景下的学业评价应以能力导向性、发展性和规范性为三大原则,把测量理论与技术运用于学业评价中,建立一套规范化的学业评价体系,借助认知诊断理论充分挖掘考试信息,具体体现学业评价的诊断功能。     

Standardization and UNDERSTANDardization in Educational Assessment

Educational tests are standardized so that all examinees are tested on the same material, under the same testing conditions, and with the same scoring protocols. This uniformity is designed to provide a level “playing field” for all examinees so that the test is “the same” for everyone. Thus, standardization is designed to promote fairness in testing. In practice, the material tested, the conditions under which a test is administered, and the scoring processes, are often too rigid to provide the intended level playing field. For example, standardized testing conditions may interact with personal characteristics of examinees that affect test performance, but are not construct-relevant. Thus, more flexibility in standardization is needed to account for the diversity of experiences, talents, and handicaps of the incredibly heterogeneous populations of examinees we currently assess. Traditional standardization procedures grew out of experimental psychology and psychophysics laboratories where keeping all conditions constant was crucial. Today, accounting for and measuring what is not constant across examinees is crucial to valid construct interpretations. To meet this need I introduce the concept of understandardization, which refers to ensuring sufficient flexibility in standardized testing conditions to yield the most accurate measurement of proficiency for each examinee.     

Defining and Evaluating Models of Cognition Used in Educational Measurement to Make Inferences About Examinees' Thinking Processes

The purpose of this paper is to define and evaluate the categories of cognitive models underlying at least three types of educational tests. We argue that while all educational tests may be based—explicitly or implicitly—on a cognitive model, the categories of cognitive models underlying tests often range in their development and in the psychological evidence gathered to support their value. For researchers and practitioners, awareness of different cognitive models may facilitate the evaluation of educational measures for the purpose of generating diagnostic inferences, especially about examinees' thinking processes, including misconceptions, strengths, and/or abilities. We think a discussion of the types of cognitive models underlying educational measures is useful not only for taxonomic ends, but also for becoming increasingly aware of evidentiary claims in educational assessment and for promoting the explicit identification of cognitive models in test development. We begin our discussion by defining the term cognitive model in educational measurement. Next, we review and evaluate three categories of cognitive models that have been identified for educational testing purposes using examples from the literature. Finally, we highlight the practical implications of "blending" models for the purpose of improving educational measures .     

效度在语言测试中的重要性

讨论了效度在四种测试中的重要作用，以及各种效度之间的关系，说明要想达到测试的目的，设计试卷一定要把握好效度这个标准。     

The Use and Misuse of Standardized Testing: A Whiteheadian Point of View

First an overview is given about the actual national and international situation concerning standardized testing. Then two major reasons are presented why accountability systems based on standardized testing have become so widespread: (a) the missing validity and reliability of teachers’ assessment of students’ achievement, and (b) the important role standardized testing plays for out-put management in educational systems.On the basis of these considerations Alfred North Whitehead’s critical remarks on external standardized testing are presented. Whitehead’s main point is that external standardized testing limits the freedom of teachers to adapt to the complex, situation specific circumstances in order to obtain a maximum of the creative learning process for students who are conceived as specialists. Instead external testings lead to “teaching to the test.” As a consequence, the attitude of creative, adventurous exploration is undermined and substituted by simple pattern recognition, narrow visions, and even boredom. Finally, the question is raised whether there is any possibility of developing a measurement tool which on the one side meets scientific test criteria, and on the other side is still flexible enough to do justice to needs of individual schools – which according to Whitehead are the essential educational unit – and not vice versa, as it is the case at present with external standardized testing. That such a flexible approach to evaluation is possible is demonstrated by the presentation of the basic ideas of the MSS (Module Approach to Self-Evaluation of School Development Projects) which has been developed and examined in ten schools by the author and his collaborators.     

Testing Students with Special Needs: A Model for Understanding the Interaction Between Assessment and Student Characteristics in a Universally Designed Environment

This article presents a model of assessment development integrating student characteristics with the conceptualization, design, and implementation of standardized achievement tests. The model extends the assessment triangle proposed by the National Research Council ( Pellegrino, Chudowsky, & Glaser, 2001 ) to consider the needs of students with disabilities and English learners on two dimensions: cognitive interaction and observation interaction. Specific steps in the test development cycle for including students with special needs are proposed following the guidelines provided by Downing (2006) . Because this model of test development considers the range of student needs before test development commences, student characteristics are supported by applying the principles of universal design and appropriately aligning accommodations to address student needs. Specific guidelines for test development are presented.     

中学生生活技能量表的编制和信效度检验

编制中学生生活技能量表,并检测其信度和效度。方法:用量表评估初高中四个班学生211人,分析量表的信效度。结果:①新编量表有较好的信效度。②我国中学生生活技能水平总体较低。结论:①该量表有较满意的心理测量指标。②需要在我国中学生中开展系统的学校生活技能教育。     

Educational Opportunities in the Netherlands: Policy, Students' Performance and Issues

Since 1985 the Educational Priority Policy (EPP) has been in effect in the Netherlands. This policy is aimed at reducing the educational disadvantage of children due to their social, economic and cultural circumstances. The first section of this article describes the outlines of the EPP. The second section presents the results of an EPP evaluation study conducted in secondary education among 20,000 students from nearly 400 schools. The emphasis is on test performance in relation to three student characteristics: gender, social class and ethnic background. The analyses show that the test results are largely determined by social class. Gender hardly plays a role, while the influence of ethnic origin is also limited. It should however be added that the performance of two migrant groups, the Turks and Moroccans, is the worst of all. As most Turks and Moroccans belong to the lowest socioeconomic category, the factor of ethnic origin is so closely interwoven with the factor of social class, that it is not really possible to distinguish the two. In the final section of the article these findings are related to three theoretical perspectives regarding the explanation of differences in educational performance. This is followed by a discussion on possible consequences for the EPP.     

神经教育学的教育观：思想启示与现实应用

如何使学习者高效内化多元知识并形成和操用心理表征体系来发展创造内外世界,乃是当代人类所面对的重大挑战。近年来问世的神经教育学,旨在借助新型的心脑科学原理,实现对教学课程、教学方法和教育—学习的结构性改善与功能性创新,以此有效促进教师的先导性发展,继而推动学生的情知意行获得主动自由、全面和谐发展。其关于人类记忆机制、情感体验的神经原理和认知决策的大脑高阶过程等重要发现,都为我们改善教育行为提供了深刻的科学启示。应当将知识传导与内化过程首先落实到教师的认知操作这个核心环节上,以期借此实现教学主体与学习主体的共同发展。     

The Educational Achievement of Pupils in Differentiated and Undifferentiated Groups

Responses on a Likert-type instrument, the Eight College Professors (Form B), from 664 education juniors and seniors in five small colleges were analyzed to examine the preferential rankings given four types of social science professors as an instructor. While some suggestive variations were observed, the overall order of preference was Teacher—Socialite—Researcher—Administrator (the least preferred) . This was in contrast to earlier findings on university students who placed Researcher ahead of Socialite. Female professors were consistently less preferred to male professors by college students of both sexes.     

口语报告法在图形推理难度分析中的应用

尝试用事后口头报告法对被试完成R.APM的思维过程进行研究,结合对R.APM项目刺激特征分析进行文本编码,分析R.APM图形推理的难度等级。结果发现：（1）口语报告法与项目刺激特征分析理论的结合有助于解决该方法使用中的编码问题;（2）口语报告法在图形推理难度分析中具有可行性,但不能作为实时考察被试思维过程的方法。     

The Complex Model of Television Viewing and Educational Achievement

Abstract

Six studies containing data obtained from over 1 million students in elementary, intermediate, and high school were meta-analyzed to examine the relationship between amount of television viewing and educational achievement. The findings led to the formulation of the complex viewing—achievement model. According to the model, the function relating achievement to viewing can be described as an inverted check mark. For small amounts of viewing, achievement increased with viewing, but as viewing increased beyond a certain point, achievement decreased. That function was found for each of the 3 ages studied, but optimal viewing time—the apex of the function—was different at each age and decreased with the age of the students.     

The impact of a rubric and friendship on peer assessment: Effects on construct validity,performance, and perceptions of fairness and comfort

Construct validity of peer assessment (PA) is important for PA application, yet difficult to achieve. The present study investigated the impact of an assessment rubric and friendship between the assessor and assessee on construct validity of PA. Two-hundred nine bachelor students participated: half of them assessed a peer's concept map with a rubric whereas the other half did not use a rubric. The results revealed a substantial reliability and construct validity for PA. All students over-score their peers’ performance, but students using a rubric were more valid. Moreover, when using a rubric a high level of friendship between assessor and assessee resulted in more over-scoring. Use of a rubric resulted in higher quality concept maps for peer and expert ratings.     

Meeting the NRC Panel's Recommendations: Commentary on the Papers by Mislevy and Haertel, Gorin, and Abedi and Gándara

In this commentary, we summarize some of the main themes of the NRC report and note ways in which the papers by Mislevy and Haertel, Gorin, and Abedi and Gándara address the Panel's recommendations. We then briefly review and offer reflections on each paper. We see much to applaud here and also in the broader effort to build bridges between the cognitive and measurement sciences. However, much work remains, not only in building bridges, be also in educating the next generation of psychometricians about cognition and the next generation of teachers about psychometrics. Until that day, even the best work here will have little impact on the business of large-scale assessment.     

The Control-Value Theory of Achievement Emotions: Assumptions, Corollaries, and Implications for Educational Research and Practice

This article describes the control-value theory of achievement emotions and its implications for educational research and practice. The theory provides an integrative framework for analyzing the antecedents and effects of emotions experienced in achievement and academic settings. It is based on the premise that appraisals of control and values are central to the arousal of achievement emotions, including activity-related emotions such as enjoyment, frustration, and boredom experienced at learning, as well as outcome emotions such as joy, hope, pride, anxiety, hopelessness, shame, and anger relating to success or failure. Corollaries of the theory pertain to the multiplicity and domain specificity of achievement emotions; to their more distal individual and social antecedents, their effects on engagement and achievement, and the reciprocal linkages between emotions, antecedents and effects; to the regulation and development of these emotions; and to their relative universality across genders and cultures. Implications addressed concern the conceptual integration of emotion, motivation, and cognition, and the need to advance mixed-method paradigms. In closing, implications for educational practice are discussed.