Correlates of Rapid-Guessing Behavior in Low-Stakes Testing: Implications for Test Development and Measurement Practice

Previous research has shown that rapid-guessing behavior can degrade the validity of test scores from low-stakes proficiency tests. This study examined, using hierarchical generalized linear modeling, examinee and item characteristics for predicting rapid-guessing behavior. Several item characteristics were found significant; items with more text or those occurring later in the test were related to increased rapid guessing, while the inclusion of a graphic in a item was related to decreased rapid guessing. The sole significant examinee predictor was SAT total score. Implications of these results for measurement professionals developing low-stakes tests are discussed.     

Changes in Intelligence Over a Quarter of a Century

The purpose of the investigation is to study changes in intelligence among Swedish 13 year‐olds during a 25‐year period. The results show that the average of spatial and reasoning intelligence has successively risen. Concerning verbal intelligence, there is a clear increase during the 1960's. After this period the growth has been more irregular. No changes in the differences between social groups were observed. On the other hand, this was the case between boys and girls. In verbal ability, there were only small differences between the sexes in 1961. However, in 1985 the girls were clearly ahead of the boys. In spatial and reasoning ability, the boys were superior in the early 1960's. Twenty‐five years later this difference had almost disappeared.     

Patterns of Solution Behavior across Items in Low-Stakes Assessments

The trustworthiness of low-stakes assessment results largely depends on examinee effort, which can be measured by the amount of time examinees devote to items using solution behavior (SB) indices. Because SB indices are calculated for each item, they can be used to understand how examinee motivation changes across items within a test. Latent class analysis (LCA) was used with the SB indices from three low-stakes assessments to explore patterns of solution behavior across items. Across tests, the favored models consisted of two classes, with Class 1 characterized by high and consistent solution behavior (>90% of examinees) and Class 2 by lower and less consistent solution behavior (<10% of examinees). Additional analyses provided supportive validity evidence for the two-class solution with notable differences between classes in self-reported effort, test scores, gender composition, and testing context. Although results were generally similar across the three assessments, striking differences were found in the nature of the solution behavior pattern for Class 2 and the ability of item characteristics to explain the pattern. The variability in the results suggests motivational changes across items may be unique to aspects of the testing situation (e.g., content of the assessment) for less motivated examinees.     

Changes in the Attitudes of Young Women Toward Child Guidance Over a Two-Year Period

Abstract

A non traditional model for teaching composition has extensive teaching and research value. The purpose of this research was to test the effect of oral discourse in teaching written discourse. The control group went from stimulus to problem to written discussion; the experimental group went from stimulus to problem to controlled oral discussion to written discussion on the model. Teachers evaluated unidentified papers of both pretest and posttest of control and experimental groups. With the use of the chi-square statistic, it was shown that the use of controlled oral discourse produced more grade increases and fewer decreases in written discourse in the experimental group than in the control group at the .01 level of significance.     

基于社会认知效度框架的大学英语听说教材课堂听力测评任务效度评析

借鉴社会认知框架中的“认知效度”和“语境效度”框架,对《全新版大学英语听说教程》(第二版)全6册教材中听力测试任务的效度进行评价。研究发现,教材中的听力习题对不同层次的听力认知能力考查不够全面,对听力元认知策略的培养体现不够突出,对作答过程中的书写负担控制不够。此外,建构性作答方式听力任务的设计有待系统性改进,听力语篇真实性有待提升,听力和口语任务的衔接不够紧密。针对这些问题,建议听力任务设计遵从科学严谨的测试编写方法和原则,以提升听力习题的认知效度和语境效度,并在此基础上进一步考察其在实际教学中的使用效果及师生反馈。     

The Role of School Climate in Explaining Changes in Social Trust Over Time

School is often ranked highly among social agents that are central to the development of various citizenship values and norms in younger members of society. In this paper, we examined the impact of two dimensions of school climate on changes in social trust among adolescents over time, namely relationships and safety. Using a series of latent change models on 3-wave panel data concerning roughly 850 Swedish adolescents aged 16 to 18 years, we found that experiences of victimization in school mattered most in predicting changes in social trust across 3 time points, when relevant demographic factors and other aspects of the school climate were controlled. In particular, social trust declined most among adolescents who experienced more victimization at school.     

Alternative Interpretations of Alternative Assessments: Some Validity Issues in Educational Performance Assessments

The use of alternative assessments has led many researchers to reexamine traditional views of test qualities, especially validity. Because alternative assessments generally aim at measuring complex constructs and employ rich assessment tasks, it becomes more difficult to demonstrate (a) the validity of the inferences we make and (b) that these inferences extrapolate to target domains beyond the assessment itself. An approach to addressing these issues from the perspective of language testing is described. It is then argued that in both language testing and educational assessment we must consider the roles of both language and content knowledge, and that our approach to the design and development of performance assessments must be both construct-based and task-based.¹     

Instructional Sensitivity as a Psychometric Property of Assessments

Standards-based reform, as codified by the No Child Left Behind Act, relies on the ability of assessments to accurately reflect the learning that takes place in U.S. classrooms. However, this property of assessments—their instructional sensitivity—is rarely, if ever, investigated by test developers, states, or researchers. In this paper, the literature on the psychometric property of instructional sensitivity is reviewed. Three categories of instructional sensitivity measures are identified—those relying on item or test scores only, those relying on item or test scores and teacher reports of instruction, and strictly judgmental methods. Each method identified in the literature is discussed alongside the evidence for its utility. Finally, recommendations are made as to the proper role of instructional sensitivity in the evaluation of assessments used under standards-based reform.     

Game Over? No Main or Subgroup Effects of the Good Behavior Game in a Randomized Trial in English Primary Schools

Abstract

This study aimed to examine the impact of a universal, school-based intervention, the Good Behavior Game (GBG), on children’s behavior, and to explore any subgroup moderator effects among children at varying levels of cumulative risk (CR) exposure. A 2-year cluster-randomized controlled trial was conducted comprising 77 primary schools in England. Teachers in intervention schools delivered the GBG, whereas their counterparts in control schools continued their usual provision. Behavior (specifically disruptive behavior, concentration problems, and pro-social behavior) was assessed via the checklist version of the Teacher Observation of Classroom Adaptation. A CR index was calculated by summing the number of risk factors to which each child was exposed. Multilevel models indicated that no main or subgroup effects were evident. These findings were largely insensitive to the modeling of CR although a small intervention effect on disruptive behavior was found when the curvilinear trend was used. Further sensitivity analyses revealed no apparent influence of the level of program differentiation. In sum, our findings indicate that the GBG does not improve behavior when implemented in this sample of English schools.     

中国高等教育六十年的变迁

新中国的高等教育已经年届花甲,与具有近千年的世界高等教育相比,显然是很短暂的。但是,在中国高等教育发展的历史上,却占据着十分重要的地位。这六十年的高等教育,是多事之秋的六十年,是天翻地覆的六十年,无疑也是发展最快和变化最大的六十年。要全面、系统地叙述我国高等教育六十年的变革与发展,显然作者是力不从心的。本文仅结合个人的亲身经历,就我国高等教育六十年变迁中的体制、规模、模式和人才规格等问题,谈谈笔者的一孔之见。     

信息化管理系统应用过程中管理者管理行为的变化

学校管理环境在信息化系统应用中发生了一系列的变化,因此,在学校信息化建设过程中,除了信息化系统设计上要有比较科学的规划外,更重要的是学校信息化系统建设要有一个科学的统筹规划,注重解决信息化系统应用过程中的非技术问题,以促进信息管理行为的变化,提高信息化管理系统的应用效率。     

Generalizability of Performance Assessments

How can the contributions of raters and tasks to error variance be estimated? Which source of error variance is usually greater? Are interrater coefficients adequate estimates of reliability? What other facets contribute to unreliability in performance assessments?     

在新课程理念下教师教学行为的变化

一切先进的教学改革都是从新的教育理念中产生出来的。先进的教育理念要通过先进的教育行为体现出来。教育理念的转变本身也要在教育行为的转变中进行。     

新课程改革中物理教师教学行为的转变

基础教育改革的目标之一是积极倡导学生"主动参与、积极探究、勤于思考",以培养学生"获取新知"、"分析和解决问题的能力".作为习惯了"旧的"教学方式的教师来说,要想适应新课程中的教学要求,必须在教学行为上有所转变.     

Probability Learning: Changes in Behavior Across Time and Development

Individuals track probabilities, such as associations between events in their environments, but less is known about the degree to which experience—within a learning session and over development—influences people's use of incoming probabilistic information to guide behavior in real time. In two experiments, children (4–11 years) and adults searched for rewards hidden in locations with predetermined probabilities. In Experiment 1, children (n = 42) and adults (n = 32) changed strategies to maximize reward receipt over time. However, adults demonstrated greater strategy change efficiency. Making the predetermined probabilities more difficult to learn (Experiment 2) delayed effective strategy change for children (n = 39) and adults (n = 33). Taken together, these data characterize how children and adults alike react flexibly and change behavior according to incoming information.