Curriculum-Based Measurement as a Predictor of Performance on a State Assessment: Diagnostic Efficiency of Local Norms

With the concurrent emphasis on accountability, prevention, and early intervention, curriculum-based measurement of reading (R-CBM) is playing an increasingly important role in the educational process. This study investigated the differences in diagnostic accuracy and utility between commercial norms and local norms when making high-stakes, local decisions. Scores on Dynamic Indicators of Early Literacy Skills Oral Reading Fluency for 1,374 students in Grades 2 to 5 were used to predict outcomes the Georgia reading achievement test, the Criterion Referenced Competency Tests. Local norms were generated using logistic regression and receiver operator characteristic curve analysis. The generated cut scores were compared to the commercial norms for differences in diagnostic efficiency. The generated cut scores were lower than the commercial norms and had improved diagnostic efficiency. Implications related to educational policy and the use of R-CBM are discussed.     

Content Validity Studies of Licensing Examinations

Can measurement specialists’current ideas about content validation be implemented with licensure examinations? Does pressure of litigation facilitate or inhibit conducting validity studies?     

Pedagogical Content Knowledge in Social Studies

Abstract: Gudmundsdottir, S. & Shulman, L. 1987. Pedagogical Content Knowledge in Social Studies. Scandinavian Journal of Educational Research 31, 59‐70. The role of teacher's pedagogical content knowledge in social studies is addressed through two case studies: a novice and a veteran teacher. We demonstrate that the important difference between the novice and the expert is manifested in a special kind of knowledge that is neither content nor pedagogy per se. It rests instead in pedagogical content knowledge, a form of teacher understanding that combines content, pedagogy and learner characteristics in a unique way.     

Validity of Curriculum-Based Scales With Young Neurodevelopmentally Disabled Children: Implications for Team Assessment

Recent surveys in early childhood special education and developmental school psychology reveal that criterion-referenced or curriculum-based developmental assessment (CBA) is the primary form of measurement employed by interdisciplinary early intervention teams. Despite their wide use, little technical adequacy data are available on the validity of CBA instruments with specific populations of handicapped preschoolers. This research analyzed the concurrent validity of two norm-based and six curriculum-based scales commonly used within community early intervention programs with a sample of 50 infants and preschool children having diverse neurodevelopmental and neurobehavioral deficits. Results supported the concur- rent validity of CBA measures despite some significant interscale differences. Based on these results, guidelines are offered regarding “best practices” in the use of CBA systems by early intervention teams.     

Validity of Curriculum-Based Scales With Young Neurodevelopmentally Disabled Children: Implications for Team Assessment

Recent surveys in early childhood special education and developmental school psychology reveal that criterion-referenced or curriculum-based developmental assessment (CBA) is the primary form of measurement employed by interdisciplinary early intervention teams. Despite their wide use, little technical adequacy data are available on the validity of CBA instruments with specific populations of handicapped preschoolers. This research analyzed the concurrent validity of two norm-based and six curriculum-based scales commonly used within community early intervention programs with a sample of 50 infants and preschool children having diverse neurodevelopmental and neurobehavioral deficits. Results supported the concur- rent validity of CBA measures despite some significant interscale differences. Based on these results, guidelines are offered regarding "best practices" in the use of CBA systems by early intervention teams.     

Curriculum-Based Measurement progress data: Effects of graph pattern on ease of interpretation

Curriculum-Based Measurement (CBM) is a system for monitoring the progress of and evaluating instructional program effectiveness for students with learning difficulties. Although a large amount of research has been conducted on CBM, little has focused on the interpretation and use of the data for instructional decision-making, despite the fact that it is data use that leads to performance gains. In this study, we examine factors affecting the interpretation of CBM data. Specifically, we examine the effect of CBM graph patterns on ease of graph interpretation. Thirty college/university students completed a two-part study in which they viewed various slope-to-goal and slope-to-slope patterns of CBM-graphed data and answered decision-making questions. Response times and accuracy were measured. Results revealed that graph patterns differed in terms of ease of interpretation. Differences depended on the type of question. Implications for interpretation and use of CBM data for decision-making are discussed.     

我国社会保障基金绩效审计评价指标体系构建

社会保障基金是用于社会保障、福利和社会救济的保障性专项基金,国家监管部门应加强监督和审计。我国社保基金审计工作一直以来以财务审计和财政审计为主,因此审计效果并不充分,也不完善。社保基金绩效审计对于促进社会保险制度的完善,提升社会保险审计的层次和水平,保障公民社会保障权利实现等具有重大而深远的意义。本文通过研究绩效审计的内容,设计绩效审计的定性指标和定量指标体系,提高社保基金使用效率,进一步完善我国社会保障制度。     

Retell as an Indicator of Reading Comprehension

The purpose of this narrative synthesis is to determine the reliability and validity of retell protocols for assessing reading comprehension of students in grades K–12. Fifty-four studies were systematically coded for data related to the administration protocol, scoring procedures, and technical adequacy of the retell component. Retell was moderately correlated with standardized measures of reading comprehension and, with older students, had a lower correlation with decoding and fluency. Literal information was retold more frequently than inferential, and students with learning disabilities or reading difficulties needed more supports to demonstrate adequate recall. Great variability was shown in the prompting procedures, but scoring methods were more consistent across studies. The influences of genre, background knowledge, and organizational features were often specific to particular content, texts, or students. Overall, retell has not yet demonstrated adequacy as a progress monitoring instrument.     

Performance Appraisal: Issues of Validity

Performance ratings are probably the most widely used criteria in personnel evaluation. To determine the psychometric quality of performance ratings, practitioners generally evaluate ratings in terms of reliability and validity. This article presents generalizability theory as a measurement procedure that can be used to assess and improve the reliability and validity of ratings of employee job performance.     

Standards of Validity and the Validity of Standards in Performance Asessment

What are six distinct aspects of construct validation? How do these aspects apply to performance assessment? Are the consequences of performance assessment on teaching and learning relevant to construct validation?     

A Curriculum-Based Passage Reading Test for Monitoring the Performance of Low-Progress Readers: The development of the WARP

Results are presented from two studies involving 190 primary and high school (Year 7) students and 146 older low-progress readers in the late primary years. These studies contribute towards the development of five 200 word standardised passages which may be employed interchangeably to monitor the performance of low-progress readers towards achieving functional literacy. The passages are shown to possess both high parallel form reliability (0.94 to 0.96) and internal consistency (0.97 to 0.99), and also good criterion validity (0.78 to 0.80) with reading accuracy. Appropriate growth in performance over the primary years is also demonstrated. These findings suggest that an extrapolated model of curriculum-based measurement (CBM), which incorporates elements of more traditional reading assessments, would overcome some of the problems associated with CBM.     

理据的寻问:西方社会指标中的教育指标

为什么设置特定的教育指标是一个关系教育指标的合理性与有效性的根本问题,同时,它也是我国当前的教育指标研究中未能充分意识或合理解决的问题。本文将以在全球有较大影响的物质生活质量指数、英格尔斯现代化指标体系、人类发展指数和欧洲社会指标体系为考察对象,分析这些社会指标中之所以提出特定的教育指标的原因与根据,以期在介绍几个典型的教育指标的基础上,反思我们的教育指标比较研究,倡导教育指标研究过程中的理据精神。     

The Social Validity of Bug-in-Ear Coaching: Findings From Two Studies Implemented in Inclusive Early Childhood Environments

Coaching is a promising method for providing professional development, which takes many forms. One such form is real-time coaching through bug-in-ear technology. This study explored the social validity of bug-in-ear coaching when provided as a form of professional development with preservice and in-service early childhood educators. Data from two studies were qualitatively analyzed to describe early childhood educators’ perceptions of the acceptability of bug-in-ear coaching with respect to the learning opportunities provided, feasibility, difficulties, and child-level outcomes. Findings suggest that BIE is deemed to be important and effective at producing educator and child outcomes. Further, educators are satisfied with the intervention and view it to be an acceptable means for receiving professional development.     

教育转型:社会转型的重要内容和必要条件

教育以主体人的培养为中介推动社会系统的发展和进步,但在不同的历史时期,教育发展与社会发展的关系表现出不同的作用机制。在当代以全球化、信息化、知识化为宏观背景的社会转型条件下,教育转型不但是社会转型的重要内容,而且是社会转型必不可少的条件。     

It Matters: Reference Indicator Selection in Measurement Invariance Tests

Conventional approaches for selecting a reference indicator (RI) could lead to misleading results in testing for measurement invariance (MI). Several newer quantitative methods have been available for more rigorous RI selection. However, it is still unknown how well these methods perform in terms of correctly identifying a truly invariant item to be an RI. Thus, Study 1 was designed to address this issue in various conditions using simulated data. As a follow-up, Study 2 further investigated the advantages/disadvantages of using RI-based approaches for MI testing in comparison with non-RI-based approaches. Altogether, the two studies provided a solid examination on how RI matters in MI tests. In addition, a large sample of real-world data was used to empirically compare the uses of the RI selection methods as well as the RI-based and non-RI-based approaches for MI testing. In the end, we offered a discussion on all these methods, followed by suggestions and recommendations for applied researchers.     

欠发达地区司法行政领域社会组织培育机制的探索——以江苏阜宁为例

在当下新一轮社会治理创新实践中,大力培育和发展司法行政领域社会组织,借助社会力量做好法律服务、法制宣传、法律援助、特殊人群管理等,既是推动司法行政事业良性发展的前提条件,也是转变政府职能与提升政府服务效能的重大创举。总结发达地区相关经验,可为当前欠发达地区司法行政领域社会组织培育和发展工作的有效展开提供借鉴。     

The Personal Statement as an Indicator of Writing Skill: A Cautionary Note

The objective of this study was to evaluate the personal statement as an indicator of writing skill. The evaluation was based on a comparison of specially evaluated personal statements with a standardized measure of writing ability requiring examinees to write timed expository essays. A sample of prospective graduate students wrote test essays and provided copies of personal statements they had submitted for application to graduate school. A majority of the sample acknowledged receiving help in either drafting or revising their statements. Correlations of the 2 indicators (test essay and personal statement) with each of several nontest indicators of writing skill (e.g., self-reports of writing ability and grades on writing assignments) revealed that the traditional expository essay was significantly more highly related to nearly all of the indicators. It is suggested that, although the personal statement may provide certain unique and important information about applicants (and thus be valid for this purpose), its validity as an indicator of writing skill (defined as the ability to present and sustain a coherent discussion of a complex issue) needs to be better established.