The Relationship of Teachers' Instructional Decisions and Beliefs about Alternate Assessments to Student Achievement

In the decade since alternate assessments based on alternate achievement standards (AA-AAS) were first required, special education teachers have had to develop content knowledge and pedagogical skills to teach academics to students with significant cognitive disabilities. The purpose of this study is to examine the impact that teachers' instructional decisions and beliefs about AA-AAS have on the academic achievement of students with significant cognitive disabilities. A survey of nearly 400 teachers across three states provided evidence of teachers' beliefs, as well as their instructional planning processes for students. Students' learning characteristics were also identified. Survey responses were analyzed in conjunction with students' 2010–2011 scores on their statewide AA-AAS. The results indicated that students with presymbolic communication systems had a lower likelihood of achieving proficiency in English language arts/reading and math than students with abstract symbolic communication. Students with concrete symbolic communication systems had lesser odds of proficiency in ELA/reading but not in math. Teachers who agreed that AA-AAS had an impact on instruction tended to have students with higher achievement proficiency. However, more negative views of the validity of AA-AAS scores were associated with greater odds of proficiency.     

Alternate Assessments of Students with Significant Disabilities: Alternative Approaches,Common Technical Challenges

This article examines three typical approaches to alternate assessment for students with significant cognitive disabilities—portfolios, performance assessments, and rating scales. A detailed analysis of common and unique design features of these approaches is provided, including features of each approach that influence the psychometric quality of their results. Validity imperatives for alternate assessments are reviewed, and approaches for addressing the need for validity evidence are outlined. The article concludes with an examination of three technical challenges—alignment, scores and scoring, and standard setting—common to all alternate assessments. In light of these challenges, existing methods and professional testing standards are endorsed as necessary guidance for understanding and advancing alternate assessment practices.     

The “Two Percent Students”: Considerations and Consequences of Eligibility Decisions

Since the 2001–02 school year, the accountability provisions of the No Child Left Behind Act (NCLB) have shaped much of the work of public school teachers and administrators in the United States. NCLB explicitly prohibits schools from excluding students with disabilities from the accountability system and requires not only participation of all students in statewide accountability assessments but also reporting of the results for students with disabilities along with other students and as a disaggregated group. From the beginning of these requirements, lawmakers recognized that there would be a small group of students with disabilities for whom the regular assessment, even with accommodations, would not be appropriate and they authorized states to develop an alternate assessment based on alternate achievement standards (AA-AAS) for this group of students. More recently, responding to pressures from the field, additional flexibility has been granted to develop an additional alternate assessment based on modified grade-level achievement standards (AA-MAS) for students with disabilities who present with persistent academic difficulties. It is expected that approximately 2% of the total student population might be included in this new alternate assessment. This article examines the decisions that need to be made by individual states to determine the target population for this new alternate assessment and the policy implications of these decisions.     

An Application of Generalizability Theory to Evaluate the Technical Quality of an Alternate Assessment

Although federal regulations require testing students with severe cognitive disabilities, there is little guidance regarding how technical quality should be established. It is known that challenges exist with documentation of the reliability of scores for alternate assessments. Typical measures of reliability do little in modeling multiple sources of error, which are characteristic of alternate assessments. Instead, Generalizability theory (G-theory) allows researchers to identify sources of error and analyze the relative contribution of each source. This study demonstrates an application of G-theory to examine reliability for an alternate assessment. A G-study with the facets rater type, assessment attempts, and tasks was examined to determine the relative contribution of each to observed score variance. Results were used to determine the reliability of scores. The assessment design was modified to examine how changes might impact reliability. As a final step, designs that were deemed satisfactory were evaluated regarding the feasibility of adapting them into a statewide standardized assessment and accountability program.     

A Validity Framework for Evaluating the Technical Quality of Alternate Assessments

This article presents findings from two projects designed to improve evaluations of technical quality of alternate assessments for students with the most significant cognitive disabilities. We argue that assessment technical documents should allow for the evaluation of the construct validity of the alternate assessments following the traditions of Cronbach (1971) , Messick (1989, 1995) , Linn, Baker, and Dunbar (1991) , and Shepard (1993) . The projects used the work of Knowing What Students Know ( Pellegrino, Chudowsky, & Glaser, 2001 ) to structure and focus the collection and evaluation of assessment information. The heuristic of the assessment triangle ( Pellegrino et al., 2001 ) was particularly useful in emphasizing that the validity evaluation needs to consider the logical connections among the characteristics of the students tested and how they develop domain proficiency (the cognition vertex), the nature of the assessment (the observation vertex), and the ways in which the assessment results are interpreted (the interpretation vertex). This project has shown that in addition to designing more valid assessments, the growing body of knowledge about the psychology of achievement testing can be useful for structuring evaluations of technical quality.     

Analysis of assessment practice and subsequent performance of third year level students in natural sciences

Abstract

Summative assessment qualifies the achievement of a student in a particular field of specialization at a given time. Questions should include a range of cognitive levels from Bloom's taxonomy and be consistent with the learning outcomes of the module in question. Furthermore, a holistic approach to assessment, such as the application of the principles of the Herrmann Whole Brain Model, needs to be used to accommodate learning style diversity. The purpose of this study was to analyse, assess and compare the summative assessment of two third year level modules in the Bachelor of Science degree programme, namely Biochemistry and Zoology as part of action research with a view to enhancing the professional development of the lecturers involved. The questions posed in summative assessments were classified in terms of Bloom's differentiation of cognitive levels and the four different learning styles determined by Herrmann. Spearman's non-parametric analysis indicated that no correlation existed in this study between cognitive level and student performance based on achievement. In addition, there was not much difference between the cognitive levels and student performance between the two disciplines. Although the students seemed to do better at application level questions, the authors need to reflect on whether the assessments were valid with respect to the learning outcomes, methods of facilitating learning, and the assessments based on cognitive levels and learning style preferences. We conclude that continuous action research must be taken to improve the formulation of learning outcomes and students' achievement of these outcomes and quality of student learning – the main aim being the successful completion of the modules.     

UNDERSTANDING TEST‐TYPE ASSIGNMENT: WHY DO SPECIAL EDUCATORS MAKE UNEXPECTED TEST‐TYPE ASSIGNMENTS?

We interviewed special educators (a) whose students with disabilities (SWDs) were proficient on the 2008 general education assessment but were assigned to the 2009 alternate assessment based on modified achievement standards (AA‐MAS), and (b) whose students with mild disabilities took the 2008 alternate assessment based on alternate achievement standards (AA‐AAS) and then the 2009 AA‐MAS. We explored teachers’ rationales for test‐type assignment, student characteristics, and quality of instruction to determine the test‐type decisions’ appropriateness. All teachers based their decisions on combinations of factors in the guidelines plus subjective and noninstructional factors. Findings raised concerns about the subjectivity of the assessment assignment system and the inappropriate grade‐level instruction for SWDs. Future research, implications of these findings, and limitations are discussed.     

Exploring the Influence of Judge Proficiency on Standard‐Setting Judgments

Setting performance standards is a judgmental process involving human opinions and values as well as technical and empirical considerations. Although all cut score decisions are by nature somewhat arbitrary, they should not be capricious. Judges selected for standard‐setting panels should have the proper qualifications to make the judgments asked of them; however, even qualified judges vary in expertise and in some cases, such as highly specialized areas or when members of the public are involved, it may be difficult to ensure that each member of a standard‐setting panel has the requisite expertise to make qualified judgments. Given the subjective nature of these types of judgments, and that a large part of the validity argument for an exam lies in the robustness of its passing standard, an examination of the influence of judge proficiency on the judgments is warranted. This study explores the use of the many‐facet Rasch model as a method for adjusting modified Angoff standard‐setting ratings based on judges’ proficiency levels. The results suggest differences in the severity and quality of standard‐setting judgments across levels of judge proficiency, such that judges who answered easy items incorrectly tended to perceive them as easier, but those who answered correctly tended to provide ratings within normal stochastic limits.     

国际化背景下跨国修读学分互认机制及管理对策

在教育国际化与信息化融合发展的背景下,推行国内外高校间的学分互认已成为全球教育改革的重要方向。跨国修读学分互认是基于学分制的一种制度安排,是实现高等教育多元化、开放式发展的重要载体。中国高校跨国修读学分互认起步晚,发展不均衡;跨国学分互认尚缺乏相关法律及配套政策支持;学分设置及质量考核不规范,缺乏统一标准。文章借鉴国外跨国修读学分互认的典型模式及其特点,基于"以人为本"理念对中国高校跨国修读学分互认机制及实施路径进行制度设计,并从政府和高校等层面提出相应的管理对策:制定统一的学分设置标准,构建校际学分互认平台;推行融合互联网的"学分银行"制度;加强师资队伍建设,提升教务管理国际化水平。     

Condensed Mastery Profile Method for Setting Standards for Diagnostic Assessment Systems

The purpose of this study was to develop a standard‐setting method appropriate for use with a diagnostic assessment that produces profiles of student mastery rather than a single raw or scale score value. The condensed mastery profile method draws from established holistic standard‐setting methods to use rounds of range finding and pinpointing to specify cut points between performance levels. Panelists are convened to review profiles of mastery and specify cut points between performance levels based on the total number of skills mastered. Following panelist specification of cut points, a statistical method is implemented to smooth cut points over grades to decrease between‐grade variability. Procedural evidence, including convergence plots, standard errors of pinpointing ratings, and panelist feedback, suggest the condensed mastery profile method is a useful and technically sound approach for setting performance standards for diagnostic assessment systems.     

Articulating the validity evidence for a science alternate assessment

Students with the most significant cognitive disabilities (SCD) are the 1% of the total student population who have a disability or multiple disabilities that significantly impact intellectual functioning and adaptive behaviors and who require individualized instruction and substantial supports. Historically, these students have received little instruction in science and the science assessments they have participated in have not included age‐appropriate science content. Guided by a theory of action for a new assessment system, an eight‐state consortium developed multidimensional alternate content standards and alternate assessments in science for students in three grade bands (3–5, 6–8, 9–12) that are linked to the Next Generation Science Standards (NGSS Lead States, 2013 ) and A Framework for K‐12 Science Education (Framework; National Research Council, 2012 ). The great variability within the population of students with SCD necessitates variability in the assessment content, which creates inherent challenges in establishing technical quality. To address this issue, a primary feature of this assessment system is the use of hypothetical cognitive models to provide a structure for variability in assessed content. System features and subsequent validity studies were guided by a theory of action that explains how the proposed claims about score interpretation and use depend on specific assumptions about the assessment, as well as precursors to the assessment. This paper describes evidence for the main claim that test scores represent what students know and can do. We present validity evidence for the assumptions about the assessment and its precursors, related to this main claim. The assessment was administered to over 21,000 students in eight states in 2015–2016. We present selected evidence from system components, procedural evidence, and validity studies. We evaluate the validity argument and demonstrate how it supports the claim about score interpretation and use.     

SOLO在显微镜技能教学中的应用

基于新课程改革评价既要强调关注学生学习结果,也要关注其学习过程,特别是对学生思维过程的评价,文中介绍了一种在国外已有深入研究与实践、可运用于学生思维评价的质性评价方法——SOLO分类评价法在显微镜技能教学与管理中的应用。它能帮助教师评价、照顾学生异质性、改进教学方法与技能评价等。     

美国严重认知障碍儿童的选择性评价

在美国,不能参加统一绩效测验的严重认知障碍儿童需要接受选择性评价。本文主要介绍了美国严重认知障碍儿童选择性评价的法律依据、评价标准、评价形式等基本情况。文章还讨论了选择性评价的积极影响及面临的问题,展望了今后的研究趋势。     

以SOLO分类为基础的学生学习质量评价初探

以SOLO分类为基础的学生学习质量评价，将教育目标表述为对学生学习能力的要求，以测验的方式了解学生解决问题时表现出的认知发展水平。它不局限于考察学生掌握知识的量和类型，而是重视评价学生学习效果的质量。SOLO分类用结构特征解释学生对问题的五种结构反应，通过学生的反应与教师预设的教学目标的比较，分析影响学生学习效果的因素，为教师改进教学提供依据，帮助不同认知水平的学生在他们自身的基础上提高。     

新课程下高考地理考试评价标准及试卷结构技术指标构想

新课程高考地理考试标准及试卷结构技术指标构建的依据是《普通高中课程方案(实验)》和《普通高中地理课程标准(实验)》。地理科试卷结构技术指标包括试卷的结构模式、内容要素、目标要素、题型要素、难度要素、分数要素、时限要素、长度要素和等值要素,它是新课程高考地理命题、审题评估监控的标准,是实现试卷及试题质量控制的依据,可以为高考地理试卷及试题质量评价提供系统的可供操作的标准体系和方法手段。     

Reliability and Validity of Play‐based Observations: relationship between the PLAY behaviour observation system and standardised measures of cognitive and social skills

Little, if any, examination of the reliability and validity of play‐based assessment exists in the current literature. Thus, the present study specifically examined the relationship between scores obtained on a classroom‐based play observation system and standardised measures of cognitive and social competence with preschool children. High reliability between observers was established for the play‐based assessment measure. In addition, concurrent validity was assessed by comparing scores on the PLA Y assessment measure with scores the children obtained on the Battelle Developmental Inventory (BDI) and Social Skills Rating Scale (SSRS‐T). Significant positive correlations in the predicted directions were obtained: Children rated as more cognitively and socially competent were found to engage in higher levels of play behaviours (e.g. associative‐constructive and co‐operative‐dramatic), while children identified as less cognitively and socially competent engaged in lower levels of play behaviours (e.g. solitary‐functional and onlooker). The study suggests that, when operationally defined play observation methods were used, observers were able to record accurately the level of play exhibited by each child, and these play behaviours did reflect the child's current cognitive and social developmental functioning.     

Science assessments for all: Integrating science simulations into balanced state science assessment systems

This article reports on the collaboration of six states to study how simulation‐based science assessments can become transformative components of multi‐level, balanced state science assessment systems. The project studied the psychometric quality, feasibility, and utility of simulation‐based science assessments designed to serve formative purposes during a unit and to provide summative evidence of end‐of‐unit proficiencies. The frameworks of evidence‐centered assessment design and model‐based learning shaped the specifications for the assessments. The simulations provided the three most common forms of accommodations in state testing programs: audio recording of text, screen magnification, and support for extended time. The SimScientists program at WestEd developed simulation‐based, curriculum‐embedded, and unit benchmark assessments for two middle school topics, Ecosystems and Force & Motion. These were field‐tested in three states. Data included student characteristics, responses to the assessments, cognitive labs, classroom observations, and teacher surveys and interviews. UCLA CRESST conducted an evaluation of the implementation. Feasibility and utility were examined in classroom observations, teacher surveys and interviews, and by the six‐state Design Panel. Technical quality data included AAAS reviews of the items' alignment with standards and quality of the science, cognitive labs, and assessment data. Student data were analyzed using multidimensional Item Response Theory (IRT) methods. IRT analyses demonstrated the high psychometric quality (reliability and validity) of the assessments and their discrimination between content knowledge and inquiry practices. Students performed better on the interactive, simulation‐based assessments than on the static, conventional items in the posttest. Importantly, gaps between performance of the general population and English language learners and students with disabilities were considerably smaller on the simulation‐based assessments than on the posttests. The Design Panel participated in development of two models for integrating science simulations into a balanced state science assessment system. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 363–393, 2012     

高校非法学专业法学教育目标之重塑

高校非法学专业法学教育目标的认知型定位以及独立性缺失,是大学生法制教育实效低下的重要原因。在素质教育的理念下,还高校非法学专业的法学教育之独立性地位,重塑认知和践行相结合、注重法律信仰养成的素质教育目标,是社会对高校人才培养质量标准的必然要求。     

学分制条件下高校教学质量评价和监控机制探微

学分制相对学年制而言，有其诸多的优越性，如自主选课给学生提供有利于自主学习和个性发展的自由空间，学习时限的灵活性、培养过程的指导性和培养计划的开放性等。但也正是由于此，它给高校教学质量的评价和监控带来了一些难题。所以必须加强对学分制条件下教学质量的评价与监控，如制定科学的教学计划、推行导师制、实行选课制，实行教考分离、严格考试管理，建立适合学分制条件下的教学质量评价标准、评价方式和评价激励机制。