The Multiple‐Use of Accountability Assessments: Implications for the Process of Validation

Implications of the multiple‐use of accountability assessments for the process of validation are examined. Multiple‐use refers to the simultaneous use of results from a single administration of an assessment for its intended use and for one or more additional uses. A theoretical discussion of the issues for validation which emerge from multiple‐use is provided focusing on the increased stakes that result from multiple‐use and the need to consider the interactions that may take place between multiple‐uses. To further explore this practice, an empirical study of the multiple‐use of the Education Quality and Accountability Office Grade 9 Assessment of Mathematics, a mandatory assessment administered in Ontario, Canada, is presented. Drawing on data gathered in an in‐depth case study, practices associated with two of the multiple‐uses of this assessment are considered and evidence of ways these two uses interact is presented. Given these interactions, the limitations of an argument‐based approach to validation for this instance of multiple‐use are demonstrated. Some ways that the process of validation might better address the practice of multiple‐use are suggested and areas for further investigation of this frequently occurring practice are discussed.     

Computer-Based Accountability: Implications for Training

Since the call for professional accountability in the early 1970s, counselors and counselor educators have attempted to assess counseling effectiveness, to upgrade programs as a result of assessments, and to communicate results of program assessment and change to their publics. Until recently, their efforts have been impeded by the interpersonal nature of counseling, insufficient training in research and evaluation, and available low-cost computer technology. This article addresses each of the impediments with the focus on how present computer technology, specifically computer-assisted systems, can be used to help counselors and counselor educators ethically input, store, and retrieve pertinent client and program data—data that can be used to achieve program evaluation and lead to accountable program development. The Counselor Accountability System is presented as an example of an easy-to-use computer program designed to facilitate counselor accountability.     

Stakeholders in Comprehensive Validation of Standards-Based Assessments: A Commentary

Linda Cracker is a Professor of Educational Psychology, University ofFlorida, P.O. Box 11 7047 Gainesville, FL 32611-7043. Her areas of specialization are assessment development, validation, and test-taking behavior.     

Searching for the Norm in a System of Absolutes: A Case Study of Standards-Based Accountability Reform in Pre-Kindergarten

Research Findings: The emergence of standards-based accountability reforms in early childhood education has created new challenges for the field. This article presents findings from a case study that explored how stakeholders in a large urban pre-kindergarten program struggled to implement an assessment tool that aligned the normative academic achievement expectations found among their teachers and administrators with the absolute measures of this construct found in their state policymakers' high-stakes standards-based accountability reforms. Analyzing the tension that emerged in this process of alignment highlights the challenges early educators face as they fold their child-centered programs into these larger high-stakes standards-based kindergarten through grade 12 education systems. Practice and Policy: The findings from this study illuminate the need for early childhood education programs to understand how high-stakes standards-based accountability reforms define student achievement. Furthermore, as early childhood programs and personnel address these reforms, their responses need to be explicit about how their assessment measures are connected to their normative conceptualizations of student achievement and what this means for the education of children in their programs.     

Integrating Accountability With Choice: Implications for School Governance

As school choice policies continue to gain momentum in the American system of public education, a push for increased accountability by different levels of government has occurred. In some instances, policymakers have begun to pair public-school choice and performance-based accountability mechanisms within the same initiative (e.g., the Florida A+ Plan). Theoretically, whereas school choice is supposed to release schools and their potential for innovation and improvement from bureaucratic control, prevailing forms of educational accountability have arisen from politically negotiated, bureaucratically administered standards for school success. The purpose of this article is to explore the consequences of integrated choice/accountability policies for the organization and governance of schooling on purely conceptual grounds. In so doing, it exposes what may be a fundamental tension.     

Evaluating Growth for ELL Students: Implications for Accountability Policies

In recent years, many U.S. states have introduced growth models as part of their educational accountability systems. Although the validity of growth‐based accountability models has been evaluated for the general population, the impact of those models for English language learner (ELL) students, a growing segment of the student population, has not received sufficient attention. We evaluated three commonly used growth models: value tables or transition matrices, projection models, and student growth percentiles (SGP). The value table model identified more ELL students as on track to proficiency, but with lower accuracy for ELL students. The projection and SGP models were more accurate overall, but classified the fewest ELL students as on track and were less likely to identify ELL students who would later be proficient. We found that each model had significant trade‐offs in terms of the decisions made for ELL students. These findings should be replicated in additional state contexts and considered in the development of future growth‐based accountability policies.     

School Organizational Effectiveness and Chronic Absenteeism: Implications for Accountability

ABSTRACT

Chronic absenteeism in K-12 schools is strongly associated with critical educational outcomes such as student achievement and graduation. Yet, the causes of chronic absenteeism are complex, with environmental, family/individual, and school factors all affecting the likelihood of a student attending school regularly. This exploratory study examines whether school organizational effectiveness has the potential to moderate external influences on chronic absenteeism. Using school-level scores from the 5Essentials surveys, we find that, in traditional public schools, schools that are organized for effectiveness have lower rates of chronic absenteeism, while controlling for student demographics and grade level. In particular, schools with higher scores for “involved families” have lower chronic absenteeism. While charter schools in Detroit have significantly lower rates of chronic absenteeism than traditional public schools, we did not find an association between organizational effectiveness and chronic absenteeism in charter schools. This suggests that student sorting by school type may produce variation in chronic absenteeism rates that is not moderated by school actions. These findings have important implications for practice and policy, as educators seek to reduce chronic absenteeism in response to pressures from high-stakes accountability systems.     

问责与改进：我国第四轮学科评估元评估

实施有效的元评估是应答社会问责和改进评估实践的国际经验及现实选择。作者基于元评估模型对第四轮学科评估进行优劣分析,发现其服务多元利益相关者的目标适切,指标与学科质量相关性增强,评估方案可行性强,成本效益高。不足之处在于缺乏元评估环节和评估结果使用说明。基于此,为第四轮学科评估元评估提出建议：①教育部学位与研究生教育发展中心自主组织终结性元评估;②参照元评估策略模型实施元评估;③提供元评估报告说明评估有效性;④多元主体参与实现评估改进功能。     

Implications of NCLB Accountability for Comprehensive School Reform

No Child Left Behind (NCLB) accountability mechanisms have the potential to derail comprehensive school reform (CSR) implementation. For those pursuing CSR, the question is how to reconcile the implementation of NCLB accountability mandates with ongoing CSR efforts. Drawing from longitudinal data from a national study of CSR, this article explores this question. Survey data indicate that schools identified for improvement are no more likely to drop their models than are other CSR schools. Moreover, it appears that when CSR schools are identified for improvement, this identification stimulates a higher rate of model implementation than in schools that are not identified under NCLB. Case study interview data revealed that a district variation in stakeholders' awareness and perceptions of NCLB and the interactions of NCLB accountability mechanisms and CSR sustainability.     

Strategies for Examining the Consequences of Assessment and Accountability Programs

This article addresses issues in evaluating the consequences of assessment programs that are developed for the purpose of holding schools accountable to state standards. After providing a brief review of research examining consequential evidence, a validation study to obtain consequential evidence for state assessment and accountability programs is proposed. The proposal includes a validity argument, a set of propositions that follow from the validity argument, a delineation of the consequential evidence needed, and a way to model the relationship between performance gains and school, principal, teacher, and student variables.     

学术本位:现代大学制度的基本意涵

大学制度改革必须立足于大学的学术组织属性,符合学术本位的大学逻辑;现代大学制度即基于学术本位的大学制度体系;从普遍所指来看,现代大学制度是以大学内在的学术价值为本、以满足其外在的社会价值为辅的制度体系;从具体涵括而言,现代大学制度包含了普遍与本土元素、正式与非正式约束要素,而这俱以学术本位为基本原则.     

Perceptions of a Military Social Work Specialization Certificate Program: Implications for Future Programs

Military social work certificates, courses, and specializations have emerged to address the need for competent and culturally aware practitioners to serve military and veteran clients. This study evaluates one of these certificate programs surveying 27 previous graduate students and 11 field supervisors affiliated with this program. The majority of the participants endorsed this certificate as a viable means to prepare for practice with these clients. Findings highlight the importance of participants understanding military culture and common psychiatric conditions. Results point to the benefits of future programs teaching this material using real-life case scenarios and addressing veteran benefits, different periods of service, macro content, and complex ethical challenges.     

A Conceptual Framework for a Psychometric Theory for Standard Setting with Examples of Its Use for Evaluating the Functioning of Two Standard Setting Methods

A conceptual framework is proposed for a psychometric theory of standard setting. The framework suggests that participants in a standard setting process (panelists) develop an internal, intended standard as a result of training and the participant's background. The goal of a standard setting process is to convert panelists' intended standards to points on a test's score scale. Psychometrics is involved in this process because the points on the score scale are estimated from ratings provided by participants. The conceptual framework is used to derive three criteria for evaluating standard setting processes. The use of these criteria is demonstrated by applying them to variations of bookmark and modified Angoff standard setting methods.     

A Realistic Evaluation of Two Training Programs on Implementing Skin-to-Skin as a Standard of Care

The authors used realistic evaluation to examine the real-world effectiveness of two 5-day training techniques on sustained optimal skin-to-skin practices that support Step 4 of the revised Baby-Friendly Hospital Initiative (BFHI). The authors found that education alone was insufficient to effect sustainable practice change. Exposure to the 5-day immersion model (Practice, Reflection, Education and training, Combined with Ethnography for Sustainable Success, or PRECESS) alone or combined with education was an effective strategy to change and sustain the standard of care for skin-to-skin practice (p < 0.00001). The intended outcome of sustained practice change toward implementation of skin-to-skin care through immersion or a combined approach shows promise and should be repeated in other localities.     

Diagnostic Profiles: A Standard Setting Method for Use With a Cognitive Diagnostic Model

This article introduces the Diagnostic Profiles (DP) standard setting method for setting a performance standard on a test developed from a cognitive diagnostic model (CDM), the outcome of which is a profile of mastered and not‐mastered skills or attributes rather than a single test score. In the DP method, the key judgment task for panelists is a decision on whether or not individual cognitive skill profiles meet the performance standard. A randomized experiment was carried out in which secondary mathematics teachers were randomly assigned to either the DP method or the modified Angoff method. The standard setting methods were applied to a test of student readiness to enter high school algebra (Algebra I). While the DP profile judgments were perceived to be more difficult than the Angoff item judgments, there was a high degree of agreement among the panelists for most of the profiles. In order to compare the methods, cut scores were generated from the DP method. The results of the DP group were comparable to the Angoff group, with less cut score variability in the DP group. The DP method shows promise for testing situations in which diagnostic information is needed about examinees and where that information needs to be linked to a performance standard.     

英国高等教育学科专业设置及其启示

从英国高等教育学科专业设置的发展及其特点入手,分析了影响英国学科专业设置的文化、经济、科技及高校自身发展等因素以及英国学科专业设置的发展趋势;在此基础上,讨论了英国学科专业设置的经验对我国的启示,认为我国研究生教育学科专业设置应增强开放性,扩大高校自主权,提高学科专业布局的合理性并逐步发展独立中介评估机构。     

对会计准则制定主体模式的思考

介绍了会计准则的制定主体模式,对政府主体模式和民间主体模式进行评价。对制定主体模式进行经济学分析,在承认目前我国政府主体模式合理性的同时,提出了其发展趋势是：应加强民间专业团体的参与。     

Accountability: A Search for a Solution

The work students create should help them develop the knowledge, skills, and attitudes needed in the adult world. In this article, the authors identify adult roles essential for young people to develop and outline ways teachers can use curriculum standards to create assignments and assessments that prepare youths for those adult roles in society. They also provide guidelines for scoring these assessments and a sample rubric.     

Understanding Youth Obesity and Media Use: Implications for Future Intervention Programs

Nearly six years have passed since the Physical Activity and Health: A Report of the Surgeon General (USDHHS, 1996) described the health and fitnesa status of Amrica's yarth. The Sugeon Guxd's Call to Action to prevent and Decrease Overweight and Obesity (USDHHS, 2001) provides a camprehensive review of research on the epidemiology, health risks, and prevalence of obesity in all segments of the poplation. Increasinngly, children and adolescents are turning to sedentary kisure-time activities. Media use has been linked to decread physical activity levels, incearsed BMI, and increased high-enugy food intake. This review of litnature pravides a comprehensive examination of the influence media use has on youth adiposity, dietary habits. and physical activity. Current and future intervention programs to decrease sedentary behavior in children are also highlighted. Reducing media use (including television, videaape viewing, nonacademic computer use, and video game playing) in youth cau address leisure-time sedentary behaviors. Future intervention program, using run interdisciplinary appach, need to decrease leisure-time sedentary behaviors while promoting active lifestyles.