Ten Years After the Spellings Commission: From Accountability to Internal Improvement

Student learning outcomes assessment has been increasingly used in U.S. higher education institutions over the last 10 years, partly fueled by the recommendation from the Spellings Commission that institutions need to demonstrate more direct evidence of student learning. To respond to the Commission's call, various accountability initiatives have been launched, profoundly reshaping how assessment has been viewed, implemented, and used in higher education. This article reviews the conceptual and methodological challenges of the assessment agenda for one of the landmark accountability initiatives, the Voluntary System of Accountability, and also documents the notable shift from a strong focus on accountability to an increasing emphasis on internal improvement. This article then discusses the most recent developments in assessment approaches and tools, and proposes a four‐element, one‐enabler assessment cycle for institutions to maximally benefit from their assessment efforts.     

“自愿问责制”:美国高等教育问责制发展的新动向

近年来,"自愿问责制"(VSA)的兴起为美国公立学院和大学改革注入了新的活力,反映了美国公立院校领导层应对新挑战的基本思考。"大学肖像"模板是"自愿问责制"的主要实现形式,包含学生与家庭信息、学生经历与体会以及学生学习结果三个截面的内容,体现了整合性与集约化并举、规范化与弹性结合、创造性与渐进性兼顾等特点。虽然"自愿问责制"的发展伴随着质疑且存在一些有待解决的难题,但其发展对美国高等教育产生了重要影响。     

Value-added assessment in higher education: a comparison of two methods

Evaluation of the effectiveness of higher education has received unprecedented attention from stakeholders at many levels. The Voluntary System of Accountability (VSA) is one of the initiatives to evaluate institutional core educational outcomes (e.g., critical thinking, written communication) using standardized tests. As promising as the VSA method is for calculating a valueadded score and allowing results to be comparable across institutions, it has a few potential methodological limitations. This study proposed an alternative way of value-added computation which takes advantage of multilevel models and considers important institution-level variables. The institutional value-added ranking was significantly different for some of the institutions (i.e., from being ranked at the bottom to performing better than 50% of the institutions) between these two methods, which may lead to substantially different consequences for those institutions, should the results be considered for accountability purposes.     

The Effects of Motivational Instruction on College Students' Performance on Low-Stakes Assessment

Assessments of student learning outcomes (SLO) have been widely used in higher education for accreditation, accountability, and strategic planning purposes. Although important to institutions, the assessment results typically bear no consequence for individual students. It is important to clarify the relationship between motivation and test performance and identify practical strategies to boost students' motivation in test taking. This study designed an experiment to examine the effectiveness of a motivational instruction. The instruction increased examinees' self-reported test-taking motivation by .89 standard deviations (SDs) and test scores by .63 SDs. Students receiving the instruction spent an average of 14 more seconds on an item than students in the control group. Score difference between experimental and control groups narrowed to .23 SDs after unmotivated students identified by low response time were removed from the analyses. The findings provide important implications for higher education institutions which administer SLO assessments in a low-stakes setting.     

Measuring learning outcomes

The growing interest for measurement of learning outcomes relates to long lines of development in higher education, the request for accountability, intensified through international reforms and movements such as the development and implementation of qualifications frameworks. In this article, we discuss relevant literature on different approaches to measurement and how learning outcomes are measured, what kinds of learning outcomes are measured, and why learning outcomes are measured. Three dimensions are used to structure the literature: Whether the approaches emphasise generic or disciplinary skills and competence, self‐assessment or more objective test based measures (including grades), and how the issue of the contribution from the education program or institution (the value‐added) are discussed. It is pointed out that large scales initiatives that compare institutions and even nations seem to fall short because of the implicit and explicit differences in context, whilst small‐scale approaches suffer from a lack of relevance outside local contexts. In addition, competence (actual level of performance) is often confused with learning (gain and development) in many approaches, laying the ground for false assumptions about institutional process‐quality in higher education.     

Transfer Student Graduation Efficiency and University Administrators: New Bedfellows

Accountability is an important focus for nearly all-public higher education institutions. In 1997 the Washington State Legislature mandated an accountability measure designed to encourage public universities to increase student efficiency toward graduation. This accountability measure is assessed by a formula called the Graduation Efficiency Index. This qualitative study details the Graduation Efficiency Index's conception and ramifications for public higher education institutions. It further examines university administrators' perceptions of the Graduation Efficiency Index and transfer students' impact on the ability of three public institutions to meet mandated accountability goals. We include implications for practice. Susan Poch is the Director of the Student Advising and Learning Center at Washington State University. She holds a Ph.D. in Education Leadership, Higher Education from Washington StateUniversity. Her research focuses on the policy and practice of student transition. Mimi Wolverton is Professor of Higher Education at the University of Nevada, Las Vegas. Her areas of research include leadership, organizational change and organizational effectiveness. She holds an M.B.A. and Ph.D. in Education Leadership from Arizona State University.     

Managing Change—Engaging Faculty in Assessment Opportunities

Regional accrediting bodies require evidence that higher education institutions are meeting their stated goals. Institutions have answered this call for accountability by assessing student learning. Managing change in order to implement assessment practices is a challenge, however, particularly when autonomy, academic freedom, and shared governance are involved. Leadership theories offer practical strategies for administrators instituting assessment-related change. Using these theories as a guide, this article provides suggestions for leading assessment initiatives, garnering faculty support, and establishing a culture of assessment. The suggestions are organized around a four-frame model based on the premise that leaders must consider multiple perspectives to be successful.     

全球化背景下的高等教育责任制

高等教育责任制是一个与效率、效果和绩效评估相关的概念,它要求通过有效的方式证明高等教育取得了预定的结果和绩效。关注效率、效果,注重结果、产出是高等教育责任制的基本特征。高等教育评估是责任制的实施形式,也是质量保障的一个基本途径。通过评估加强质量保障,反映了全球化背景下高等教育管理中一种新的责任制文化。高等教育过程的内在复杂性决定了责任制及绩效测量的难度,我国高等教育评估要实现保障和提高高等教育质量的目的,同时避免对高校办学自主权造成不利影响,需要总结已有评估经验并借鉴国外有益经验,进行创造性的探索。     

美国公立高校绩效评价体系内容与特点分析——以田纳西州为例

美国公立高校绩效评价是美国州高等教育绩效问责制度中的重要环节。本文对田纳西州2010年~2015年绩效资助政策中的绩效评价体系进行分析,认为其具有以下特点:是一种自我评价基础之上的第三方评价;强调计划性和周期性,目标明确,重点突出;聚焦于教育质量保障,尤其关注本科生的学习成效、学科专业项目的质量和学生的满意度;利用美国国内现有的、影响较大的各类测评机构的测试结果或调查结果作为评价依据;注重对绩效评价本身的质量保障。     

Contesting educational assessment policies in Australia

ABSTRACT

Assessment is a major component of education, significant in directing what is identified as valued student learning. This paper is framed within an understanding of imperative and exhortative policy. Two paradigmatically different, and potentially contesting, assessment policy directions in Australian education – educational accountability to monitor school and teacher performance, and teacher assessment practices to improve learning (assessment for learning [AfL] or formative assessment) – are examined for their impact on teacher professionalism. Both approaches have official endorsement in Australian policy. Mandated participation in national tests is indicative of educational accountability assessments under national direction. While also endorsed nationally, AfL implementation is reliant on state and territory direction. Our examination reveals tensions in the alignment of both policies. This is evident in the impact of accountability assessment on AfL implementation, in particular, teachers’ understandings of valued assessment evidence. We conclude that a paradigmatic shift to support student learning in Australian schools is a policy imperative that includes the need for professional development and learning support for teachers.     

PURSUING EXCELLENCE: THE COMMUNITY COLLEGE OF DENVER

This article is based on an in-depth, yearlong qualitative case study focusing on the Community College of Denver and a comprehensive review of research examining developmental education and institutional effectiveness issues in higher education. The authors identify critical issues, discuss implications, and suggest recommendations for improving accountability of practices; (b) increasing student success, with an emphasis on valuing diversity; and (c) implementing a new developmental program of strategically improving an existing one. Recommendations include (a) being knowledgeable about accountability mandates required of others and initiating self-imposed action; (b) establishing a commitment to quality results and sharing results with internal and external constituents; (c) creating an internal grassroots process that regularly provides for open discussion, planning, promoting and supporting student performance, and celebrating success; (d) cultivating informal leadership; (e) recruiting and hiring the best faculty members available then monitoring, mentoring, and training them; (f) creating a centralized developmental education model; (g) integrating technology into all college services and instructional plans; (h) seeking external funding to support retention and achievement initiatives; and (i) being committed to and taking steps toward leveling the educational playing field. Study results suggest that, when implemented as a combined effort by an institution committed to student learning, these actions can contribute to significantly higher levels of student success.     

Public Administration Education in Britain

Abstract

Students, quality assessment bodies established by the Government to determine output‐related funding, and other groups in society are placing more and more pressure on tertiary institutions to improve the quality of their teaching practices. Some institutions are responding to this pressure by implementing student‐centred approaches to teaching, where the students and course leaders become equal partners in determining the course work assessment process, teaching methods and learning outcomes. To a certain extent the learning outcomes are set by the Government's quality assessment bodies. However the course work assessment process, teaching methods and interpretation of these pre‐specified learning outcomes, can still be negotiated with the students in a student‐centred manner.

Many disciplines in tertiary institutions currently face, or can expect to face in the future, extremely large class sizes. Many argue that it is not possible to implement a student‐centred approach in the large class setting. Indeed, they suggest that there is a negative relationship between class size and the quality of the learning experience.

This paper examines the application of a student‐centred approach to a final year business studies course involving over 100 students completing their BSc (Hons). A survey of the students at the end of the year indicated that, relative to the other courses being taken by the students, this course achieved an unusual treble: higher levels of student learning, higher student workload, and yet, higher levels of student enjoyment. These results would tend to dispel the myth that student‐centred approaches can only be successful in the smaller class setting.     

Educational assessment and accountability: a New Zealand case study

This paper discusses the increasing use of assessment as a market signal and as an index of educational accountability. It is argued that assessment policies in New Zealand reflect an uneasy balance between the interests of the new right and more progressive educationists. These influences are examined using three largely contradictory models of educational accountability (professional, market and management). Each model reflects a range of epistemological and ideological assumptions. Thus student assessment serves different and largely conflicting purposes. The paper uses a recent New Zealand policy document (Tomorrow's Standards) to examine the interaction of each model. It is argued that through a failure to state clearly the purpose of assessment, educational reform in this area is overly concerned with the means rather than the ends of education. This has important implications for student motivation and learning. The paper concludes with a comment on current policy development and concludes that some recent initiatives provide the hope that a system of assessment that is both meaningful and relevant to individual learners may be developed.     

Articulating and Utilizing Communication Student Learning Outcomes in Community Colleges

Academia in the United States stands face-to-face with the constant call for greater levels of assessment and accountability. Academics question whether our students are learning what they’re taught? Are they able to secure employment upon completing their education? Are they prepared to be active, productive citizens in society? Community colleges provide education to those who may typically not have the opportunity to attend an institution of higher education. A foundation of any assessment process is the articulation of the core learning outcomes. It is from this foundation, then, that community college faculty can start assessing the effectiveness of teaching and curricular design. The present investigation details a project that focused on creating discipline-specific student learning outcomes and how these outcomes can be utilized to aligning learning outcomes and mapping to curriculum. Communication studies faculty can utilize the learning outcomes to strengthen and clarify curriculum, demonstrate the acquisition of competencies, and improve departmental offerings.     

透视美国高等教育中的学生学习结课评价

学生学习结果评价是美国高等教育评价由关注资源输入到全面注重受教育者学习结果的一次转变。它侧重评价学生学习后所获得的各方面知识、能力和品质。该评价成为一场运动的主要原因是培养高技能劳动力的需求、政府问责的加强以及认证组织的推动。学生学习结果评价的实施在政府、高校以及专业、课堂等众多层面展开,并表现出以学生为中心的价值取向、评价参与者广泛以及评价方法灵活多样等特点。     

Coming to terms with the accountability versus improvement debate in assessment

Concerns about quality and accountability in higher education have led the majority of states to mandate student outcomes assessment. While many institutions have begun to address these requirements as matters of accountability, discussion in most state capitals seems to be more about program improvement than about summative evaluation. These dichotomous perspectives frame a fundamental debate over the nature and direction of the assessment movement in the United States. Accountability and improvement are twin concerns of effective assessment, but their coexistence depends upon the resolution of suspicions and different interpretations of the mission of higher education between state and institutional authorities.     

Neoliberal Ideologies,Governmentality and the Academy: An examination of accountability through assessment and transparency

Colleges and universities exist within a political arena where external demands for accountability materialize within a market-driven environment. As a result, government agencies pressure colleges and universities to rely on assessment and transparent reporting to become more market-driven assuming that the competition within the market, led by public choice and institutional selection, will drive improvements in learning and will also self-govern the institutions. This article explores how Foucault informs our conception of neoliberal governmentality through political rationality and technologies of self-governance in order to inform our understanding of accountability in higher education and to challenge institutions to develop a counter dialogue which meets institutional obligations to the public and market.     

教育质量与质量证据：2013年美国高等院校认证标准

美国中北部区域高等教育认证委员会2013年新认证标准对教育质量的评估提出了进一步的要求,强调教育要有质量,质量要有证据,评估重点转型到对学生学习结果的评估,作为判断学生学习质量的主要证据之一。在了解和借鉴境外同行的经验中,处理学校运行常态信息与质量信息的关系,思考教育理念共识与基于文化之教育理念之间的衔接点,可以使教育工作者对教育质量在现实社会大环境中的内涵有比较合理的认识。     

市场竞争中的高校评估及其范式的更新

在市场力量的推动作用下,当今的高校评估不再局限于学校内部,而是依据高校自身设定的使命完成评估的循环过程,以期同步实现问责制。作者提出了一个新的三级评估范式,以对公众需求的信息做出回应。新评估范式的三个等级需要展示学生学习的成果、显示高等教育投资回报率和确保高校的社会效益。在这个三级评估范式中,评估活动根据信息接收端的需求展开,并且上一级的评估总是建立在下一级评估所提供的信息基础之上。     

基于学生学习评估的美国高等教育问责

通过文献分析,对美国斯坦福大学教授Richard J. Shavelson所提出的基于学生学习评估的美国高等教育问责制进行了综述性介绍,对其中的学习评估、问责体系构建作了深入探讨。这一以改进高等教育质量为根本目标的新型问责体系对我国高等教育的改革与发展同样具有适用价值。