Outcomes Assessment in Higher Education: Challenges and Future Research in the Context of Voluntary System of Accountability

Higher education institutions are becoming increasingly accountable to stakeholders at many levels as the federal government, accrediting organizations, and the public place a stronger emphasis on student learning outcomes. Evidence of student learning that is comparable across institutions is of urgent demand. Various accountability initiatives such as the Voluntary System of Accountability (VSA) mark a milestone in the evaluation of instructional effectiveness of higher education using standardized outcomes assessment. Although the current accountability initiatives hold great potential in evaluating institutional effectiveness, some design and methodological issues need to be addressed before outcomes assessment can be best utilized. In this paper, I (a) review the need for outcomes assessment in higher education, (b) discuss the VSA initiative, (c) summarize the most prominent challenges facing outcomes assessment for accountability purposes, and (d) discuss future directions of research in advancing accountability assessment in higher education.     

Developing educational goals: insights from a Peer Assisted Teaching Scheme

The rapid changes facing higher education are placing increased focus on the quality of the student experience, achieving learning outcomes, and employability expectations. As a result, academics in teaching roles are increasingly measured on performance via student evaluations amplifying attention on professional development initiatives for academics. One such initiative is the Peer Assisted Teaching Scheme currently in practice across many Australian universities. The critical component of the scheme is the establishment of teaching goals that provide focus and direction for the peer partnership. This study addresses two questions: (1) Around which aspects do academics set their goals for teaching improvement? and (2) How do academics’ goals align with the SMART goal-setting framework that is prescribed in the scheme? Findings from five Australian universities showed that goals align with a variety of educational areas yet many were underdeveloped, or misaligned with the proposed strategy. The implications of this are discussed and a framework for setting education goals for teaching improvement is developed.     

Ghana’s education reform 2007: A realistic proposition or a crisis of vision?

Ghana’s recent “Education Reform 2007” envisions a system that strives to achieve both domestic and internationally-oriented goals emanating (1) from the Education for All (EFA) initiative, (2) from the UN Convention on the Rights of the Child and (3) from global trends in education. Emboldened by the implementation of foreign-donor-funded programmes such as EFA, the restructuring of the Ghana Education Sector Project (EdSeP) and the Science Resource Centres (SRC) project, both the education reform of 2007 and recent educational policy debates have reiterated the need to emphasise the teaching of science and information and communication technology to make Ghana’s students/graduates more competitive in the global labour market. However, the bulk of Ghana’s economic activity actually remains domestic or unglobalised. And given a weak economy and declining social spending due to strict adherence to the prescribed structural adjustment policies of the International Monetary Fund (IMF) and the World Bank (WB), there is concern that a focus on international competitiveness may be a crisis of vision. On the basis of the Ghanaian government’s failure to meet the stated goals of previous reforms such as that of 1974, and the education system’s continuing dependence on foreign donor support, this paper argues that the goals of the new reform may be unachievable on a sustainable basis. It also argues that rather than subjugate national domestic priorities to a mirage of international credibility/competitiveness, Ghana should concentrate on capacitating her students/graduates to make maximum impact at domestic and local community levels.     

Unmasking complexities involved in operationalising UPE policy initiatives: Using the ‘fCUBE’ policy implementation in Ghana as an exemplar

In most parts of the world today, the goal of providing all children with free and Universal Primary Education (UPE) has received broad national and international support and some educational systems have evolved from predominantly ??fee-charging?? towards ??fee-free?? status in recent times. In Ghana, for example, the endorsement of Education for All (EFA) and millennium development goals (MDGs) agreements coupled with commitment to internal constitutional reforms have resulted in the initiation of the Free Compulsory Universal, Basic Education (fCUBE) policy. Dishearteningly however, in many low-income countries (including Ghana), verbal commitments to these laudable social goals do not appear to be translated into the needed changes in policy and practice. This article draws on a case study of the fCUBE policy implementation to provide insights into the complexities involved in operationalising UPE policy initiatives in sub-Saharan Africa. The methodological approach involved the critical discourse analysis of interviews with Ghanaian education officials who mediate policy at the ??meso-level??. Owing to the commitments of the fCUBE policy to enhancing the educational opportunities and outcomes for the socially and economically disadvantaged, the paper sees it (i.e. the fCUBE policy) as deeply rooted in social democracy. However, it is argued that as long as there is a blurring in meaning of the intentions encapsulated in its title, primary education in Ghana cannot be said to be ??free??, ??compulsory?? and ??universal??. It is concluded that accentuating policy purposes in low-income countries is not inherently problematic but that the challenges lie with how the intentions and provisions of policy are conceptualised and operationalised in context.     

Secondary Education and Research in New Zealand

Through the 1990s the secondary sector was a site of considerable change as the Government sought to ensure that the curriculum and associated assessment practices were designed to meet student needs and the long-term goals of New Zealand society. The new regime aimed to promote student achievement and in particular to attain more equitable educational outcomes for all New Zealand students. These were policy rather than practice driven initiatives that linked somewhat tenuously to research based evidence. This paper begins by giving an overview of the secondary sector in New Zealand and these educational reforms. The overview sets the context for the following section that describes links between educational research, policy and practice during this time. Key issues within the sector are highlighted and some of the ways research has been used to monitor changes and inform policy are identified. It is argued that much of the research over the past decade has been small scale and short term and while this has served the immediate policy agenda it has provided only fragmentary evidence to inform longer term goals such as improving student performance. The paper concludes by discussing some of the current initiatives that might contribute to improving the interface between research and policy/practice in secondary education. This revised version was published online in July 2006 with corrections to the Cover Date.     

Government Policy and School Development

The last 10 or 15 years of this century have been a time of great challenge as well as considerable excitement for educational systems around the world. Governments everywhere have been embarking on substantial programmes of reform in an attempt to develop more effective school systems and raise levels of student learning and achievement. We see in these policy initiatives an unfortunate paradox that inhibits them from realising their aspirations. The community of educational change researchers and practitioners has finally begun to learn something about how ongoing improvement can be fostered and sustained in schools. However, government policy on education has not taken adequate account of this knowledge about school development, with the result that an enormous potential source of synergy has been lost and student learning continues to lag behind its potential. Our argument in this article is that as a consequence of this gap government efforts to improve schooling are less effective than they might be and that many school improvement efforts have to swim against the current of government regulation. Following our analysis of this central irony in educational policy, we go on to outline an approach that would be more likely to help governments achieve their educational objectives by building policy initiatives more explicitly on the knowledge base of school development.     

Understanding student/alumni satisfaction from a consumer's perspective: The effects of institutional performance and program outcomes

The benefits of understanding and promoting student/alumni satisfaction are numerous. While this article draws on previous studies of the determinants of satisfaction in higher education, it has a different research focus. The study investigates the student/alumni satisfaction phenomenon in a consumer satisfaction framework. Results indicate that the formation of satisfaction judgments is a multidimensional process, and that the process is dependent on the degree of goal development that a student has for a particular aspect of his or her educational experience. If students have poorly formed goals for an aspect of education, they are likely to base their satisfaction judgments on institutional performance. If their goals are well formed, they are likely to base satisfaction judgments on the outcomes of the institutional performance. In general, however, student/alumni assessments of satisfaction with higher education are influenced by both the perceived quality of the service provider's performance and the perceived outcomes of that performance.The authors' names are ordered alphabetically to reflect equal contributions.     

A Model for Programmatic Evaluation by Student Assessment

The goal of the Department of Biological Sciences faculty was to evaluate how effectively our undergraduate biology program supports the goals and intended outcomes of our department and institution. Student responses in written assessment tests have provided the foundation of this evaluation. Each semester questions were selected by a departmental faculty committee to address student outcomes. The questions were administered in the first examination/quiz of the semester in several introductory through upper level courses. Student grades were analyzed by class rank (freshman, sophomore, junior, and senior) and course level (100, 200, 300, and 400) for biology, non-science, and other science majors. This protocol provided a useful way to (1) assess student knowledge and understanding of biology and (2) evaluate the biology program. The protocol, sample questions, answers, grading protocol, cumulative results with discussion (fall 2000 semester−fall 2010 semester), initiatives instituted in response to assessment results, and dissemination of results are presented for illustrative purposes. We offer this approach as a model for student assessment and program evaluation for other departments to consider.     

Research on teacher–student relationships and learning environments: Context,retrospect and prospect

The program of research on teacher–student relationships described in this issue is an important part of the field of classroom learning environments, although it has its own distinctive and significant features. The questionnaire on teacher interaction (QTI), the main instrument used in this research, follows the strong tradition in learning environments research of using the perceptions of the participants in the classroom. Although this research program originated in the Netherlands, it now is truly international and the QTI has been translated into and validated in over a dozen languages. Not only has past research consistently replicated the advantages of positive teacher–student relationships in terms of promoting improved student outcomes, but positive teacher–student relationships also are worthwhile process goals of education. In the future, it would be desirable for the QTI to be used more frequently by teachers as a feedback instrument for guiding improvements in their classroom relationships with their students, and that qualitative data-collection methods are used more often in conjunction with the use of the QTI in research on teacher–student interaction.     

The role of student and alumni associations in the democratization process in Spain

Alumni associations are institutions that are used to create, maintain, or modify relations between educational organizations and their publics. Thus, student and alumni associations were a fundamental part of the workings of the education and employment engine in Spain during the political transition. Public relations managers of higher education institutions could gain information from alumni associations, professional associations, student associations, government, and the public in general, and therefore, achieve their goals with greater ease.     

透视美国高等教育中的学生学习结课评价

学生学习结果评价是美国高等教育评价由关注资源输入到全面注重受教育者学习结果的一次转变。它侧重评价学生学习后所获得的各方面知识、能力和品质。该评价成为一场运动的主要原因是培养高技能劳动力的需求、政府问责的加强以及认证组织的推动。学生学习结果评价的实施在政府、高校以及专业、课堂等众多层面展开,并表现出以学生为中心的价值取向、评价参与者广泛以及评价方法灵活多样等特点。     

Student loans in Ghana

Student loans were first introduced in Ghana in 1971, but the scheme was abandoned the following year, due to a change of government. A revised scheme was introduced in 1975, but faced problems of high rates of default. Attempts to improve recovery since 1986 have had some success, but in 1989 the Government of Ghana introduced a new student loan scheme, which is administered by the Social Security and National Insurance Trust. This article summarises the current pattern of finance of higher education, gives a brief history of student loans in Ghana and describes the new scheme, introduced in 1989.     

Preservice Secondary Science Teachers’ Teaching and Reflections During a Teacher Education Program

The 39 preservice teachers (PSTs) who participated in this study were enrolled in a masters program for secondary science teacher certification. Initially they held broad ideas about teaching and learning gleaned from their own experiences. Guided by the program course work, some PSTs embraced the pedagogical approaches introduced in the program, applied them in their teaching, and reflected on the outcomes. Their reflections showed that they were focused on keeping all of their students interested in science and on student participation in the process of meaning-making. Some PSTs embraced the program goals but struggled to achieve them in teaching. Others focused on transmission of content and did not attempt to develop an environment of student agency. There were nine career-changer PSTs and most of them remained teacher-centered throughout the program. The implications of student- and teacher-centered approaches adopted by the PSTs and the rationales provided by them are discussed in the paper.     

Inclusive physical education? A study of the management of national curriculum physical education and unplanned outcomes in England

One key aspect of the growing policy emphasis on educational inclusion in England has been the tendency for physical education (PE) to be used as an important vehicle of social policy targeted at promoting the inclusion of young disabled people and those with special educational needs in mainstream schools. Drawing on aspects of figurational sociology, the central objective of this study is to examine the extent to which PE teachers have been able to achieve the government’s inclusion policy goals articulated in the 2000 National Curriculum for Physical Education (NCPE) for England. The findings indicate that using the NCPE as a means to pursue the government’s educational inclusion policy goals has had unplanned outcomes that have the effect of undermining the extent to which the government is able to use inclusion to make a greater contribution to the promotion of young people’s experiences of PE. It is also argued that these outcomes that no‐one had planned or intended may, for some of the groups involved, be seen as undesirable consequences that emanate from teachers’ attempts to manage the contradictory pressures brought about by pursuing the priorities of government.     

Curriculum reform and ‘Quality Education’ in China: An overview

This paper provides an overview of Chinese government attempts to promote a revised school curriculum reflecting a more holistic approach to education under the banner of suzhi jiaoyu (“quality education”), and positions the policies in the Chinese historical, educational, social and economic context. While the push has inspired innovative reforms in a number of schools and met with considerable in-principal support, on a wider scale implementation of the reforms is hampered by insufficient resources, conceptual ambiguity and conservative resistance. The goals of the new curriculum are still not reflected in the examinations and frequently conflict with teacher, student and parental goals in a society where examination outcomes have life long consequences.     

开放教育选修类课程考核方案改革探析——以法学专业为例

远程开放教育的一个重要特点是师生处于准分离的状态,因此借助一定的手段加强师生互动、监督和评价学生的学习效果,变得尤为重要,加强课程考核环节是实现上述目标的重要内容。以法学专业《环境法学与草原法学》为样本,从分析传统课程考核方案存在的问题入手,积极探索开放教育课程考核方案改革,以期强化教学质量。     

Is the Physical Being Taken out of Physical Education? On the Possible Effects of High-Stakes Testing on an Embattled Profession’s Curriculum Goals

Building on recent discussions regarding how current national standards for physical education promote cognitive outcomes over physical outcomes, the authors explore how a new era in high-stakes testing is also contributing to an emphasis on the cognitive, over the physical. While high-stakes testing has been linked to reducing the amount of physical education schools offer, less attention has been given to the newest phase of high-stakes testing: the use of student achievement test data to evaluate teachers. To explore how this new high-stakes testing may influence physical education curriculum goals, the authors examine new teacher evaluation policies in New York State. The authors then propose an alternative rational for physical education. By promoting the physical—physical activity, exercise, and structured physical movement through physical education programming—physical education can play a unique and key role in creating the conditions for both student cognitive development and improved public health.     

Achieving standards in urban systemic reform: An example of a sixth grade project‐based science curriculum

A challenge for urban systemic reform initiatives in science education has been to achieve local, state, and national standards for teaching and learning. We have collaborated with teachers in the Detroit Public School System to design project‐based curriculum materials that contextualize the learning of science in meaningful real‐world problems, engage students in science inquiry, and use learning technologies. We present a sixth grade project‐based science unit in which students explored the driving question “How Do Machines Help Me Build Big Things?” and address the science learning goals of balanced and unbalanced forces, simple and complex machines, and mechanical advantage. Twenty‐four teachers and over 2500 students in Detroit participated in enactments of this project over 4 years. Student learning outcomes were determined for the three learning goals and inquiry process skills using pre‐ and postachievement tests. Achievement outcomes as measured by the pre/posttest show significant and consistently high learning gains, even as participation in the project increased to include greater numbers of teachers and students in successive enactments, and leadership of the professional development support for this project transitioned from university researchers to district teacher leaders. These results illustrate that materials which contextualize learning and support student inquiry as part of an urban systemic reform effort can promote learning of important and meaningful science content aligned with standards. © 2004 Wiley Periodicals, Inc. J Res Sci Teach 41: 669–692, 2004     

Outcomes Assessment and Accreditation in US Engineering Formation

American higher education is characterized by enormous program and quality diversity among the 3842 institutions (1998 data) with two- or four-year programs (of which 319 institutions have one or more engineering programs). Sharp distinctions emerge along several axes: funding (private vs public), size (hundreds to tens of thousands of students) and mission (research vs teaching). Recently the accreditation organizations for universities broadly and for engineering specifically have addressed this diversity by moving to outcomes-based assessments. Rather than judging student performance in terms of classes passed, institutions must (1) define their distinctive mission, (2) design a curriculum to help students achieve these goals, (3) assess student learning outcomes according to both institutional and professional criteria, and (4) create a culture of continuous improvement to belter align steps (1) and (2). Adoption of similar procedures may help European engineering institutions to measure programs across boundaries and to foster ‘trans-national recognition’.