加拿大高校学生学习成果评估的内容、方法及启示

作为全球第二个实现高等教育普及化的高等教育强国,加拿大高校学生学习成果评估体系富有特色:在评估内容方面,构建起了高等教育系统、院校以及学科(专业)等不同层面的学生学习成果体系;在评估方法方面,形成了由标准化测试、评价量规、表现性评估以及问卷调查等四种类型构成的多元化学生学习成果评估方法体系。其对我国的启示是:构建和完善高校学生学习成果评估的内容体系,为学生学习成果评估奠定共识基础;开发多元化的学生学习成果评估方法与工具,提高评估活动的科学性;加强以结果为导向的评估制度建设,促使学习成果评估常态化。     

Attaining competencies in Programme Outcomes through Open-Ended Experiments

Abstract

Laboratory exercises have created benefits in student learning. Making laboratories open ended pushes students to have an independent perspective and to think harder. An openended experiment is where students are given the flexibility to develop their own experiments. Outcome-based education is an educational theory that bases each part of an educational system around objectives. Measuring programme outcomes (POs) in engineering education is a challenge for teaching faculty in general, and doing so for a laboratory programme and through an open-ended experiment, is even more challenging. This article explains the planning and execution of conducting an open-ended experiment in a metal cutting laboratory to enhance active learning. If also focuses on the evaluation process, rubrics developed, and attainment of POs in the Industrial and Production Engineering (IPE) programme.     

The use of scoring rubrics for formative assessment purposes revisited: A review

The mainstream research on scoring rubrics has emphasized the summative aspect of assessment. In recent years, the use of rubrics for formative purposes has gained more attention. This research has, however, not been conclusive. The aim of this study is therefore to review the research on formative use of rubrics, in order to investigate if, and how, rubrics have an impact on student learning. In total, 21 studies about rubrics were analyzed through content analysis. Sample, subject/task, design, procedure, and findings, were compared among the different studies in relation to effects on student performance and selfregulation. Findings indicate that rubrics may have the potential to influence students learning positively, but also that there are several different ways for the use of rubrics to mediate improved performance and self-regulation. There are a number of factors identified that may moderate the effects of using rubrics formatively, as well as factors that need further investigation.     

Examining the Effectiveness of a Learning Outcomes Assessment Program: a Four Frames Perspective

Assessment of learning outcomes at the program level is essential to evaluate whether students are achieving what is expected of them as graduates. In this article we present the results of a study in which faculty focus groups were consulted so as to understand the subjective issues that surround the learning outcomes assessment program of an institution. We hope that our study contributes to continuous improvement in institutional assessment practices and to the improvement of student learning. We analysed the data through the lens provided by a leadership model since leadership is a key driver of assessment practices that lead to changes to improve student learning.     

The contribution of rubrics to the validity of performance assessment: a study of the conservation–restoration and design undergraduate degrees

Rubrics have attained considerable importance in the authentic and sustainable assessment paradigm; nevertheless, few studies have examined their contribution to validity, especially outside the domain of educational studies. This empirical study used a quantitative approach to analyse the validity of a rubrics-based performance assessment. Raters evaluated the performance of 84 first-year university students producing service-learning projects for the Conservation–Restoration and Design degrees. The study data comprised the 9240 scores given by two teachers and three student tutors, who assessed the students’ projects on three occasions during the semester. Factor analyses confirmed that the students attained the expected learning outcomes and made significant learning progress. This learning progress was also corroborated by analyses of variance. The attainment of the learning goals and the evidence of learning progress demonstrated the validity of the inferences drawn from the assessment system. In addition, the results highlighted the need to consider rubrics as a first-order teaching resource and not only as a scoring tool.     

Toward Improving Student Learning: Policy issues and design structures in course-level outcomes assessment

Conscientious instructors at all levels are constantly making decisions about how to improve their teaching. This is a natural, intuitive part of being a good instructor, but even faculty who value student learning and work hard to improve their teaching can fail to make the connection between what they do in the classroom and the continuous improvement philosophy of outcomes assessment. The connection can be missed because understanding course-level outcomes assessment requires understanding evaluation policies and structures that differ markedly from those employed for unit-level program accreditation. The purposes of this paper are two-fold: first, to address policy issues that impede adoption of systematic, course-level outcomes assessment for improving student learning; and second, to describe a design structure for applying outcomes assessment that addresses factors under control of the instructor that affect students' learning. The paper concludes with observations on the crossed purposes of collecting information for judging faculty merit in the annual review process as opposed to collecting information for course improvement in the outcomes assessment process.     

建构主义教学与学习评估方法的探讨

简要分析建构主义教学的特征,并从建构主义教学的角度,对目前英美等国许多大学普遍采用的大学生学习评估方法以及各种方法的利弊进行阐述和比较。同时,对提高中国高校学习评估的可靠性和有效性提出建议。     

REDUCING TRANSFER SHOCK THROUGH FACULTY COLLABORATION: A CASE STUDY

This study examined the teaching techniques used by community college faculty and the student learning outcomes that faculty sought to accomplish. Data were collected through a survey designed to assess the degree to which a variety of techniques were used and the related student learning outcomes that were intended. Participants (N = 197) were career and transfer faculty at a large metropolitan community college in the midwestern United States. Analysis of the data included determining the frequency of use of each technique and examining the relationships between frequency of use, intended learning outcomes and four independent variables (gender, years of experience, rank, and teaching area). Results indicated that the lecture was the predominant technique used, although a majority of the faculty also used other techniques. When faculty used techniques other than lecture a majority of the time, they did so to accomplish higher order learning outcomes. Significant relationships were found between frequency of use, intended learning outcomes, and teaching area. Faculty teaching in the transfer curriculum were more likely to lecture a majority of the time and to intend higher order learning outcomes than career faculty who lectured to impart information.     

Using Action Research to Innov8: Facilitating Culture Shifts in the Scholarship of Teaching and Learning

ABSTRACT

The mission of a community college is distinct from a research one university, where empirical research is valued over investigation of discipline-based teaching and learning. The open-access, affordable, workforce, and transfer emphasis that serves many non-traditional community college students is a rich, meaningful environment for fostering the scholarship of teaching and learning, yet despite an emphasis on best practices and student success outcomes, teaching, and learning in this environment is an underappreciated element of the faculty role. This study explores the use of action research to facilitate engagement in the scholarship of teaching and instructional innovation. The mixed-method action research design incorporated quantitative and qualitative analysis from a wide variety of data sources and triangulated findings through a variation of techniques, including ongoing researcher dialogue and ratings. Action research serves as both the method for this study and the mechanism for faculty research on classroom innovations. This was substantiated as a catalyst for cultivating ongoing inquiry and professional growth. The Innov8 program and integration of action research appears to have a long-term, substantive impact on faculty perspectives, specifically influencing faculty culture, continued educational investigation, and ultimately student learning outcomes.     

Facilitating Student Learning,the Assessment of Learning,and Curricular Improvement Through a Graduate Social Work Integrative Seminar

Social work programs undertake to educate competent professionals and to promote critical thinking among students who are expected to become lifelong learners. In addition, programs are called on to develop reliable and valid measures for assessing student learning outcomes and to use assessment results to reflect upon and strengthen curriculum. This study explores the use of a graduate integrative seminar to accomplish these goals. The authors describe the learning and assessment processes of the seminar. The authors used a mixed-method research design to analyze data from students over a 3-year period regarding the effectiveness of the seminar as well as from graduate faculty regarding student achievement of curricular objectives. Results indicate that formative assessment processes promote student learning in a manner that mobilizes students to strive for excellence, and that an embedded measure of learning outcomes in the form of a final integrative project may serve as a rigorous assessment tool.     

Balancing Optimal Assessment with Part-Time Faculty Participation: A Discipline's Dilemma

Part-time faculty need to be brought into the student learning outcomes assessment loop not only to help accreditation, but because they, like full-time faculty, can benefit from assessment. When part-time faculty are not participating in assessment, a sizable percentage of courses are being less effectively taught than they could be. In an attempt to increase part-time faculty in discipline-level assessment at Norco College, we experimented with several assessments that required varied levels of effort by part-time faculty. This piece discusses our experience. We found that peer mentoring and communication were better predictors of part-time faculty participation than stipends, adaptable assessment prompts, or administrative pressure.     

Student perceptions of assessment and student self-efficacy in competence-based education

The purpose of this study was to provide insight into the interplay between student perceptions of competence-based assessment and student self-efficacy, and how this influences student learning outcomes. Results reveal that student perceptions of the form authenticity aspect and the quality feedback aspect of assessment do predict student self-efficacy, confirming the role of mastery experiences and social persuasions in enhancing student self-efficacy as stated by social cognitive theory. Findings do not confirm mastery experiences as being a stronger source of self-efficacy information than social persuasions. Study results confirm the predictive role of students’ self-efficacy on their competence outcomes. Mediation analysis results indicate that student’s perceptions of assessment have an indirect effect on student’s competence evaluation outcomes through student’s self-efficacy. Study findings highlight which assessment characteristics, positively influencing students’ learning, contribute to the effectiveness of competence-based education. Limitations of the study and directions for future research are indicated.     

Assessing Millennials in College‐Level Fashion Design Studios: A Study of Evidence‐Based Practice

Providing quality feedback that is both positive and effective while maintaining the required student grade confidentiality is a conundrum for faculty. This issue is particularly complex for faculty teaching studio courses (e.g. fashion design) with performance or creativity components where feedback has traditionally been given publicly as desktop reviews in the open studio. The difficulty of giving feedback in the studio‐style classroom is compounded when teaching millennial college students who tend to reject any criticism, public or otherwise, of their work. Previous research on assessment, rubrics and feedback as well as personal experience with studio classes provide background to this evidenced‐based study comparing three feedback and delivery methods: abbreviated rubric, extensive rubric and one‐to‐one meetings (e.g. dialogical assessment) guided by a project list. Grade consistency and ranking of grades were evaluated. Studio projects from a fashion design course were graded across two semesters using the multiple methods. Comparisons were made based on: (a) student reaction, (b) quality of rework and (c) time to grade. The outcomes showed that the one‐to‐one meetings as a feedback and delivery method of assessment had several benefits over two other methods, including added student engagement and improvement of rework.     

The paradigm shifts: Examining quality of teaching through assessment of student learning

Student learning is the strongest criterion for evaluating effective teaching. This article looks at three practices which partner students, teachers, and faculty peers in understanding the quality of teaching and learning through student course portfolios and teacher course portfolios, majors portfolios and teaching portfolios, and collaborative classroom assessment. Two outcomes of using a learning paradigm are building community and representing successes in higher education through documentation of effective teaching and learning.Barbara L. Cambridge, Professor of English and Associate Dean of the Faculties at Indiana University-Purdue University Indianapolis, serves on the IUPUI Program Review and Assessment Committee and recently chaired the Task Force on Accountability and Assessment for Indiana University's Strategic Directions Longrange Planning. She has made presentations and written on portfolio assessment, assessment and evaluation of faculty work, and issues in undergraduate education. Editor of theJournal of Teaching Writing, she is past President of the National Council of Writing Program Administrators.     

Implementing Outcomes Assessment in an Academic Affairs Support Unit

While outcomes assessment has become a focus for institutions of higher education, assessment in academic support units remain sparse. Traditionally, assessment has been comprised of one-time student satisfaction surveys and/or supervisor evaluations tied specifically to job criteria. Although informative, these methods fail to account for or measure specific student learning outcomes completed by students via their interaction with faculty and staff. This article presents the development and implementation of a program of outcomes assessment at a 4-year public university in the northeast. Results indicate that students come to college with varying expectations of the role of academic advisors and range on a continuum of academic and social preparation to meet the demands of higher education. A majority of students are meeting the stated learning outcomes. Feedback from this assessment model is being used to improve the quality of services provided to successive cohorts of students at this institution.     

Articulating and Utilizing Communication Student Learning Outcomes in Community Colleges

Academia in the United States stands face-to-face with the constant call for greater levels of assessment and accountability. Academics question whether our students are learning what they’re taught? Are they able to secure employment upon completing their education? Are they prepared to be active, productive citizens in society? Community colleges provide education to those who may typically not have the opportunity to attend an institution of higher education. A foundation of any assessment process is the articulation of the core learning outcomes. It is from this foundation, then, that community college faculty can start assessing the effectiveness of teaching and curricular design. The present investigation details a project that focused on creating discipline-specific student learning outcomes and how these outcomes can be utilized to aligning learning outcomes and mapping to curriculum. Communication studies faculty can utilize the learning outcomes to strengthen and clarify curriculum, demonstrate the acquisition of competencies, and improve departmental offerings.     

Chinese business students’ changes in beliefs and strategy use in a constructively aligned PBL course

This study adopted a longitudinal retrospective case study approach to investigate Chinese business students’ transitional learning experience in a problem-based learning (PBL) course with innovative assessment practices. The study focused on students’ beliefs and strategy use in a constructively aligned PBL course for business communication. Eight students who had made notable progress were chosen for retrospective analysis. The data included 48 journal entries, interviews, and writing samples collected at different stages of the course. This study identified taxonomies of participants’ beliefs about learning and writing, their perceptions of assessment, and their strategy use for learning. It also examined changes in beliefs, perceptions, and strategy use to determine the nature of the students’ learning experience in this PBL course. Findings suggest a recognised need to design PBL courses that align social constructivist learning principles with students’ beliefs and strategies. The results also highlight the importance of developing appropriate assessment rubrics to enhance student engagement with PBL learning for improved outcomes.     

Transitioning to learning outcomes at the coalface: An academic’s quantitative evaluation at the course level

Implementation of learning outcomes in Universities has not been seamless – their alignment with assessments requires faculty to specify learning outcomes and show transparent evaluation of this alignment. Difficulties arise transitioning from grading within a range of pre-existing assessment formats to grading common learning outcomes across those different formats. Experiencing this, we performed a quantitative evaluation of students completing an academic literacy course. We aimed to see if student achievement was determined by assessment format or by learning outcomes, and then to identify whether achievement of a learning outcome was equivalent across assessment formats. A twostep cluster analysis identified three clusters of students: high, medium, and low achievers. Strongest predictors of cluster-membership were learning outcomes assessed in the written essay, however scores for each learning outcome when assessed across assessment formats correlated poorly. Faculty should ensure consistent standards in learning outcomes achievement when assessed across different formats, or clearly separate learning outcomes.     

Testing the Engagement Theory of Program Quality in CACREP‐Accredited Counselor Education Programs

This study examined the engagement theory of program quality ( Haworth & Conrad, 1997 ), which highlights positive student learning outcomes that result from stakeholder involvement in program evaluation within master's‐level graduate programs. A total of 481 master's‐level counseling students and 63 faculty members, representing 68 Council for Accreditation of Counseling and Related Educational Programs (CACREP)–accredited counselor education programs, participated in the study. Findings reveal that engagement theory is a potentially useful quality assessment resource for CACREP‐accredited programs in their efforts at enhancing and sustaining program quality.