Assessing ‘inexperienced’ students' ability to self‐assess: exploring links with learning style and academic personal control

The study sought to establish the level of students' self‐assessment skill—particularly inexperienced students—and to examine the relationship between self‐assessment skill and learning style, student perceptions of academic locus of control and academic self‐efficacy. Students were asked to evaluate and provide estimated marks for their own work, were which compared with tutors' actual marks. Students also completed measures of learning style, academic locus control and academic self‐efficacy. Comparisons of student estimated and tutor marks indicated a good level of self‐assessment skill in the majority of students. A significant minority of students did however fail to exhibit such skills. There was also some evidence of a tendency for students to underestimate their performance. While both strategic and deep approaches to learning were shown to be positively correlated with tutor mark, only surface approach was negatively correlated with students' estimated mark, suggesting that surface learners are inclined to provide lower evaluations of their own performance. Deep approach was also correlated with accuracy of student self‐assessment skill, suggesting that deep learning is associated with self‐assessment competency. No clear or convincing associations between self‐assessment skill and perceptions of academic locus of control or academic self‐efficacy were identified. Findings suggest that while self‐assessment skill undoubtedly develops, becoming more effective during students' academic career, inexperienced students do have the capacity for self‐evaluation and should therefore be included in self‐assessment activities. In the light of findings related to learning style and the heterogeneous nature of student groups, student monitoring and skill development are proposed in order to allow the integration of self‐assessment into the learning and assessment process.     

Are health professional competency assessments transferable across cultures? A preliminary validity study

This study investigated if professional competency assessments are transferable across cultures using COMPASS®: Competency assessment in speech pathology, a tool developed and validated in Australia. Students in Hong Kong were assessed by clinical educators using COMPASS® and the usual clinical evaluation forms. Analyses compared Hong Kong data with the original Australian field trial data. Rasch analysis was used to evaluate how well the ratings and score generated represented students’ development of competency. Hong Kong clinical educators’ ratings represented the same seven distinct categories of judgement as Australian clinical educators. The order of item difficulty was very similar for the two samples. However, Hong Kong clinical educators were not rating students in a pattern that reflected increasing competency with experience and very few year 4 students were rated at entry level. It is concluded that an assessment tool validated and developed in one culture may well support valid judgements and yield measures that can be used to judge student competency in another culture. Further evaluation is required to investigate the differences in the judgement of student progress in another culture and strengthen the validity of using its measures to judge students' competency performance.     

THE DETERMINATION AND EVALUATION OF RESULTS IN THE STUDY PROCESS

Student assessment and grading schemes in higher education institutions should be adapted and applied in such a way that they stimulate students to achieve to the utmost of their abilities. It is not sufficient to simply give a grade to a student for a task accomplished or even to give him an average grade for a series of tasks accomplished during a given time period, a semester, for instance. Rather it is necessary to create an action framework of a kind which links the performance dynamic of both students and teachers so as to ensure the accurate determination, evaluation, and analysis of the joint student‐teacher performance behaviour in such a way that the individual procedures subject to evaluation can be grouped into joint procedures which stimulate self‐determination self‐evaluation, and self‐analysis on the part of students, teachers, and eventually, the employers of graduates. The authors have designed two models which illustrate optimum evaluation procedures of the kind which they favour.     

The usefulness of student evaluations in improving college teaching

The present study was designed to assess the effects on faculty performance of a combination of feedback and personal consultations using college student evaluations. Student evaluation feedback and personal consultations were conducted at least a semester before any follow-up data were gathered. The results indicate that providing computerized results of college student evaluations along with individual faculty consulting sessions helped the instructors to significantly improve their student ratings on two instructional dimensions.This study was conducted while the author was Head of the Measurement and Research Division, Office of Instructional Resources, University of Illinois at Urbana-Champaign. The author is indebted to Nancy F. Halff for her valuable editorial and statistical assistance, and to Pamela Z. Hexner and Jeffrey A. Slinde in conducting the data analysis.University of Arizona     

STUDENT EVALUATION OF INSTRUCTION: PERCEPTIONS OF COMMUNITY COLLEGE STUDENTS,FACULTY, AND ADMINISTRATORS

The purpose of this study was to compare the opinions of students, teachers, and administrators relative to student evaluation of instruction in selected community colleges. While important educational decisions in community colleges are made on the basis of students’ evaluations (as in retention, promotion, tenure, and pay), little has been accomplished in testing the assumptions behind student evaluation of instruction. The student evaluation process assumes that students are honest, serious, and evaluate instruction, not some incidental activity.

A 25‐item Student Evaluation Process Scale was completed by 607 students, 130 faculty, and 45 administrators in five Illinois community colleges. Findings revealed little significant differences in the opinions of students regarding evaluation of instruction based on variables of sex, age, school location, student type (transfer or occupational), and class standing. There were little significant differences in faculty opinion and within the administrative groups based on selected variables. There were significant differences when the opinions of students, faculty, and administrators were compared. Students and faculty tended to agree with those items that questioned the objectivity of student evaluation of instruction. Administrators and students tended to agree with items reflecting the seriousness with which students evaluate instruction. Faculty and administrators indicated that student evaluation of instruction impacted faculty members’ instructional performances. Neither students, faculty, nor administrators supported the concept of merit pay tied to student evaluation of instruction.

The role of student evaluation of instruction in a faculty evaluation system must be investigated. A variety of groups should participate in this investigation.     

The mystery of student selection: are there any selection criteria?

First‐year students are still failing at an alarming rate. This is an international issue that universities face and there is currently no clear indication of the cause of the problem as universities move from being elite to providing mass education. This article examines the possible correlation between students’ high school performance and first‐year performance. The focus is primarily on the students’ performance in mathematics and English. National Senior Certificate results as well as academic and mathematics competency results are used to determine any correlations and therefore any predictors. The research is done to determine whether these results are a true reflection of a student’s performance and can be used as selection criteria. The results were disappointing since no correlation was found between students’ high school performance and first‐year performance. There is an indication, however, that the competency results might be an indicator. The results imply that more research is needed and that the current selection criteria are either not valid or need to be adjusted.     

The Doctoral Portfolio: Centerpiece of a Comprehensive System of Evaluation

The authors describe the process used to revise a traditional doctoral student evaluation system from one that consisted of written comprehensive and final oral examinations to one that features portfolio development. Student competence, expected student outcomes in each competency area, procedures for portfolio development, and documents and procedures for faculty review of the portfolios are described. Recommendations for implementing such a system and implications for counselor education programs are discussed.     

基于能力本位的高职课程学习成果真实性评价模式

真实性评价迎合了当代高职教育人才培养改革和发展的需要.真实性评价中的"真实",是要求在课程学习成果评价的任务、能力、产出、标准四个要素上同时体现真实性,是对高职能力本位教学实践的补充与衔接.通过与职场现实工作情境及评价操作的参照比对,构建高职课程真实性评价设计模型,按照评价设计过程分为六个环节:能力要求、评价任务、任务条件、学生成果、评分规则、评分情境,并在真实性评价体系设计中融入对评价信度和效度的考量,以期促进真实性评价在教学实践中的应用.     

The student advisory committee: Program evaluation that develops students

A student advisory committee was formed to complement program evaluation for an open learning center. The center's staff found that the student committee members not only gave valuable feedback about instruction and services but also served as excellent problem-solvers. As a result of this committee, the center improved its evaluation process, the staff gained insight into problems from the student's perspective, and, most important, the students were given an opportunity to develop themselves.Girard Weber holds an MA degree in Educational Measurement and Statistics from the University of Iowa and MA in Linguistics and English Literature from Northern Illinois University. His chief research interests are in assessment, and retention, especially of nontraditional students.Steven Prater holds a Masters of Education in Guidance and Counseling from the University of Illinois, Champaign-Urbana. He is currently Counselor for the Open Learning Center of John Wood Community College. He has worked as a counselor for nine years and has been a staff sponsor and mentor for nontraditional students and student groups.     

Building an evaluation framework for a competency‐based graduate program at the University of Southern Mississippi

This article describes an ongoing project to build an evaluation framework for a competency‐based graduate program at The University of Southern Mississippi. Many traditional methods of evaluating performance at academic institutions provide only a partial assessment of individual program performance. In an increasingly competitive global economy, program evaluations should also consider the perspectives of students, graduates, and employers in order to develop curricula that will address the critical skills sets needed for strategic and value‐added performance improvement work in the 21st century.     

Using Online Studio Groups to Improve Writing Competency: A Pilot Study in a Quality Improvement Methods Course

Communication skills are a significant contributor to an individual's success in the workplace. Unfortunately, students often have trouble expressing their ideas in written form and the poor quality of students’ written work often impedes the learning process. This pilot study investigates the use of online writing studios within a quality improvement methods course and the impact of this intervention on students’ writing competency and perceptions about the writing process. The impact of this approach on student performance is evaluated within and across semesters using comparative statistical analyses. Results show that students who participated in online writing studios performed better (i.e., at least one rubric level higher) on their final written assignments for the course compared to students who did not, and that this approach is associated with improved students’ perceptions about the writing process. This research suggests that the use of online writing studios within a content‐specific course can incrementally improve students’ writing competency over the course of one semester, and can positively affect the learning of written communication skills and change students’ perceptions about various aspects of the writing process.     

An Evaluation of Oral Presentation Competency in Interior Design Education

As digital modelling programmes become increasingly prevalent in interior design education, there is concern that graduates are entering the workforce relying too much on strong graphic presentation skills while lacking the basic ability to speak about design. This study explores the gap between practitioners’ perceptions of importance regarding oral presentation competency and students’ perceptions of their oral presentation performances. Additionally, the study explores correlations between in‐class activities and students’ perceptions of their oral presentation competency. Mixed‐methods of investigation include a Delphi study with a panel of interior design practitioners and a survey questionnaire of both practitioners (n = 102) and currently active interior design students (n = 91) in the USA. An Importance‐Performance framework is employed for comparison. Results identify performance criteria for evaluating oral presentation competency and indicate variances between students’ perceptions of their performance and industry perceptions of importance. Furthermore, students’ in‐class activities including studio critiques and written peer assessments show significant correlation with student oral presentation performance indicating activities already frequently incorporated into a design curriculum may have a greater impact on improving performance than specific oral presentation instruction alone.     

Application of bayes’ theory in designing computer‐based adaptive instructional strategies 1

Presented are variables and conditions for design of a computer‐based adaptive instructional system. The design strategy uses Bayes’ theory of conditional probability to determine an instructional sequence according to individual student characteristics and needs. The adaptive strategy uses prior estimates based on student pretask and on‐task performance in an algorithm that makes selections of instructional sequence from a table of generated values. The computer system provides a dynamic environment which updates each students on‐task learning progress and modifies the instructional sequence accordingly. The purpose of this adaptive strategy is to improve the effectiveness of the learning environment, resulting in reductions in student learning time while improving task acquisition.     

A responsive evaluation of a graduate seminar

It was in the late sixties when the formal evaluation of teachers and their courses by students in their classes first became routinized at the University of Illinois at Urbana. Students and teachers alike were told that the purpose of the evaluations was to improve college teaching. The instruments were advocated on the basis of being vehicles for student expression in the higher education process and as part of the whole move towards accountability. In practice, however, the evaluations were used by department heads or chairpersons as part of the basis for decisions about salary, retention, promotion, and tenure. Thus, the need for a new approach to course evaluation, concentrating on improving instruction was recognized. This paper is an account of a course evaluation using the responsive approach.     

Environmental Education Network within the EEC

Pascual‐Leone (1972) has conducted task analyses in terms of M‐demand (amount of information processing required by the task) and demonstrated that the phenomenon of ‘horizontal decalages’, described by Piaget, can be accounted for by increasing demands of information processing. It has been shown that formal operational reasoning is required to balance even simple, one‐step chemical equations, while formal reasoning and a sufficiently large M‐capacity are required to balance more complex, multi‐step chemical equations. The objective of the present study is to investigate the relation between functional M‐capacity (M_f) and student performance in solving chemistry problems of increasing M‐demand. It was found that student performance decreased as the M‐demand of the problems increased. Similarly, the correlation coefficients between M_f‐ and student performance increased progressively as the M‐demand of the problems increased.     

The reflection on the blackboard: student teacher self‐evaluation

A common aim of teacher education is to encourage prospective teachers to analyze carefully their instructional performance. Yet, research on teacher cognition heretofore has concentrated primarily on experienced teachers’ planning and instructional thinking. We need more information on how student teachers think about and engage in the evaluation of their teaching performance. This study used data from initial structured interviews to elicit student teachers’ self‐evaluation concerns and examined the student teachers’ journals, a final written self‐evalution, and tapes from post‐teaching interviews to create a profile of each student teacher's responses about self‐evaluation. In this paper I analyze the student teachers’ pre‐conceptions about success, examine their processes of self‐evaluation, and explore a conception of “interactive self‐evaluation.” I offer suggestions about the conditions that may enable student teachers to enhance their analytical processes.     

Alignment of Human Resource Practices and Teacher Performance Competency

In this article, we argue that human resource (HR) management practices are important components of strategies for improving student achievement in an accountability environment. We present a framework illustrating the alignment of educational HR management practices to a teacher performance competency model, which in turn is aligned with student achievement goals. We identify and illustrate the various HR practices that could be aligned to the performance competency model and to each other. These HR practices include recruitment, selection, induction, mentoring, professional development, compensation, performance management, and instructional leadership. We then describe HR practices in 2 districts where empirical links between teacher competency and student achievement were shown (Cincinnati and Washoe County) and evaluate how much alignment was in place. We discuss the importance of HR alignment analysis for diagnosing districts' teacher quality improvement efforts, and we present suggestions for future research on the strategic use of HR management in K-12 educational organizations.     

Using Paper Helicopters to Teach Statistical Process Control

This hands‐on project uses a paper helicopter to teach students how to distinguish between common and special causes of variability when developing and using statistical process control charts. It allows the student to experience a process that is out‐of‐control due to imprecise or incomplete product design specifications and to discover how the process can be brought into control through the use of standardized product specifications and associated construction procedures. It also allows the student to experience common causes of variability, to distinguish these from the special causes of variability resulting from the incomplete design specifications, and to discover how standardization reduces process variability. The project has been used in both undergraduate and MBA classes and demonstrates these concepts in a way that is easy for the student to understand and remember.