Validity of student self-reported ratings of learning

This study examined the relationship between individual student self-reported ratings of progress on relevant learning objectives and performance on exams administered during a college course. Across three sections of the same course taught by a single instructor, 188 students rated themselves at the end of the course on two objectives identified by the instructor as either essential or important. They also rated themselves on 10 other objectives the instructor identified as having minor or no importance. Self-ratings on course-relevant objectives correlated significantly and positively with four out of five exams and the course total, whereas ratings on irrelevant objectives did not. Students who rated their progress as either exceptional or substantial generally performed better on course examinations than those who rated their progress as moderate or less. These findings support the validity of student self-reported ratings of learning.     

Undergraduate Students’ Preferences for Constructed Versus Multiple-Choice Assessment of Learning

Students (N = 161) in seven sections of an undergraduate educational psychology course rated ten performance-assessment options in collegiate courses. They rated in-class essay exams as their most preferred assessment and multiple-choice exams (in-class and out-of-class) as their least preferred. Also, student ratings of multiple papers and a term paper did not differ significantly from the rating for in-class essay exams. Overall, students preferred constructed forms of assessment over more objective assessment. With minor exceptions, student ratings of assessment preferences were generally consistent across gender and academic levels. In the main, student ratings of assessment options did not significantly correlate with exam performance in the course.     

Effects of classroom evaluation strategies on student achievement and attitudes

This study investigated the effects of teacher evaluation and the combination of teacher evaluation and student self-evaluation on student performance and attitudes. Participants in the study were 189 Latvian high school students and their six teachers. The six teachers were assigned to one of three treatment conditions: (a) no evaluation, (b) teacher evaluation, and (c) self-evaluation plus teacher evaluation. All groups completed a 12-lesson instructional program on how to conduct experiments and produce research reports. Students in the teacher-evaluation group received teacher evaluation on their initial research reports. Students in the self-plus-teacher evaluation group self-evaluated their reports and received teacher evaluation on them. The no-evaluation group received no formal evaluation instructions. Students in the teacher-evaluation and the self-plus-teacher evaluation groups received significantly higher ratings on their final projects than those in the no-evaluation group. However, the no-evaluation group had more favorable attitudes toward the program than the other two groups, while the self-plus-teacher evaluation group was significantly more confident of their ability to independently conduct future research experiments. This study was conducted while Dr. Olina was a student at Arizona State.     

Even after thirteen class exams,students are still overconfident: the role of memory for past exam performance in student predictions

Students often are overconfident when they predict their performance on classroom examinations, and their accuracy often does not improve across exams. One contributor to overconfidence may be that students did not have enough experience, and another is that students may under-use their knowledge of prior exam performance to predict performance on their upcoming exams. To evaluate the former, we examined student prediction accuracy across 13 exams in an introductory course on educational psychology. For the latter, we computed measures that estimate the extent to which students use the prior exam score when predicting performance and whether students should use the prior exam scores. Several outcomes are noteworthy. First, students were overconfident, and contrary to expectations, this overconfidence did not decline across exams. Second, students’ prior exam scores were not related to subsequent predictions, even though prior exam performance showed little bias with respect to predicting future performance. Thus, students appear to under-use prior performance despite its utility for improving prediction accuracy about future exam performance.     

Incorporation of continuous student assessment into lectures in engineering education

A continuous student assessment system was incorporated into an advanced microelectronic course. This study investigated the relationship between the continuous assessment system based on home exams and individual student achievement. The perspective was based on the learning frameworks of the social constructivist theory. Six fourth-year engineering students participated in the study, which covered 13 lectures and 5 home exams. Feedback sessions concerning the particular exam were held after every exam. Correlations between the exams, the feedback, and individual student achievement were computed. The results indicated a positive correlation between continuous assessment and student achievement. Rather than being improved a lot, student achievement stabilised statistically at a higher level. Additionally, student's absence was very low (5%) despite the voluntary participation in the course. Continuous assessment realised with home exams induced two-way discussions between the teacher and the students. Unprompted, the students learned additional material and discussed it in the exam essays, confirming the principles of social constructivist theory.     

Relationships between psychometric dimensions of item quality and student ratings of item relevancy and ambiguity

Students rated the quality of the items on a classroom test that had been taken previously. On the same test, psychometric item indices were calculated. The results showed that the student ratings were related to the item difficulty, but not to the item-test correlation. In addition, the better-achieving students tended to rate the items as less ambiguous. Finally, the ambiguity ratings were more highly related to the item-test correlations for the better achieving students. These findings support opinions held by many instructors of students' judgments of item quality.     

Predicting student ratings: the relationship between actual student ratings and instructors’ predictions

This study examined the relationship between student ratings and instructors’ predictions of these ratings, taking into account other instructor, student, and course characteristics. Participants in the study were 198 instructors in the School of Education at a major teacher training college in Israel. Data corresponding to one randomly selected course per instructor were collected using student and instructor questionnaires and college records. Results indicate a systematic positive relationship between instructors’ predictions and actual student ratings with respect to overall ratings and the ratings of three dimensions of teaching. Results also demonstrate a systematic trend whereby low‐rated instructors tend to overestimate their student ratings, high‐rated instructors underestimated ratings, and moderately rated instructors gave accurate predictions. Results have implications for using predictions to motivate teaching improvement.     

Challenges and strategies for assessing student workplace performance during work-integrated learning

This study explores the challenges of assessing student workplace performance during work-integrated learning. It highlights the need for, yet difficulties with, combining positivist and constructivist assessments where workplace supervisors make evaluative judgements on performance yet students are also agents in their own assessment. It examines the ratings awarded by 163 workplace supervisors for 213 business undergraduates completing a work placement as part of their degree program in Western Australia. Students were rated on 17 capabilities associated with employability and results indicate, in alignment with previous studies, a tendency among supervisors to assign inflated marks across capabilities. The mean capability rating awarded to each student was significantly higher than their weighted course average, suggesting workplace supervisors mark more highly than academics in coursework units. To identify solutions to manage leniency bias, the study examined variations in supervisor ratings for a range of personal and contextual variables such as gender, organisation size, work area and sector. Although supervisor ratings were inflated, they were consistent across the sample with variations recorded for only four capabilities in certain work areas. Reasons for leniency bias among workplace supervisors are explored in light of the findings and alternative approaches to evaluating student workplace performance are presented.     

A comparison of Australian and Chinese teachers' attributions for student problem behaviors

The present study compared Australian and Chinese teachers' causal attributions for student behavior. A total of 204 Australian teachers and 269 Chinese teachers rated the importance of four causes (ability, effort, family, teacher) of six student problem behaviors. Results showed that both groups of teachers attributed misbehaviors most to student effort and least to teacher factors. Chinese teachers emphasized family factors more while Australian teachers placed greater importance on ability. There was significant variation in attribution patterns for different types of problems, with effort attribution being equally and strongly emphasized across cultural contexts and behavior types. The results are interpreted in the light of how individualistic and collectivistic values influence teacher thinking, and implications for school‐based interventions for behavior problems are discussed.     

Educational evaluation schemes and gender gaps in student achievement

This paper investigates whether gender gaps in student achievement are related to evaluation schemes. We exploit different evaluations at the end of compulsory education in Norway in a difference-in-differences framework. Compared to the results at anonymously evaluated central exit exams, girls get significantly higher grades than boys when the same skills are assessed by their teacher. This gender grading gap in favor of the girls is found in both languages and mathematics. We find no evidence that the competitiveness of the environment can explain why boys do relatively better on the exam. We find some evidence that the gender grading gap is related to teacher characteristics, which indicates that the teacher–student interaction during coursework favors girls in the teacher grading.     

Evaluating the quality of peer and self evaluations as measures of student contributions to group projects

ABSTRACT

Educators in higher education commonly use peer and self evaluations to help assess student performance on group projects. Although these evaluations provide multiple benefits, many educators are wary of using them due to concerns about their quality. This study addresses three questions debated in the literature regarding the quality of these assessments. How much do students differentiate among peer contributions through their ratings? How reliable are peer ratings? How much agreement exists between peer and self ratings? Although these questions have been addressed to varying degrees in past work, their answers have been far from settled. While many studies focus on just one of the questions, this study’s data make it possible to address all three questions for the same group of students as well as examine each question by student performance level. The evaluations assessed in this study were completed by a large number of students under conditions associated with obtaining more valid and reliable ratings. Overall, the results provide support for using peer and self evaluations to help assess student contributions to group projects. Peer ratings were largely reliable as group members generally agreed on the scores given to their peers. In addition, most students differentiated among group member contributions through their ratings. Students also tended to rate themselves higher than their peers rated them. This study has implications for how peer and self evaluations can be most effectively used by educators to measure student performance in group work.     

A Systematic Replication Comparing Interteaching and Lecture in the Community College Classroom

Researchers demonstrated the effectiveness of interteaching relative to lecture in 4-year university classrooms, but exploration in other settings is deficient. This systematic replication examines the extent to which interteaching leads to increased exam scores compared to traditional lecture in the community college classroom. Participants in two introductory psychology sections took identical exams following counterbalanced alternating lecture and interteaching conditions. Most student exam scores following interteaching were slightly higher than exams following the lecture condition, with statistically significant differences in two of the six exams. Students in both sections correctly answered more interteaching-based than lecture-based questions on the cumulative final exam, although these differences were not significant. During interteaching, students earned significantly more points throughout the semester compared to the lecture condition. Also, more students reportedly preferred interteaching relative to lecture. The results of this study comparing interteaching to lecture in a community college setting are consistent with the results in the original study within a 4-year university classroom. Students’ exam score gains and significant cumulative point differences suggest interteaching may be an effective alternative to traditional lecture, potentially producing meaningful differences in student performance within the community college setting.     

Assessment of the effects of student response systems on student learning and attitudes over a broad range of biology courses

With the advent of wireless technology, new tools are available that are intended to enhance students' learning and attitudes. To assess the effectiveness of wireless student response systems in the biology curriculum at New Mexico State University, a combined study of student attitudes and performance was undertaken. A survey of students in six biology courses showed that strong majorities of students had favorable overall impressions of the use of student response systems and also thought that the technology improved their interest in the course, attendance, and understanding of course content. Students in lower-division courses had more strongly positive overall impressions than did students in upper-division courses. To assess the effects of the response systems on student learning, the number of in-class questions was varied within each course throughout the semester. Students' performance was compared on exam questions derived from lectures with low, medium, or high numbers of in-class questions. Increased use of the response systems in lecture had a positive influence on students' performance on exam questions across all six biology courses. Students not only have favorable opinions about the use of student response systems, increased use of these systems increases student learning.     

Intellectual Ability,Social Desirability,and Differential Teacher Effectiveness

This study investigated the relationship between intellectual ability and differential teacher effectiveness. The relationship between intellectual ability and student ratings of student teachers was considered separately for different types of students. In addition, the Marlowe-Crowne Measure of Social Desirability was used as a moderator variable. The principal findings were: (1) student teachers who were low on intellectual ability measures were rated more favorably by their non-college-bound than by their college-bound students provided they were low on the Marlowe-Crowne and (2) student teachers who were low in intellectual ability and high on the Marlowe-Crowne were rated more favorably by their college-bound than by their non-college-bound students.     

Increasing Student Success Using Online Quizzing in Introductory (Majors) Biology

Students often complain about their perceived disconnect between the time and effort spent studying and their subsequent performance on exams. Robert Bjork''s research asserts that retrieval of stored information acts as a memory modifier, and that using tests as learning events creates “desirable difficulties that enhance learning.” To determine the effect of utilizing testing as a learning event in the introductory (majors) biology classroom, we used an online homework platform to give required quizzes throughout the course. Analysis of exam grades earned by those taking 100% of pre-exam quizzes indicates that not only does this group have a significantly higher exam average than the group of students who took 0% of the pre-exam quizzes, but they also have a significantly higher exam average than the class average. Through detailed, statistical analysis, the benefit of quizzing is demonstrated to be significant for students of diverse academic abilities. Pre-exam quizzing using an online homework platform is an effective way to increase student performance on exams and allows class time to be utilized for teaching activities.     

The generalizability of student ratings of instructors: Item specificity and section effects

A number of recent studies have used generalizability theory to examine the dependability of student ratings of instruction. This study extends this line of research by examining the consistency of ratings between different sections of a course taught in a given semester by the same instructor, and by comparing the performance of global- and attribute-type instructor rating items. Five samples of physics instructors, varying in size from 5 to 12 instructors, were rated by their students on a form containing two global and eight attribute items. Each instructor taught two sections of a course. The study found that the section effect was small (ratings of instructors were consistent across different sections of the same course), and that the generalizability of ratings was substantially influenced by item specificity. For summary purposes, one global item seemed sufficient.     

The quality of student ratings of teacher behaviour

In a research project into the effectiveness of mathematics teaching in the first year of secondary education, external observers and students rated teachers' behaviour. The reliability and validity of both methods were established. The results show that teacher behaviour is assessed well when student ratings are aggregated at the classroom level. The quality of aggregated student ratings is as good as the quality of data from external observers. The predictive validity of aggregated student ratings is higher than the predictive validity of external observations when subject motivation is taken as a dependent variable.     

Frequency of examinations and student achievement in a randomized experiment

We carry out a randomized experiment involving undergraduate students enrolled at an Italian University attending two introductory economics classes to evaluate the impact on achievement of examination frequency and interim feedback provision. Students in the treated group were allowed to undertake an intermediate exam and were informed about the results obtained, while students in the control group could only take the final exam. The results show that students undertaking the intermediate exam perform better both in terms of the probability of passing the exams and of grades obtained. High ability students appear to benefit more from the treatment. The experiment design allows us to disentangle “workload division or commitment” effects from “feedback provision” effects. We find that the estimated treatment impact is due exclusively to the first effect, while the feedback provision has no positive effect on performance. Finally, the better performance of treated students in targeted examinations seems not to be obtained at the expenses of results earned in other examinations.     

Observational,student, and teacher perspectives on interpersonal teacher behavior: Shared and unique associations with teacher and student emotions

Teachers’ interpersonal behavior in class is important for teacher and student emotions. Often the same rater (either teacher or students) is used to assess both perceptions of teacher behavior and emotions, which makes it vulnerable to common-method bias. Including other perspectives on teacher behavior has been proposed as a solution, but it is unclear to what extent different perspectives are correlated and how to separate their shared and unique variance in explaining emotions. Behavior of 80 teachers was rated from three perspectives (observers, students, and teachers) in terms of Agency (i.e., social influence) and Communion (i.e., friendliness). The three perspectives overlapped more strongly for teacher agency than for communion. Especially for students, teacher communion was a stronger predictor of emotions than agency. Our innovative statistical approach showed that the strong association between ratings of teacher behavior and emotions of the same rater are unlikely to result from common-method bias only.     

Relationship of teachers’ ratings of kindergarteners’ 21st century skills and student performance

This study investigated the relationship of kindergarten teachers’ ratings of their students’ 21st century skills (college readiness skills) with students’ behavioral and academic performance. Teachers rated the frequency that their students (n = 579) demonstrated persistence, curiosity, affective, and cognitive (e.g., critical thinking) behaviors within their classrooms via the Human Behavior Rating Scale: Brief (HBRS: Brief, a teacher rating scale. The relationship of the HBRS: Brief teachers’ ratings was compared with data the school annually collected (behavioral ratings, academic performance, student office discipline referrals [ODRs], and absences). Hierarchical linear modeling indicated that teachers’ ratings of students’ persistence and cognition behaviors were significantly associated with students’ academic performance. Teachers’ persistence, curiosity, and externalizing affect ratings were predictive of behavioral ratings and teachers’ externalizing affect ratings were significantly associated with ODRs. The results support the efficacy of investigating teacher perceptions of students’ 21st century skills with kindergarteners.