The Efficiency Quotient as a Measure of Achievement

Abstract

Students of 102 Elementary and Secondary school classes in Illinois rated their teachers on Rogge’s Style of Teaching Inventory, Reducing the item ratings to class means, the inter-item correlation matrix was computed and subjected to factor analysis. Three main factors were extracted: (a) Friendly and warm teacher; (b) Understanding and flexible teacher; and (c) Encouraging students’ initiative and participation teacher.

Teachers were classified along the four main types of personality preference as determined by the Myers- Briggs Type Indicator and in terms of their cognitive modes (high vs. low convergent and divergent thinking) by means of Guildord’s tests. The six bipolar classifications (Extrovert vs. Introvert, Sensing vs. Intuition, Thinking vs. Feeling, Judging vs. Perceiving, High Convergent vs. Low Convergent, and High Divergent vs. Low Divergent) provided the scheme for comparing the ratings assigned to the teachers by their students on the three factors. Comparisons were carried out separately for males and females.

The results indicated that statistically significant ratings were received by the male Sensing vs. Intuition and female High Convergent vs. Low Convergent. The Sensing male teachers were rated high as friendly and warm and low as understanding and flexible; the Intuitive male teachers were perceived and rated high as understanding and flexible and low as friendly and warm. The female Low Convergent received higher ratings on all three factors. They were perceived as more friendly and warm, understanding and flexible, and encouraging students’ participation and initiative than the High Convergent thinking female teachers.     

Predicting student ratings: the relationship between actual student ratings and instructors’ predictions

This study examined the relationship between student ratings and instructors’ predictions of these ratings, taking into account other instructor, student, and course characteristics. Participants in the study were 198 instructors in the School of Education at a major teacher training college in Israel. Data corresponding to one randomly selected course per instructor were collected using student and instructor questionnaires and college records. Results indicate a systematic positive relationship between instructors’ predictions and actual student ratings with respect to overall ratings and the ratings of three dimensions of teaching. Results also demonstrate a systematic trend whereby low‐rated instructors tend to overestimate their student ratings, high‐rated instructors underestimated ratings, and moderately rated instructors gave accurate predictions. Results have implications for using predictions to motivate teaching improvement.     

The Effectiveness of Feedback in -Teaching Principles of Educational Psychology

Sixty-seven male and seventy-three female undergraduates rated self, father, mother, and a male teacher on MARS. They also rated the teacher on a 17-item scale (RST) at the end of the course and took five teacher-made achievement tests during the semester. Students were divided into high and low achievement levels (AL), using the median as the cutoff. Two-way multivariate analysis of variance showed significant (p<.0001) differences in teacher ratings on MARS and RST factors between the sexes and between high and low AL Ss (p<.05) . Also, significant sex x AL interaction (p<.05) was found for the RST data.     

Parental responses to infant crying: the influence of child physical abuse risk and hostile priming

OBJECTIVE: Managing a crying infant is a challenge universally faced by new parents. This study examined whether parental interpretations, feelings, and behaviors following exposure to a 2-minute videotaped segment of a crying infant varied as a function of child physical abuse (CPA) risk and exposure to cues of hostility (i.e., hostile priming). METHOD: Participants included 84 general population parents (52 low and 32 high CPA risk) with valid and complete protocols. It was predicted that (1) negative trait ratings, (2) feelings of hostility, and (3) ability to modulate grip strength would differ across CPA risk groups (high vs. low) and priming conditions (hostile vs. neutral). RESULTS: As expected, high, compared to low, CPA risk parents rated the crying infant more negatively and reported higher levels of hostile feelings after watching the crying infant video. Hostile priming independently increased feelings of hostility, such that high CPA risk parents who were primed with hostile words reported higher levels of hostile feelings relative to all other conditions. Hostile priming also was modestly associated with increased use of excessive force when parents attempted to produce a half-strength grip; however this difference was apparent only among high CPA risk parents. CONCLUSIONS: High, compared to low, CPA risk parents rated the crying infant more negatively and reported higher levels of hostile feelings after watching the crying infant video. Hostile priming independently increased hostile feelings and was modestly associated with use of excessive force in the hand grip task among high CPA risk parents.     

College students' ratings of student effort,student ability and teacher input as correlates of student performance on multiple-choice exams

In the class session following feedback regarding their scores on multiple-choice exams, undergraduate students in a large human development course rated the strength of possible contributors to their exam performance. Students rated items related to their personal effort in preparing for the exam (identified as student effort in the paper), their ability to perform well on the exams (identified as student ability), and teacher input that might have affected their exam performance. Students rated most student effort items higher than teacher input and student ability items. Notwithstanding, across all exams, ratings of student ability and teacher input correlated more strongly with exam performance than did student effort ratings. High and low performers on the exams differed significantly on ratings of student ability and teacher input, but were more similar on ratings of student effort.     

Undergraduate Students’ Preferences for Constructed Versus Multiple-Choice Assessment of Learning

Students (N = 161) in seven sections of an undergraduate educational psychology course rated ten performance-assessment options in collegiate courses. They rated in-class essay exams as their most preferred assessment and multiple-choice exams (in-class and out-of-class) as their least preferred. Also, student ratings of multiple papers and a term paper did not differ significantly from the rating for in-class essay exams. Overall, students preferred constructed forms of assessment over more objective assessment. With minor exceptions, student ratings of assessment preferences were generally consistent across gender and academic levels. In the main, student ratings of assessment options did not significantly correlate with exam performance in the course.     

A study of the validity of four types of student ratings of college teaching assessed on a criterion of student achievement gains

The purpose of the study was to investigate relationships between student ratings of college teaching using four types of student rating instruments and pre- vs. post-student achievement gains in 36 sections of an undergraduate analytic geometry and calculus course. Student rating instruments used varied according to type of items (high vs. low inference) and focus (students rating their own perceived growth vs. rating the instructor). Data were collected on 799 students (66% freshmen; 16% sophomore; and 15% juniors) at the University of Florida, and relationships were analyzed using the Pearson product-moment correlation technique. Significant relationships were not found between student ratings and student achievement.     

Teachers Attitudes About Teacher Bonuses Under School-Based Performance Award Programs

The present study examines several attitudes of teachers working under school-based performance award programs in which they received monetary bonuses if their school met or exceeded measurable student achievement goals. Teachers in Kentucky and Charlotte-Mecklenburg rated the desirability of receiving a bonus and sixteen other extrinsic and intrinsic outcomes. Factor analysis collapsed these into factors labeled Goal Attainment Rewards (includes the bonus outcome), Learning, Sanctions, and Stress. Outcomes in the first two factors received high desirability ratings while outcomes in the latter two factors were rated low in desirability. In separate analyses, Kentucky teachers reported a relatively low level of motivation by the bonus program and a low desire to see the program continue but were neither more, nor less, likely to want to withdraw from the program through turnover or job transfer. Regression analyses revealed these attitudes to be a function of degree of satisfaction with base salary, the likely value of the bonus, and several aspects of the fairness of the bonus itself and of the bonus program. It was concluded that while bonuses for goal achievement have high motivational potential, this could be offset by the occurrence of multiple undesirable outcomes from the program and by a lack of careful program planning, design, and administration.     

Manager Coaching Skills: What Makes a Good Coach?

The concept of “manager as coach” is increasingly popular in the management literature as a mechanism for improving employee involvement and performance. This paper summarizes a study we conducted to evaluate “coaching skills” among sales managers. The study used telephone interviews with managers' subordinates to collect data regarding managers' coaching effectiveness. During the interviews, subordinates rated their supervisors' coaching skills and provided a rationale for their rankings. This study analyzes the ratings and the comments, and outlines the behaviors associated with both high and low ratings of manager coaching skills.     

Evaluating Teacher Effectiveness Using Classroom Observations: A Rasch Analysis of the Rater Effects of Principals

In most U.S. schools, teachers are evaluated using observation of teaching practice (OTP). This study investigates rater effects on OTP ratings among 421 principals in an authentic teacher evaluation system. Many-facet Rasch analysis (MFR) using a block of shared ratings revealed that principals generally (a) differentiated between more and less effective teachers, (b) rated their teachers with leniency (i.e., overused higher rating categories), and (c) differentiated between teaching practices (e.g., Cognitive Engagement vs. Classroom Management) with minimal halo effect. Individual principals varied significantly in degree of leniency, and approximately 12% of principals exhibited severe rater bias. Implications for use of OTP ratings for evaluating teachers’ effectiveness are discussed. Strengths and limitations of MFR to analyze rater effects in OTP are also discussed.     

Validity of student self-reported ratings of learning

This study examined the relationship between individual student self-reported ratings of progress on relevant learning objectives and performance on exams administered during a college course. Across three sections of the same course taught by a single instructor, 188 students rated themselves at the end of the course on two objectives identified by the instructor as either essential or important. They also rated themselves on 10 other objectives the instructor identified as having minor or no importance. Self-ratings on course-relevant objectives correlated significantly and positively with four out of five exams and the course total, whereas ratings on irrelevant objectives did not. Students who rated their progress as either exceptional or substantial generally performed better on course examinations than those who rated their progress as moderate or less. These findings support the validity of student self-reported ratings of learning.     

Moral Development and Empathy in Counseling

Of interest to counselor educators are variables associated with helper empathy. The authors investigated the relationship between empathy and moral development. Students enrolled in a facilitative skills development course completed a measure of moral development before making their first counseling audiotape. After approximately 12 weeks of skills training, they were subsequently rated for their demonstration of empathic understanding to client statements on an analog videotape and on a counseling audiotape made for their course evaluation. Empathy ratings of their responses to the analog videotape correlated .61 (p < .001) with moral development scores and .35 (p < .05) with supervisory empathy ratings of their final audiotape and moral development scores. Implications for counselor education are discussed.     

Course Factors that Motivate Students to Submit End-of-Course Evaluations

We surveyed students (N = 152) in several sections of an undergraduate educational psychology course to determine what course factors would most motivate them to submit course evaluations. The survey directed students to choose among several pairs of course characteristics as to their relative impact on their decision to submit a course evaluation. After tabulating their choices, we ranked the course characteristics for the total sample and then for various demographic and performance subgroups. In general, students indicated that positive aspects of a course would motivate them to submit course evaluations more than would negative aspects.     

Similarity of student ratings across instructors,courses, and time

This study raised three questions about the similarity or generalizability of student ratings of courses and instructors. First, how stable are student ratings of the same instructor giving the same course during twodifferent semesters? Second, how similar are student ratings of the same instructor in twodifferent courses? Third, how similar are student ratings of a given course being taught bydifferent instructors? Instances were identified in which student ratings on seven different factors were available for pairs of courses for each of these questions. For the case of the same instructor — same course—different semesters, student ratings were reasonably similar (median r for seven factors about 0.70). For the case of the same instructor—different courses, the median r was surprisingly low — about 0.40. For the case of the same course—different instructors, substantial correlations were obtained for some factors and insignificant correlations for other factors. Implications of these findings for practical use of student ratings and suggestions for further research in the area are discussed.     

Relations of Preschoolers' Social Acceptance to Peer Ratings and Self-Perceptions

To examine the relations of preschoolers' social acceptance to peer ratings and self-perceptions, 53 preschoolers were asked to rate how much they liked or disliked their peers and to justify these ratings. Preschoolers also rated their peers' aggressive, prosocial, and sociable behavior. Finally, they completed a pictorial self-perception scale that assessed their views of their physical competence and their relationships with mother and with peers. Children who were better liked by peers were rated as more prosocial, more sociable, and less aggressive than less liked children. Preschoolers often reported liking certain peers because they perceived that those peers liked them; they often reported disliking certain peers because they perceived those peers as aggressive. In contrast to findings with older children, preschoolers' social acceptance was not significantly related to any aspect of their self-perceptions. The results provide evidence for the validity of peer ratings by preschool-age children and bring up issues related to the development and assessment of self-perceptions among preschoolers.     

Student perceptions of organizational features in relationship to course ratings

Students' perceptions of the organization and importance of features of a course of instruction may differ from the instructors' and may influence their achievement and attitudinal reactions to the course. Two hundred twenty-nine undergraduates identified and rated the importance of organizational features of their courses, rated the course and instructor, and supplied six items of personal information about themselves. Their final course grades were also secured. A perceptual agreement score between students and instructors on course features and the importance of those features was calculated for each student. Agreement on the presence and importance of course features, along with other personal variables, were significant predictors of student achievement and ratings of courses.     

Interpretations of Parenting by Mainland Chinese and U.S. American Children

Objective. This study examined whether mainland Chinese and U.S. American children’s interpretations of their parents’ coercive authority assertion and critical comparison and shaming moderate relations between their reports of parenting and adjustment. Design. Middle-school children from mainland China (n = 217) and the United States (n = 207) rated their parents on coercive authority assertion and critical comparison and shaming, indicated whether they approved of their parents’ practices, rated their parents’ underlying intentions, and reported their own depression, antisocial behavior, and school motivation. Results. Moderation analyses showed that associations between coercive authority assertion or critical comparison and shaming and child depression were stronger for American and Chinese children whose approval ratings for these practices were relatively low. Greater coercive authority assertion was related to lower antisocial behavior for children who rated their parents high for the child beneficial interpretation and to lower school motivation for children who rated their parents low for the parent beneficial interpretation. For American children, greater coercive authority assertion also was related to greater depression for those who rated their parents relatively low for the child beneficial interpretation. For Chinese children, greater critical comparison and shaming was related to increased school motivation for those who rated their parents high on the child beneficial and/or parent beneficial interpretations. Conclusions. When children interpret their parents’ behavior in a more positive manner (i.e., they approve or think it is motivated by concern for the child), negative effects of coercive authority assertion and critical comparison and shaming may be mitigated. However, some cultural differences were found, particularly with respect to school motivation.     

More Than Meets the Eye: Parental and Infant Contributors to Maternal and Paternal Reports of Early Infant Difficultness

Objectives. This study examined correspondences among maternal and paternal ratings of infant temperament, parental psychological functioning, and infant behavior. Design. Participants included 120 families. When infants were 6 weeks old, mothers and fathers completed the Infant Characteristics Questionnaire (ICQ); reported on their own levels of anxiety, depressive symptoms, and parenting stress; and completed a 3-day diary of their infants' behavior. Infant irritability was assessed in a laboratory situation. Results. Fathers rated their infants' temperament somewhat more negatively, but there was significant correspondence between maternal and paternal ratings on the temperament factors of fussiness, unadaptability, dullness, and a difficultness composite. Higher infant difficultness was consistently associated with parenting stress. Infant behavioral fussiness, as measured by 3-day diaries, was significantly correlated with temperament ratings by both parents and with irritability observed in the laboratory setting. Maternal psychological distress was weakly predictive of ratings of infant difficultness; infant behaviors (diary and laboratory-based irritability) accounted for 15 to 17% of the variance. Paternal psychological distress and infant behaviors contributed equally to difficultness ratings. Conclusions. Mothers and fathers are influenced by somewhat different factors in perceiving their babies' temperament, but both maternal and paternal reports have a basis in laboratory- and diary-based behaviors. Results indicate the strong contributing influence of infant irritability to the perception of difficult temperament and support the validity of parental reports of infant irritability in the first 6 weeks of life.     

Evaluating the quality of peer and self evaluations as measures of student contributions to group projects

ABSTRACT

Educators in higher education commonly use peer and self evaluations to help assess student performance on group projects. Although these evaluations provide multiple benefits, many educators are wary of using them due to concerns about their quality. This study addresses three questions debated in the literature regarding the quality of these assessments. How much do students differentiate among peer contributions through their ratings? How reliable are peer ratings? How much agreement exists between peer and self ratings? Although these questions have been addressed to varying degrees in past work, their answers have been far from settled. While many studies focus on just one of the questions, this study’s data make it possible to address all three questions for the same group of students as well as examine each question by student performance level. The evaluations assessed in this study were completed by a large number of students under conditions associated with obtaining more valid and reliable ratings. Overall, the results provide support for using peer and self evaluations to help assess student contributions to group projects. Peer ratings were largely reliable as group members generally agreed on the scores given to their peers. In addition, most students differentiated among group member contributions through their ratings. Students also tended to rate themselves higher than their peers rated them. This study has implications for how peer and self evaluations can be most effectively used by educators to measure student performance in group work.