A Tribute to Robert L. Ebel: Scholar, Teacher, Mentor, and Statesman

The authors describe the significant contributions of Robert Ebel to educational measurement theory and its applications. A biographical sketch details Ebel's roots and professional résumé. His influence on classroom assessment views and procedures are explored. Classic publications associated with validity, reliability, and score interpretation are described in terms of their contributions to the field and their relevance to current practice. An analysis of the diversity of Ebel's contributions over his career projects a model of "psychometric statesman" for contemporary measurement specialists to emulate.     

Course characteristics and college students' ratings of their teachers: What we know and what we don't

From showing in a general way that there is room for course context to influence class (average) ratings of instruction, this review proceeds to a search for specific course characteristics that are associated with these ratings. Extant research has centered around five such characteristics: class size, course level, the electivity of the course, the particular subject matter of the course, and the time of day that the course is held. Although statistically significant zero-order relationships do not appear in every piece of research located for review, such relationships are more likely to be found than not for the first four of these characteristics. The associations may not be particularly strong, but rather clear-cut patterns do emerge. Of the studies reporting an association between size of class and class ratings, most find it to be inverse, although several studies show a curvilinear (U-shaped) relationship. Teacher (and course) ratings tend to be somewhat higher for upper division courses and elective courses. Compared to other instructors, those teaching humanities, fine arts, and languages tend to receive somewhat higher ratings. The possible reasons for these relationships are many and complex. A precise understanding of the contribution of course characteristics to the ratings of teachers (and the courses themselves) is hampered by two circumstances. Studies in which relevant variables are controlled are far fewer in number than are the studies in which only the zero-order relationships between course characteristics and ratings are considered. More importantly, existing multivariate studies tend to underplay or ignore the exact place of course characteristics in a causal network of variables.     

Student ratings of instruction: Validity and normative interpretations

Using 3,355 class section means, the relationship between six predictor variables and student ratings of instruction (CEQ) was investigated by computing the intercorrelation matrix among all variables and by performing several regression analyses. Results of the study indicated that all linear interactions were negligible and that more than one-fourth of the criterion variance was shared with all the predictor variables. Two predictor variables, expected grade and required-elective, provided extremely large contributions to the prediction of the criterion measure, however. Implications of the validity results with respect to normative data and thus to the administrative use of the ratings were illustrated.     

Different administrative directions and student ratings of instruction: Cognitive versus affective effects

A manipulation of the instructions students received prior to completing the 7-item Endeavor Instructional Rating card differentially affected their ratings on two types of items. Specifically, when students were led to believe their ratings would have a strong impact on the instructor's career, they tended to be more lenient on items measuring rapport (i.e., the affective domain); this same effect was not observed for items measuring pedagogical skill (i.e., the cognitive domain). The different items on our instructional rating instrument appear to be measuring different things. One implication of this observation is that the inconsistent findings reported in past research on student ratings of instruction may be due to the differential mix of items from one instrument to another. When instructors are compared on ratings given them by students, unbiased interpretation requires that the multidimensional nature of teaching (and of the rating instrument) be considered.     

The significance of circumstances for college students' ratings of their teachers and courses

Although firm generalizations and conclusions cannot yet be drawn from the extant research on the effects on teacher and course ratings of the circumstances surrounding these evaluations, at least some studies have shown that college students' ratings of their teachers and courses are somewhat higher when students remain anonymous rather than identifying themselves, when the purported use of the ratings is an official or administrative one for use in salary, promotion, or tenure considerations rather than otherwise, and when the instructor is present rather than absent during the rating session. (The differences between each of these contrasted circumstances are usually rather small and do not inevitably appear across studies.) Certain variations in rating format have been found to make a difference in the ratings obtained, whereas other have not. From limited evidence, the exact timing (or occasion) of evaluation appears not to be important to ratings. Variability in sampling procedures, as it affects the composition of students available to complete rating forms, may or may not turn out to be a generally important element in ratings (as directly relevant data are collected). The analysis concludes with a discussion of (1) the presumed bias in ratings produced in certain of the rating conditions and (2) the more general issue of the comparability of ratings made in different circumstances of evaluation.     

Professors and examinations: ideas of the university in nineteenth-century Scotland

The separation of examining from teaching, pushed furthest in the ‘examining university’ of which London University, founded in 1836, was the model, was a much-debated principle in nineteenth-century Britain. This separation was generally rejected in Scotland, but only after complex controversies that illustrate how Scots defined their university tradition in comparative terms, and how Scottish developments interacted with those in England and Ireland. Among the issues involved were proposals for a National University or central examining board, and claims that graduates should have a right to give ‘extramural’ teaching in competition with professors. The paper traces this aspect of university reform in Scotland from the 1820s to the 1890s, and argues that the professorial model and the integration of teaching and examining were successfully consolidated and defended.     

Investigating substantive and consequential validity of student ratings of instruction

This study aimed at exploring faculty and student perspectives on student evaluations, as well as identifying their perceptions of the usefulness and appropriateness of the ratings for evaluating teaching effectiveness. More specifically, the study aimed at identifying the consequences, both intended and unintended, of using the evaluations, in addition to better understanding the process students used in responding to evaluations and what use faculty members made of them. Two surveys were developed and placed on the website of the Office of Institutional Research and Assessment. Emails were sent to all students and faculty participating in evaluations soliciting their cooperation and requesting their input. Faculty and student perceptions were compared qualitatively and statistically. Results revealed that students and faculty believe in the effectiveness and usefulness of the system with the need to overcome some negative consequences and biases inherent in its application at the University. Recommendations for system improvement are provided.     

Assessing the impact of learning environments: How to use student ratings of classroom or school characteristics in multilevel modeling

In educational research, characteristics of the learning environment (e.g., social climate, instructional quality, goal orientation) are often assessed via student reports, and their associations with outcome variables such as school achievement or student motivation then tested. However, studying the effects of the learning environment presents a series of methodological challenges. This article discusses three crucial elements in research that uses student reports to gauge the impact of the learning environment on student outcomes. First, from a conceptual point of view, it is argued that ratings aggregated at the relevant level (e.g., class or school level), and not individual student ratings, are of primary interest in these studies. Second, the reliability of aggregated student ratings must be routinely assessed before these perceptions are related to outcome variables. Third, researchers conducting multilevel analyses need to make very clear which centering option was chosen for the predictor variables. This article shows that conclusions about the impact of learning environments can be substantially affected by the choice of a specific centering option for the individual student ratings.     

考试的功能及其对提高教学质量的影响

高等学校的考试具有引导功能，是提高教学质量的“指挥棒”；具有激励功能，是提高教学质量的“源动力”；具有检洲功能，是提高教学质量的“试金石”；具有评价功能，是提高教学质量的“催化剂”；具有诊断功能，是提高教学质量的“听诊器”。     

Student ratings of college teaching: a comparison of faculty and their students

Based on the review of student ratings myths by Aleamoni (1987, 1999), a survey research design was used to analyse differences between college students' (n = 968) and faculty's (n = 34) perceptions. Generally, students held stronger beliefs in these myths, in that they believed faculty with excellent publication records were better qualified to evaluate teaching and that student ratings on single general items are accurate measures of teaching effectiveness. On the other hand, faculty believed that student ratings were invalid and unreliable. Further examination of student characteristics revealed that male students held stronger beliefs in these myths. Finally, students' beliefs in these myths were correlated with their actual ratings of nine dimensions of the Student Evaluation of Educational Quality. A discussion as well as suggestions for using student ratings is provided.     

A path analysis of factors often found to be related to student ratings of teaching effectiveness

The relationships among several variables outside of the instructor's classroom control and student ratings of teaching effectiveness are investigated in a causal network. Student ratings are relatively independent of external variables. Students may be able to take into account more factors than generally assumed when they rate their instructors.     

Relationships between psychometric dimensions of item quality and student ratings of item relevancy and ambiguity

Students rated the quality of the items on a classroom test that had been taken previously. On the same test, psychometric item indices were calculated. The results showed that the student ratings were related to the item difficulty, but not to the item-test correlation. In addition, the better-achieving students tended to rate the items as less ambiguous. Finally, the ambiguity ratings were more highly related to the item-test correlations for the better achieving students. These findings support opinions held by many instructors of students' judgments of item quality.     

Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes

The contribution examines theoretical foundations, factorial structure, and predictive power of student ratings of teaching quality. Three basic dimensions of teaching quality have previously been described: classroom management, cognitive activation, and supportive climate. However, student ratings, especially those provided by primary school students, have been criticised for being biased by factors such as teacher popularity. The present study examines ratings of teaching quality and science learning among third graders. Results of multilevel confirmatory factor analyses (N = 1556 students, 89 classes) indicate that the three-dimensional model of teaching quality can be replicated in ratings of third graders. In a longitudinal study (N = 1070 students, 54 classes), we found ratings of classroom management to predict student achievement, and ratings of cognitive activation and supportive climate to predict students' development of subject-related interest after teacher popularity is controlled for. The analyses show that student ratings can be useful measures of teaching quality in primary school.     

Interpreting and using student ratings data: Guidance for faculty serving as administrators and on evaluation committees

This article is about the accurate interpretation of student ratings data and the appropriate use of that data to evaluate faculty. Its aim is to make recommendations for use and interpretation based on more than 80 years of student ratings research. As more colleges and universities use student ratings data to guide personnel decisions, it is critical that administrators and faculty evaluators have access to research-based information about their use and interpretation.The article begins with an overview of common views and misconceptions about student ratings, followed by clarification of what student ratings are and are not. Next are two sections that provide advice for two audiences—administrators and faculty evaluators—to help them accurately, responsibly, and appropriately use and interpret student ratings data. A list of administrator questions is followed by a list of advice for faculty responsible for evaluating other faculty members’ records.     

Teacher effectiveness and teacher growth from student ratings: An action research of school-based teacher evaluation

Teacher effectiveness evaluations are considered an effective mechanism for promoting teacher growth. We implemented teacher effectiveness evaluations in three Chinese secondary schools over two years and (a) developed an action research framework for school-based teacher evaluations composed of five links, i.e., purpose and consensus, data collection and analysis, data interpretation and feedback, data use, and evaluation of the actions and (b) found that “school-based” instruments of student ratings have valid statistical properties, and teacher effectiveness can be regarded as a single-factor holistic concept in different schools. We also (c) analysed 2-year longitudinal data and identified changing trajectories of teacher effectiveness as an example for data feedback and use. Teacher effectiveness presents changing trajectories for different subgroups, representing different ecologies in three schools and provides insights for teacher growth. A (d) meta-evaluation of the actions shows that most teachers are positive about this evaluation, although differences are observed between schools.     

模拟集成电路频率特性自测试技术研究

针对模拟集成电路参数多、性能易受环境条件影响、测试困难的问题,提出了一种利用脉冲测试技术测试模拟集成电路动态性能的脉宽测试方法。脉宽测试法以最为接近二位数字信号的脉冲方波作为激励,通过分析被测电路输出信号的频率响应来判断电路性能,为实现模拟集成电路的自测试提供技术基础。实验以集成滤波芯片MAX292构成的核心电路为研究对象,分别以不同参数的脉冲作为激励,测试其频带宽度,与传统点频法的测试结果基本一致,测量误差在5%以内。     

Reliability and agreement of student ratings of the classroom environment: A reanalysis of TIMSS data

In educational research, characteristics of the learning environment are generally assessed by asking students to evaluate features of their lessons. The student ratings produced by this simple and efficient research strategy can be analysed from two different perspectives. At the individual level, they represent the individual student’s perception of the learning environment. Scores aggregated to the classroom level reflect perceptions of the shared learning environment, corrected for individual idiosyncrasies. This second approach is often pursued in studies on teaching quality and effectiveness, where student-level ratings are aggregated to the class level to obtain general information about the learning environment. Although this strategy is widely applied in educational research, neither the reliability of aggregated student ratings nor the within-group agreement between the students in a class has been subject to much investigation. The present study introduces and discusses different procedures that have been proposed in the field of organisational psychology to assess the reliability and agreement of students’ ratings of their instruction. The application of the proposed indexes is demonstrated by a reanalysis of student ratings of mathematics instruction obtained in the Third International Mathematics and Science Study (N = 2,064 students in 100 classes).

新世纪自学考试存在的主要问题及其补正

当前,自考存在着教育功利主义及应试教育倾向,存在着专业课程设置僵化、考试评价方式和自学媒体单一等弊端。新世纪,必须进一步改革和完善自考制度,明确教育培养目标,克服自考的应试教育倾向,改革考试内容,建立多元化评价体系,实行开放的课程设置体系,加强对自学媒体的建设,以促进考生综合素质的提升和全面发展。     

江流天地外山色有无中--2005年全国中考语文考题回顾与2006年命题预测

近年来，中学语文教育改革的力度非常大，而与之相适应的中考命题也在变与不变的交替中思辨着、前进着。笔者在此从考试内容的继承与发展、考试形式的创新与求活、考查目的工具性与人文性三个方面，结合有代表性的试题对2005年全国各省区语文中考做一评析，并试图从中显现出2006年中考命题的趋势，为指导中考复习的老师们提供一些参考意见。