Evaluating tertiary teaching: A New Zealand perspective

The purpose of this research was to test the applicability of two American instruments designed to assess tertiary students' evaluations of teaching effectiveness with New Zealand students. The scales were found to have high internal consistency reliability coefficients, most of the items were seen to be appropriate, and every item was considered of importance by at least some of the students. In addition, all but the Workload/Difficulty items clearly differentiated between “good,” “average,” and “poor” lectures. Further analyses generally supported both the factor structure identified in earlier research and the convergent and discriminate validity of the scales from both instruments. This research has provided strong support for the applicability of these American instruments for evaluating effective teaching at a New Zealand university.     

ASSESSING TEACHING EFFECTIVENESS: AN INDIAN PERSPECTIVE

>An investigation is reported which tests the applicability of two American instruments designed to assess tertiary students’ evaluations of teaching effectiveness with 111 Indian graduate students. The scales were found to have generally high internal consistency reliability coefficients, most of the items were seen to be appropriate, and every item was considered of importance by at least some of the students. In addition, all but the Workload/Difficulty items clearly differentiated between ‘good’ and ‘poor’ lecturers. Further analysis generally supported the convergent and discriminant validity of those scales hypothesised to measure similar or dissimilar components of effective teaching. However, this analysis did indicate more overlap between aspects of teaching skill and enthusiasm than evident in Western studies. Factor analysis confirmed this finding as a strong main factor of teaching effectiveness plus minor factors referring to specific aspects of teaching were obtained.     

Students' Evaluations of University Teaching: A China Perspective

This study investigates the applicability of two American instruments, designed to assess students' evaluations of teaching effectiveness, to university students in China. Based on the responses of 367 Chinese undergraduates, the results indicated that the scales had high internal consistency reliability coefficients and the items were generally considered appropriate and clearly distinguished between good and poor lecturers. Moreover, every item was deemed important by at least some of the students. Further analysis lent support to the convergent and discriminant validities of the scales hypothesised to measure similar or dissimilar components of teaching effectiveness. However, this analysis and factor analytic findings did indicate more overlap among aspects of Learning/Value, Organisation/Clarity, Enthusiasm, and Breadth of Coverage, than that found in Western studies. Hence, this study only provided partial support for the cross‐cultural validity of the two instruments and the multidimensional model of teaching effectiveness on which they were based.     

Student Evaluations of Tertiary Teaching: a Nepalese investigation

An investigation is reported which tests the applicability of two American instruments designed to assess tertiary students’ evaluations of teaching effectiveness with 297 Nepalese graduate students. The scales were found to have generally adequate internal consistency reliability coefficients, most of the items were seen to be appropriate, and every item was considered of importance by at least some of the students. In addition, all but the Workload/Difficulty items clearly differentiated between ‘good’ and ‘poor’ lecturers. Further analysis failed to support the convergent and discriminant validity of three of the scales hypothesised to measure similar or dissimilar components of effective teaching. Factor analysis confirmed this finding as a strong main factor of teaching effectiveness plus minor factors referring to workload difficulty and grading were obtained. This research therefore indicated more overlap between aspects of teaching skill and enthusiasm than is evident in Western studies. Taken together with the results of similar investigations in developing countries, there must be doubt about the cross‐cultural validity of a multidimensional model of teaching effectiveness.     

大学生职业生涯辅导中引入教练技术的可行性分析

教练技术是一种简洁有效的辅导工具,将它引入高校大学生职业生涯辅导中,通过采用"奇迹式发问""生命平衡轮""换框"等实用性工具,不仅可以有效地帮助学生解决很多认识问题,改善学习、工作和生活模式,激发潜能并促进内在素质的全面提升,而且还可以带动职业生涯教学方法的改革、教学质量的提升。因此,大学生职业生涯辅导中引入教练技术,对于构建新型高校毕业生就业服务体系具有积极意义。     

Student evaluations of teaching effectiveness: a Nigerian investigation

An investigation is reported which tests the applicability of two American instruments designed to assess tertiary students' evaluations of teaching effectiveness with 158 Nigerian undergraduates. The scales were found to have generally high internal consistency reliability coefficients, most of the items were seen to be appropriate, and every item was considered of importance by at least some of the students. In addition, all but the Workload/Difficulty items clearly differentiated between good and poor lecturers. Factor analysis found a strong main factor of teaching effectiveness plus a minor factor referring to course workload and difficulty. Further analysis generally supported the convergent and discriminant validity of those scales hypothesized to measure similar or dissimilar components of effective teaching. However, this analysis supported the factor analytic results as more overlap between aspects of teaching skill and enthusiasm was found than has been evident in Western studies. Thus there must be doubt about the cross-cultural validity of a multidimensional model of teaching effectiveness.     

The acquisition of formal operational schemata during adolescence: The role of the biconditional

To test the hypothesis that the basic “logic” utilized by individuals in scientific hypothesis testing is the biconditional (if and only if), and that the biconditional is a precondition for the development of formal operations, a sample of 387 students in grades eight, ten, twelve, and college were administered eight reasoning items. Five of the items involved the formal operational schemata of probability, proportions and correlations. Two of the items involved propositions and correlations. Two of the items involved propositional logic. One item involved the biconditional. Percentages of correct responses on most of the items increased with age. A principal-component analysis revealed three factors, two of which were identified as involving operational thought, one of which involved propositional logic. As predicted, the biconditional reasoning item loaded on one of the operational thought factors. A Guttman scale analysis of the items failed to reveal a unidimensional scale, yet the biconditional reasoning item ordered first supporting the hypothesis that it is a precondition for formal operational reasoning. Implications for teaching science students how to test hypotheses are discussed.     

ITEM BEHAVIOR ASSOCIATED WITH CHANGES IN THE NUMBER OF ALTERNATIVES IN MULTIPLE CHOICE ITEMS1

Language reading examinations in French and Spanish were administered to students in order to compare the behavior of “natural” four-choice items with “natural” five-choice items rescored as four-choice items after removing the least popular incorrect alternative. No significant differences in the regression systems of these items were found. However, “natural” four-choice items were significantly less reliable than “natural” five-choice items.     

The development of a science process assessment for fourth-grade students

In this study, a multiple-choice test entitled the Science Process Assessment was developed to measure the science process skills of students in grade four. Based on the Recommended Science Competency Continuum for Grades K to 6 for Pennsylvania Schools, this instrument measured the skills of (1) observing, (2) classifying, (3) inferring, (4) predicting, (5) measuring, (6) communicating, (7) using space/time relations, (8) defining operationally, (9) formulating hypotheses, (10) experimenting, (11) recognizing variables, (12) interpreting data, and (13) formulating models. To prepare the instrument, classroom teachers and science educators were invited to participate in two science education workshops designed to develop an item bank of test questions applicable to measuring process skill learning. Participants formed “writing teams” and generated 65 test items representing the 13 process skills. After a comprehensive group critique of each item, 61 items were identified for inclusion into the Science Process Assessment item bank. To establish content validity, the item bank was submitted to a select panel of science educators for the purpose of judging item acceptability. This analysis yielded 55 acceptable test items and produced the Science Process Assessment, Pilot 1. Pilot 1 was administered to 184 fourth-grade students. Students were given a copy of the test booklet; teachers read each test aloud to the students. Upon completion of this first administration, data from the item analysis yielded a reliability coefficient of 0.73. Subsequently, 40 test items were identified for the Science Process Assessment, Pilot 2. Using the test-retest method, the Science Process Assessment, Pilot 2 (Test 1 and Test 2) was administered to 113 fourth-grade students. Reliability coefficients of 0.80 and 0.82, respectively, were ascertained. The correlation between Test 1 and Test 2 was 0.77. The results of this study indicate that (1) the Science Process Assessment, Pilot 2, is a valid and reliable instrument applicable to measuring the science process skills of students in grade four, (2) using educational workshops as a means of developing item banks of test questions is viable and productive in the test development process, and (3) involving classroom teachers and science educators in the test development process is educationally efficient and effective.     

Achievement as a Function of Test Item Complexity and Difficulty

The effects of training tests on subsequent achievement were studied using 2-test item characteristics: item difficulty and item complexity. Ninety Ss were randomly assigned to treatment conditions having easy or difficult items and calling for rote or complex skills. Each S was administered two training tests during the quarter containing only items defined by his treatment condition. The dependent measure was a sixty item final examination with fifteen items reflecting each of the four treatment condition item types. The results showed greater achievement for those trained with difficult items and with rote items. In addition, two interaction of treatment conditions with type of test items were found. The results are discussed as supporting a hierarchical model rather than a “similarity” transfer model of learning.     

Does difficulty-based item order matter in multiple-choice exams? (Empirical evidence from university students)

This empirical study aimed to investigate the impact of easy first vs. hard first ordering of the same items in a paper and-pencil multiple-choice exam on the performances of low, moderate, and high achiever examinees, as well as on the item statistics. Data were collected from 554 Turkish university students using two test forms, which included the same multiple-choice items ordered reversely, i.e. easy first vs. hard first. Tests included 26 multiple-choice items about the introductory unit of “Measurement and Assessment” course. The results suggested that sequencing the multiple-choice items in either direction from easy to hard or vice versa did not affect the test performances of the examinees no matter whether they are low, moderate or high achiever examinees. Finally, no statistically significant difference was observed between item statistics of both forms, i.e. the difficulty (p), discrimination (d), point biserial (r), and adjusted point biserial (adj. r) coefficients.     

大学数学小班化教学改革实施的实证研究--以四川大学为例

通过实证研究、调查研究及比较研究法,对2011年以来四川大学大学数学课程教学班级、教师及学生人数、学生成绩等相关数据进行分析,发现教学班级数量不断增加,班级人数不断减少,但是距离30人的小班教学还有较大差距;大班、小班中学生成绩受课程及其难度的影响较大;师生对小班化教学的期望存在差异,“大班授课,小班研讨”的实施出现异化。建议正确认识适合中国国情、适合大学数学课程特点的小班化教学;加强高校教师教育,促进高校教师教学理念、教学方式方法转变;新进适量的教师,鼓励大学数学在职教师多上课,鼓励教授为本科生上课;有效推行“大班授课,小班研讨”,不断提升助教教学和管理能力。     

Multiple audience rating form strategies for student evaluation of college teaching

Michael Scriven has suggested that student rating forms, for the purpose of evaluating college teaching, be designed for multiple audiences (instructor, administrator, student), and with a single global item for summative functions (determination of merit, retention, or promotion). This study reviewed approaches to rating form construction, e.g., factor analytic strategies of Marsh, and recommended the multiple audience design of Scriven. An empirical test of the representativeness of the single global item was reported from an analysis of 1,378 forms collected in a university department of education. The global item correlated most satisfactorily with other items, a computed total of items, items that represented underlying factors, and various triplets of items selected to represent all possible combinations of items. It was concluded that a multiple audience rating form showed distinct advantages in design and that the single global item most fairly and highly represented the overall teaching performance, as judged by students, for decisions about retention, promotion, and merit made by administrators.     

Same,Similar, or Something Completely Different? Calibrating Student Surveys and Classroom Observations of Teaching Quality Onto a Common Metric

Using item response theory, this study explores whether student survey and classroom observation items can be calibrated onto a common metric of teaching quality. The data comprises 269 lessons of 141 teachers that were scored on the International Comparative Analysis of Learning and Teaching (ICALT) observation instrument and the My Teacher student survey. Using Rasch model concurrent calibration, items from both instruments were calibrated onto a common one‐dimensional metric of teaching quality. Most items were found to fit the model. Challenges pertain mainly to items measuring teaching students learning strategies and differentiation. Explanations for these difficulties are discussed.     

Educational psychology at the university of Wisconsin

A 44 item questionnaire and free response instrument was given to 280 students selected from two different kinds of introductory educational psychology courses: (a) A “module”; approach involving selection of 3 five‐week modules from a list of such courses; and (b) A specially prepared composite learning and development course covering 1 semester. Analysis of the results lead to the following conclusions: (1) “In theory”; students preferred the module approach; (2) “In practice”; students preferred the composite 1 semester course; (3) The outstanding variable contributing to students’ assessment of relevancy, and teaching procedures, was the instructor. The implication of these findings for innovations in teaching students is discussed.     

Student and Faculty Perceptions of an Effective University Instructional Climate

Abstract

This study sought to ascertain perceptions of an effective instructional climate through use of a seventy-one item questionnaire. Usable responses were received from 1,237 students and 268 faculty members. Statistical analysis yielded a list of forty “significant” attributes grouped into seven clusters. Factor loading values were derived for the attributes to facilitate utilization in describing and assessing instructional climate. Students and faculty agreed generally on a global concept of instructional climate and as to the “significance” of many individual attributes. Greater “significance” was attached to instructors and their teaching role than to students and their learning role. Practical uses of the results are suggested.     

Construction and Initial Validation of the Chinese Reading Motivation Questionnaire

This paper describes the development and initial validation of a Chinese reading motivation questionnaire (CRMQ) which was designed to assess Chinese students' reading motivation in Hong Kong. The development of the CRMQ consisted of two steps. In the first study, groups of items were constructed based on achievement motivation theories and existing instruments. The initial version of the CRMQ was administered to a small group of Grade 7 students in Hong Kong and exploratory factor analysis, item‐total correlation, and reliability analyses were carried out to assess its psychometric quality. The CRMQ was then revised and administered to a larger sample of Grade 7 students in the second study. Findings of confirmatory factor analysis provided further validation for the proposed factor structure of the revised questionnaire. In addition, findings indicated that students' reading motivation was positively related to their strategy use, reading comprehension, and academic achievement. The findings of this study could enrich our understanding of Chinese reading motivation among Hong Kong students. Implications of these findings for implementing effective reading instruction in Chinese language teaching are also discussed.     

“加涅教学设计九步曲”视域下的大学体验英语教学设计案例

本文尝试在"加涅教学设计九步曲"视域下,结合《大学体验英语》(第三版)第三册第二单元Nobel Prize Winners的教学内容,将美国知名学者加涅的"教学九步曲"与大学英语课堂教学的实践活动相结合。从引起学生的注意、激发与学习相关的记忆开始,向学生告知学习的目标,呈现与学习相关的内容,向学生提供按部就班的指导,引发能够达到学习目标的练习,并针对练习的表现进行反馈,评估学习成果,并使学生最终能够举一反三。     

Research and Teaching in the Science Department of the University of London Institute of Education

This study was designed (1) to analyse the relationship between the answer profile from multiple‐choice questions on stoichiometric problems and the students’ reasoning patterns and (2) to examine the effect for certain variables on the facility values of test items. The instruments used were mainly paper‐and‐pencil tests. The subjects were 6262 grammar school students from all parts of the Federal Republic of Germany. They were randomly assigned to the test items.

The results indicated that many students arrived at their answers by mixing up amount and reacting mass, or molar mass and reacting mass. It was also found that the variables ‘easy/hard calculations’, and ‘formula given/to be developed’ determined the facility values of test items.

From the results, it was possible to make recommendations to practising teachers as well as to examiners. Knowing students’ ideas, the teacher can think of how to make use of them before entering the classroom. A teaching unit may start off with easy problems leaving the more difficult ones for later. Examiners developing new tests on stoichiometry should consider two essential preconditions for the formulae of chemical compounds, used in the item: formulae of the type AB should be avoided and the molar masses of the elements involved must be clearly different.