A Closer Look at Using Judgments of Item Difficulty to Change Answers on Computerized Adaptive Tests

Recent studies have shown that restricting review and answer change opportunities on computerized adaptive tests (CATs) to items within successive blocks reduces time spent in review, satisfies most examinees' desires for review, and controls against distortion in proficiency estimates resulting from intentional incorrect answering of items prior to review. However, restricting review opportunities on CATs may not prevent examinees from artificially raising proficiency estimates by using judgments of item difficulty to signal when to change previous answers. We evaluated six strategies for using item difficulty judgments to change answers on CATs and compared the results to those from examinees reviewing and changing answers in the usual manner. The strategy conditions varied in terms of when examinees were prompted to consider changing answers and in the information provided about the consistency of the item selection algorithm. We found that examinees fared best on average when they reviewed and changed answers in the usual manner. The best gaming strategy was one in which the examinees knew something about the consistency of the item selection algorithm and were prompted to change responses only when they were unsure about answer correctness and sure about their item difficulty judgments. However, even this strategy did not produce a mean gain in proficiency estimates.     

Reasons for Changing Answers: An Evaluation Using Personal Interviews

Researchers investigating answer changing have consistently found the preponderance of changes on objective items to be from wrong to right, but little is understood about the mechanisms involved in this phenomenon. In this study, personal interviews were combined with instruction in answer-changing research to investigate further the processes involved in answer changing. Students changed answers and gained from changing, with those in the upper two thirds of the classes gaining the most. Each test-taking strategy produced a mean gain, but particular strategies were not significantly correlated with percentage of gain or percentage of change. Most students reported changing answers for thoughtful reasons such as rereading, rethinking, or remembering more information; very few changes were due to clerical errors. For each reason, most changes were wrong-to-right. We conclude that reconsideration of test items is probably underestimated in answer-changing studies. The role of memory should be considered in why people change and in how successful they judge their changing to have been.     

Prospective Primary Teachers' Understanding of Climate Change, Greenhouse Effect, and Ozone Layer Depletion

Climate change is one of the most serious global environmental problems and for that reason there has been lately a great interest in educating pupils, the future citizens, about it. Previous research has shown that pupils of all ages and teachers hold many misconceptions and misunderstandings concerning this issue. This paper reports on research concerning student teachers' perceptions about aspects of climate change as well as about greenhouse effect and ozone layer depletion. The aim of this research is to take findings into account for teaching student teachers about these issues. An open-ended questionnaire was used in order to gain a more comprehensive understanding of their thought. From their answers it appeared that these students believe that climate change is under way and base their beliefs on their own experience. They are unaware of the proper actions to be taken for slowing down the climate change, they also hold the misconception that ozone depletion, acid rain, and pollution in general are conducive to climate change. They confuse greenhouse effect with ozone depletion as far as the mechanisms through which they occur is concerned and the causal compounds. By taking into account these research findings the possible implications for teaching are discussed and some suggestions for more effective teaching are made.     

The Consistency of Detecting Item Bias Across Different Test Administrations: Implications of Another Failure

Several item bias detection methods were applied to the analysis of bias among males and females for items from a curriculum-based mathematics test. The focus of this analysis was the consistency of the methods across different test administrations of the same items. The results indicated that, of the methods studied, the Mantel–Haenszel (M–H) and IRT-based sum-of-squares methods were the most consistent. However, the degree of reliability and agreement for these methods was modest at best. As with most prior research, no reasonable explanation could be found for the most consistently flagged items. A likely reason for this lies in the confusion of visible genetic group characteristics with their instructional backgrounds. A multidimensional perspective of item bias is proposed for future research that will take such confounding into account.     

Teachers’ and students’ perceptions of assessments: A review and a study into the ability and accuracy of estimating the difficulty levels of assessment items

In today's higher education, high quality assessments play an important role. Little is known, however, about the degree to which assessments are correctly aimed at the students’ levels of competence in relation to the defined learning goals. This article reviews previous research into teachers’ and students’ perceptions of item difficulty. It focuses on the item difficulty of assessments and students’ and teachers’ abilities to estimate item difficulty correctly. The review indicates that teachers tend to overestimate the difficulty of easy items and underestimate the difficulty of difficult items. Students seem to be better estimators of item difficulty. The accuracy of the estimates can be improved by: the information the estimators or teachers have about the target group and their earlier assessment results; defining the target group before the estimation process; the possibility of having discussions about the defined target group of students and their corresponding standards during the estimation process; and by the amount of training in item construction and estimating. In the subsequent study, the ability and accuracy of teachers and students to estimate the difficulty levels of assessment items was examined. In higher education, results show that teachers are able to estimate the difficulty levels correctly for only a small proportion of the assessment items. They overestimate the difficulty level of most of the assessment items. Students, on the other hand, underestimate their own performances. In addition, the relationships between the students’ perceptions of the difficulty levels of the assessment items and their performances on the assessments were investigated. Results provide evidence that the students who performed best on the assessments underestimated their performances the most. Several explanations are discussed and suggestions for additional research are offered.     

微型化学实验的开发

微型化学实验的开发和开展是在绿色化学理念指导下的实验创新,是对研究性学习的探索和发展,使教师由知识的传播者转化为知识的创建者和研究者,使学生由单纯记住知识转化为使用知识和理解知识,让化学实验由“照方抓药”变为“开方抓药”,发展学生创造、思维、培养其决策的能力。     

Action learning in virtual higher education: applying leadership theory

This paper reports the historical foundation of Northeastern University’s course, LDR 6100: Developing Your Leadership Capability, a partial literature review of action learning (AL) and virtual action learning (VAL), a course methodology of LDR 6100 requiring students to apply leadership perspectives using VAL as instructed by the author, questionnaire and survey results of students who evaluated the effectiveness of their application of leadership theories using VAL and insights believed to have been gained by the author administering VAL. Findings indicate most students thought applying leadership perspectives using AL was better than considering leadership perspectives not using AL. In addition as implemented in LDR 6100, more students evaluated VAL positively than did those who assessed VAL negatively.     

The influence of instruction,prior knowledge,and values on climate change risk perception among undergraduates

We evaluated influences on the climate change risk perceptions of undergraduate students in an introductory Earth Science course. For this sample, domain‐specific content knowledge about climate change was a significant predictor of students' risk perception of climate change while cultural worldviews (individualism, hierarchy) and political orientation were not. These results contrast with previous studies highlighting worldviews as a dominant influence on risk perception. At the beginning of the semester, students' climate change content knowledge was relatively low, with average scores on a 21‐item test less than 50%. Post instruction results indicated that students learned climate change science during the course, and their perceptions of risks associated with climate change increased. Unlike most prior research evaluating links between climate change knowledge and risk perception, our measure of content knowledge was a validated assessment specific to climate change. Use of this specific climate knowledge test may be one reason that we detected a relationship between climate knowledge and risk perception whereas most of the previous research has not. Another—possibly complementary—explanation may be a generational shift between our study sample and prior samples. Undergraduates today, having grown up with more exposure to climate change in schools and the media than previous generations, may be diverging from average adults in that learning climate science appears to also increase their perceptions of the risks climate change poses. Undergraduate courses with embedded climate‐related activities present an opportunity to both increase climate science knowledge and risk perceptions of future decision makers.     

Ethics, deafness, and new medical technologies

In the last 50 years, several new technologies have become enormously important within the Deaf community and have helped significantly to improve deaf people's lives in a hearing world. Current public attention and admiration, however, seems unduly focused on medical technologies that promise to solve "the problem" of being deaf. One reason for this interest, we argue, is the public's preoccupation with deafness as a disability and promises that technological breakthroughs such as the cochlear implant will "cure" deafness. Pressure on parents to make quick and early decisions and lack of adequate information about alternatives often leave them unprepared for the consequences of these decisions. To allow deaf individuals and their families to make better informed decisions about their lives and their futures, we argue finally that professionals who interact with these families adopt inclusive and individualizing ethics.     

"说课"研究述评

"说课"的产生和发展,有力地促进了教育教学理论的发展和教师素质的提高."说课"有其自身的内涵、特征、发展历史、研究得失和发展趋向.     

项目教学法在电大计算机网络课程中的应用

传统的计算机网络课程教学以教师在课堂上教,学生听的教学模式已经不能适应实用型计算机人才培养的需要。在教学中引入项目教学法,可以充分调动学生的学习主动性,从而使学生达到学以致用的目的。文章结合计算机网络课程的特点,探索了项目教学法在电大计算机网络课程教学中的应用。     

缘事而发因事说事——试运用系统功能理论解析公文正文要素及格式

统观《办法》和《条例》里所有18种公文,公文正文的中心内容都是“事”,即事件、事实、事项、事理。也就是“要读者知道某事或做某事”,因此公文正文的主要功能是知行功能,这个知行功能的中心就是“事”,具体地说,也就是知行功能中的行事功能。公文是“缘事而发”,“事”是公文正文必不可少的一个要素。公文是行事文体,它告诉读者的主要是“是什么”、“怎么做”,因事“说”事,而不就事“论”事、感情用“事”。     

The effect of knowledge of correct results per item on verbal learning and retention

A report of an experiment to measure the effects of both overt and covert responses to educational stimuli with KCR (knowledge of correct results) feedback provided. In most studies of feedback, KCR (or KR) is given as a consequence of an overt response: Stimulus-Response-Feedback. In the case of a covert response, the student constructs his answer but does not react overtly to the presentation of a stimulus item. Forty Ss were randomly assigned to 1 of 4 groups further divided into 1 control group receiving no feedback and 3 groups receiving information (KCR) according to either overt or covert response patterns. A 25-item multiple-choice test was administered 5 times by means of a terminal consisting of a screen and a teletype. Results showed that the covert paradigm was the most effective. The experiment did not demonstrate that the covert process leads to better performance on a test administered later to measure retention. A tentative explanation for this negative result was that the time interval between the provision of feedback and presentation of the next item was of some relevance. A multiple comparison test revealed that the 3 treatment groups performed better than the control group.     

对广东省普通高校体育教师科研现状的调查与研究

采用专家访谈法和问卷调查法,对广东省普通高校体育教师的科研现状、开展科研最常见的困难等方面进行分析 结果表明,自主选题,努力进行科研,发表成果已成为广东省普通高校体育教师科研的主流;结合教学进行科研已成为广东省普通高校体育教师科研的热点 但存在薄弱环节,表现在:(1)科研方向不清,难于入手;(2)繁重的教学和训练,使教师把教学和训练任务放在第一位,科研放在了次要的位置上,造成时间少,精力不够;(3)理论根基薄弱,高层次的成果不多     

Lakatos’ Scientific Research Programmes as a Framework for Analysing Informal Argumentation about Socio‐scientific Issues

The purpose of this study is to explore how Lakatos’ scientific research programmes might serve as a theoretical framework for representing and evaluating informal argumentation about socio‐scientific issues. Seventy undergraduate science and non‐science majors were asked to make written arguments about four socio‐scientific issues. Our analysis showed that the science majors’ informal arguments were significantly better than the non‐science majors’ arguments. In terms of the resources for supporting reasons, we find that personal experience and scientific belief are the two categories that are generated most often in both groups of the participants. Besides, science majors made significantly greater use of analogies, while non‐science majors made significantly greater use of authority. In addition, both science majors and non‐science majors had a harder time changing their arguments after participating in a group discussion. In the study of argumentation in science, scholars have often used Toulmin’s framework of data, warrant, backing, qualifiers, claims, and rebuttal. Our work demonstrates that Lakatos’ work is also a viable perspective, especially when warrant and backing are difficult to discern, and when students’ arguments are resistant to change. Our use of Lakatos’ framework highlights how the ‘hard core’ of students’ arguments about socio‐scientific issues does, indeed, seem to be protected by a ‘protective belt’ and, thus, is difficult to alter. From these insights, we make specific implications for further research and teaching.     

Effect of encoding instructions on children's spatial and color memory: is there evidence for automaticity?

This experiment is concerned with assessing whether there is evidence for the automatic processing of spatial and color information by children and with the development of control of encoding operations. First, third, and fifth graders viewed pictures of simple objects that varied in color (red or green) and spatial location (left or right). Subjects were instructed to encode (a) only the item, (b) the item and its color, (c) the item and its position, (d) the item, its color, and its position. The data yielded evidence for nonchance incidental memory of spatial, but not color, information. The data also suggested that children at all age levels were able to control their encoding operations similarly, as there was no evidence for an interaction of development with encoding instructions. These findings are discussed with respect to a model for automatic encoding processes and work on memory for central and incidental information.     

项目反应理论测验信度及其研究述评

项目反应理论下的测验信度能够评价潜在特质估计的可靠性与稳定性,由于具有宏观性的特点,项目反应理论信度的作用并不能被测验信息函数所取代,是IRT测验的一个重要指标。本文参考国内外文献,首先介绍国内外学者关于IRT信度作用的观点,并介绍和评价了多种IRT信度估计方法,然后简要介绍IRT信度的影响因素,最后展望了IRT信度领域后续研究尚可着力之处。     

Comparative learning and retention of conventional and instructional TV methods

Summary Each of the 20 elementary schools in the Anaheim City School District were randomly assigned to one of the three method groups. The present studies included the entire group of fourth and fifth grade pupils who received instruction either conventionally or with one of the two ITV methods. The two studies compared pupils’ achievement and retention by method group, intelligence, and sex. Groups receiving the televised social studies instruction performed significantly better on the end-of-term examinations than did the control groups. On the one and two-year delayed posttests, contrary to expectation, all groups had significantly higher means than on the immediate posttests. The basic explanation for the phenomenon was thought to be maturation since many items appear to have tapped reasoning ability as well as content knowledge. This interpretation was supported by the fact that greater gain was noted on the two-year delayed posttest than on the one-year detention measure. The hypothesis was further strengthened by the analysis by item type, the factual items showing the least increase over the time interval. No differential change or gain existed for method groups for the oneyear study. In the two-year study the control group showed greater gain, but its mean remained below that of the TV group. Both authors have served as research consultants for the Anaheim Instructional Television Project for the past five years. They are presently writing a monograph presenting the results from their five years of research with this project.     

What Can Research Tell us about Teacher Education?

If it is true that educational research should not merely offer us better understandings of educational processes, but also offer us suggestions for good practice by ascertaining 'what works' most effectively in diverse contexts, then there are potentially four ways in which such research could be of use to those engaged in teacher education. This paper explores three of them-those which it argues are of most use to teacher educators. From research into the processes of teaching and learning we have learned a great deal about the complex nature of teachers' expertise and the extent to which it is determined by highly specific contexts. In seeking to understand more about the processes involved in learning to teach we have come to recognise the strength of the personal agendas that beginning teachers bring with them and the active role they play in negotiating their own learning. On this basis research offers teacher educators a number of suggestions for good practice: about the ways in which we should expect beginning teachers to make use of research-based generalisations; about how we can help them gain access to the wealth of expertise embedded in the practice of experienced teachers; and about how we can encourage them to subject their own preconceptions to serious critical evaluation.     

Score Gains, Attitudes, and Behavior Changes Due to Answer-Changing Instruction

Six undergraduate and three graduate classes were given multiple-choice tests with subsequent evaluation of answer changes. The 300 students were tested twice, once before and once after instruction on answer changing. After each test, students were asked to complete two forms. The forms evaluated attitude toward answer changing, reasons for changing, and confidence in final answers. Students showed a significant increase in favorability toward answer changing after instruction. No significant change was found in number of answers changed. Psychology students were found to change significantly more items than were business students. Mean gain score did not change significantly after instruction. It was concluded that although instruction does lead to a change in attitude in answer changing, the number of changes and overall gain due to answer changing do not change. It was also determined that students continue to make significant gains even when their confidence in the final answer is less than 50 on a 100-point scale.