Formative evaluation of instructional development

Summary In this paper we have tried to describe the research pertinent to the activity of formative evaluation. While the number of good examples of formative evaluation is expanding, the level of research into the process is relatively limited. Perhaps a compromise to the difficult task of accumulating research data on formative evaluation might be suggested. When formative evaluation activities have been successful in terms of program effects and staff satisfaction, then detailed technical reports might be made available to the evaluation public. Formative evaluation might improve as a consequence of the technology developed in the course of finding solutions to development data problems. Both the increase in technology as a result of dissemination of successful procedures and the pursuit of experimental research on the process of evaluation might ultimately remove formative evaluation recommendations from the realm of seers and clairvoyants and thrust it closer to, if not into, the domain of scientific application. This article is the second ERIC/AVCR Annual Review Paper, preparation of which was supported by the ERIC (Educational Resources Information Center) Clearinghouse on Media and Technology, Stanford University, Stanford, California. The authors acknowledge with appreciation the assistance of Laura Spooner and Bea Kass.     

Design languages,notation systems,and instructional technology: A case study

Notational systems, used in mature fields of study, are closely related to design languages. The future of a technological field depends on the ability to communicate ideas and changes with others in the field. Instructional technology is one field that can benefit from a notation system enabling designers to duplicate, execute, and communicate their ideas. In this paper, we look at several mature fields, including chemistry, musicology, and choreography, to discover elements where a mature design language and notation system have developed. We illustrate a number of dimensions that can help designers form their design languages and select the characteristics of their notation systems.     

Formative assessment and the design of instructional systems

The theory of formative assessment outlined in this article is relevant to a broad spectrum of learning outcomes in a wide variety of subjects. Specifically, it applies wherever multiple criteria are used in making judgments about the quality of student responses. The theory has less relevance for outcomes in which student responses may be assessed simply as correct or incorrect. Feedback is defined in a particular way to highlight its function in formative assessment. This definition differs in several significant respects from that traditionally found in educational research. Three conditions for effective feedback are then identified and their implications discussed. A key premise is that for students to be able to improve, they must develop the capacity to monitor the quality of their own work during actual production. This in turn requires that students possess an appreciation of what high quality work is, that they have the evaluative skill necessary for them to compare with some objectivity the quality of what they are producing in relation to the higher standard, and that they develop a store of tactics or moves which can be drawn upon to modify their own work. It is argued that these skills can be developed by providing direct authentic evaluative experience for students. Instructional systems which do not make explicit provision for the acquisition of evaluative expertise are deficient, because they set up artificial but potentially removable performance ceilings for students.     

Learning from web-based instructional systems and cognitive style

Two of the principal issues, which have been addressed in assessments of the benefits of web‐based instructional systems, are firstly, whether the segmentation of information provided by the web structure aids users in apprehending the interrelationships between the units of information featured in the web. Secondly, whether providing the user with an overview of the web system assists in facilitating his/her learning. It is suggested in the present study that these two issues may be more effectively understood by a consideration of an individual's cognitive style. Fifty participants were assigned to one of two web‐based instructional systems featuring information on the subject of psychological ethics. The information in one of the web systems was segmented to a greater degree than the information in the other. Half the participants using each web system were given an overview of the system and half were not. After a given time using the system, participants were tested on the information from the web. The findings suggest that cognitive style and segmentation had an effect on performance, although the provision of the overview had little effect. The results are discussed in terms of a consideration of cognitive style in the design of web‐based instructional systems.     

基于Blackboard网络学堂的教学设计与教学评价研究

探讨了教学测量与教学评价的现实意义,通过4个方面深入研究了基于Blackboard网络学堂的教学设计和教学评价过程。这4个方面包括:现在在哪里——课堂教学的主体分析;要去哪里——课堂教学目标的设计;如何去那里——课堂教学内容、媒体、组织形式与方法等的设计;是否到达了那里——课堂教学的监控与评估。实践表明,信息技术的应用为教学设计和教学评价的有效应用提供了便利的工具,对高等学校教学活动的开展具有积极的推动作用。