Evaluating instructional software

This article describes a new model for evaluating instructional software. Also described is a study in which the new model was field tested. Unlike most such models, which focus on the instructional and technical characteristics of software, the model focuses on the extent to which students learn the skills a software package is intended to teach. It is argued that by using this approach, educators will be better able to reliably identify software that is instructionally effective.     

Evaluating instructional quality

ABSTRACT

Building on the papers in this special issue, this article uses modern conceptions of validity theory to provide a framework for considering the evaluation of teaching quality. The 3 facets of teaching quality focused on are domain conceptualization, evidence and inferences, and their evaluation. Domain definitions vary in their specificity with tradeoffs in their range of applicability and specificity of inference. Evidence collection can range from highly standardized assessments to observations that must attend to evidence from a myriad of classroom interactions. For all assessments, however, even the most standardized, different interpretations of assessment tasks can threaten the validity of score interpretations. The papers consider a range of processes that are designed to generate, support, and interrogate the validity of inferences based on assessment scores. A fundamental question underlying this type of measurement is whether differences in the quality of teaching that students experience can be causally attributed to the teacher.     

Student attitudes toward closed-circuit instructional television

Summary The objectives of this study were to examine (1) the relative effectiveness of closed-circuit television for classroom teaching compared to conventional teaching and (2) the acceptability of closed-circuit television teaching to the students. A course was offered both by closed-circuit television and by conventional teaching. Students within each group were asked to rate the course and other related concepts. In addition, their examination grades were obtained. No differences were found with respect to performance. The students taught via television rated the course and their instructors relatively lower than those who received conventional instruction. Howard M. Bobren is assistant to the director, Institute of Communications Research and Sheldon L. Siegel is research assistant, College of Education, both at the University of Illinois.     

A strategy for instructional television research

Conclusion In this article an empirical approach to instructional television research has been defined. This approach does not seek to replace theoretical studies but seeks rather to establish, within those institutions using instructional television, a body of knowledge based on empirical generalizations. Many authors have suggested the importance of defining objectives in assessing instruction, but their ideas filter only very slowly through educational systems. Part of this resistance may be traced to the complete revision of assessment instruments required by instruction by objectives, or standards. However, by asking the question, What happens during instruction? it may be possible to devise instructionally useful measures. Similarly, this question presents teaching, research, and production staff with a problem they may be eager to define and solve.     

Evaluating instructional software: A simplified model

This article describes the evaluation of a simplified version of the new software evaluation model proposed by Reiser and Dick (1990). This approach to evaluation of instructional software focuses upon the collection of performance data to determine the extent to which students learn the skills the software is designed to teach. In two field evaluations of instructional software packages, it was found that use of the simplified software evaluation model generally produced evaluative conclusions similar to those reached using the original, more complex model. It was also found that student learning data did not support the high ratings assigned by subjective software evaluation services. These findings support the use of the simplified model by teachers and media specialists who have limited time and resources, and stress the need for local on-site evaluation efforts.     

Implications of instructional psychology for the design of educational television

This article examines the implications of a cognitive model of learning for the design of educational broadcast television. Specifically examined are research studies with instructional implications for such functions as pacing, cueing, modeling, and transformation of the television presentation. Robert B. Kozma is Associate Professor at School of Education, Associate Research Scientist at the Center for Research on Learning and Teaching, and Project Director, National Center for Research to Improve Post Secondary Teaching and Learning, The University of Michigan, Ann Arbor, MI, 48109. An earlier version of this paper was presented at the American Educational Research Association Annual Conference, New Orleans, LA, 1984. Appreciation is expressed to Barbara Beath, Edward Saunders, anonymous reviewers, and students in the author’s seminar on educational television for their helpful comments.