Modeling student performance in diagnostic tasks: A decade of evolution

Since 1976 the authors have been involved in an evolving research and development program focusing on the application of computer-based simulation to diagnostic training in technical environments. Eight simulation-oriented training systems are used to describe the authors' understanding of and approach to student diagnosis. Each of the systems described capitalized on experimental and/or empirical findings from previous experiences. The article describes how design decisions were driven first by research findings and then by the reality of building effective computer-based training systems within the constraints of operational environments. The paper on which this article is based was presented at the Cognitive Skill Acquisition Workshop, Air Force Armstrong Laboratory Human Resources Directorate, Brooks Air Force Base, Texas, in July 1990. It is also published in Regian, W. J., & Shute, V. J. (Eds). (1992).Cognitive approaches to automated instruction. Hillsdale, NJ: Lawrence Erlbaum.     

Cognitively based models of courseware development

Many researchers are attempting to develop automated instructional design systems to guide subject matter experts through the courseware authoring process, a process that usually includes analysis, design, production, implementation, and maintenance. What is lacking in many existing efforts, however, is a systematic method for analyzing the interplay between user characteristics, the authoring environment, and the resulting quality of computer-based instruction (CBI). In this article, a cognitively oriented method for developing a useful and predictive model of the process of designing CBI is proposed, and its use in three small-scale case studies in Air Force training settings is reported.     

Intelligent tutoring and assessment built on an understanding of a technical problem-solving task

In this paper we describe Hydrive, an operational computer-based intelligent tutoring system built to help Air Force technicians develop generalizable skills for troubleshooting hydraulics systems. We use Hydrive as an example of how effective training and assessment is developed from a coherent understanding of a target task and how this understanding can be consistently represented in all aspects of a training/assessment instrument. We show how an organizing principle of expert behavior, active path analysis, is used to inform the design and content of the tutor's system, student and instructional models and the design of the tutor's interface. We conclude with information about Hydrive's field trial evaluation and some thoughts on knowledge acquisition and generalizability.     

An experimental comparison of “live” and filmed lectures employing mobile training devices

Conclusion It may reasonably be assumed on the basis of this study that important savings in money and manpower could be realized, with no loss in training effectiveness, and the possibility of a real gain in training effectiveness, by the application of well-designed films to supplement or replace mobile training devices such as are currently being used in Air Force transitional training programs. This article reports an Air Force study which was made to investigate the feasibility of using filmed recordings of familiarization lectures, given to mechanics transitioning to new types of aircraft. The study was conducted as a part of a U. S. Air Force program of research on the utilization of audio-visual media under the direction of A. A. Lumsdaine and S. M. Roshal of the Air Force Personnel and Training Research Center. H. H. Shettel is now with the Perceptual Development Laboratories in St. Louis, and E. J. Faison is now with Needham, Louis and Brorby in Chicago. The opinions and interpretations given by the authors are their own and are not to be construed as representing official views of the Air Force.     

CBT OR IVD?—That’s the question

Summary Computer-Based Training is usually based upon modular branching paradigms familiar in print-based programmed learning materials. It successfully involves students actively in the instruction, but due to inherent limitations lacks the realism and immediacy of television. Interactive video is a recent development in instructional technology, merging of computers and television. This marriage has produced a hybrid technology which can provide the structure and involvement of computerassisted instruction and the realism of television. For interactive video to realize its potential, rigorous design and technical requirements must be satisfied. Educators and trainers can make informed decisions about the use of CBT versus IVD by studying the large body of research that deals with the instructional effectiveness of these technologies. Also, answering questions regarding learning outcomes, budget constraints, and other issues is critical before committing to one medium over the other. Knowing the advantages and disadvantages of each instructional system is key to selecting an appropriate and cost-effective instructional approach.     

A conceptual design model for CBT development: A NATO case study

CBT (computer-based training) can benefit from the modern multimedia tools combined with network capabilities to overcame traditional education. The objective of this paper is focused on CBT development to improve strategic decision-making with regard to air command and control system for NATO staff in virtual environment. A conceptual design for the CBT of the Web-based command and control system including a methodology and a model of content relations is described. The system helps the personnel to effectively plan, execute, monitor and assess Air operations in a responsive and timely manner. The developed prototype of CBT for Web-based air command and control system for NATO is illustrated.     

Effects of Training in the Use of Social and Cognitive Strategies: An Intervention Study in Secondary Mathematics in Co‐Operative Groups

ABSTRACT

This article reports on an intervention study into the effects of a training in the use of social and cognitive strategies on the learning outcomes of students in secondary mathematics education. Special attention is given to differential effects for high‐ and low‐achieving students. The focus on differential effects is derived from studies into learning in small co‐operative groups, and from the results of meta‐analyses into the effects of training in learning strategies. From these studies it can be concluded that in general such programs contribute to learning. However, it seems that low‐achieving students are unable to benefit from interventions of the kind investigated (i.e., co‐operative learning and training in learning strategies). The main question is whether it is possible to design an instructional program from which all students benefit, and from which the low‐achieving students profit more than their counterparts in the control‐program. In the present study three instructional programs for co‐operative learning were compared: (i) an experimental program with special instruction in the use of social strategies; (ii) an experimental program with special instruction in the use of cognitive strategies; and (iii) a control program without training in either cognitive or social strategies. The programs were identical with respect to mathematical content and general instructional settings (a combination of whole‐class instruction, working in co‐operative groups and individual work). The experiment addressed the following research question: what are the general and differential effects of a training in the use of social and cognitive strategies on the results of learning in secondary mathematics? The research was conducted in two schools for secondary education in a total of 21 classes, involving a total of 511 students. The design was a pretest‐posttest control group design, using two experimental groups and one control group. The data were analysed from a multi‐level perspective. The outcomes of the investigation clearly show the effects of the intervention. Teaching cognitive and social strategies has the expected, positive effects. In addition to this main effect, a compensatory effect for the low‐achieving students was found. The low‐achieving students in the experimental conditions outperformed their counterparts in the control group.

     

Integrating cognitive task analysis into instructional systems development

Traditional methods for task analysis and training design, such as those embodied in Instructional Systems Development (ISD), decompose jobs into discrete tasks composed of specific action sequences and identify prerequisite knowledge and skills for each task. Although these methods have been effective for designing training for simple procedural skills, they offer little insight for analysis or training of jobs involving complex cognitive skills, which increasingly require training today. Because of this, cognitive considerations need to be incorporated into ISD, particularly in the task analysis phase. Recently, cognitive methods have begun to be used to conduct task analysis for training program development and human-computer system development. In this article, recent developments in cognitive task analysis are reviewed, and The Integrated Task Analysis Model (ITAM), a framework for integrating cognitive and behavioral task analysis methods within the ISD model, is presented. Discussed in detail are ITAM's three analysis stages—progressive cycles of data collection, analysis, and decision making—in three components of expertise: skills, knowledge, and mental models. This research was supported in part by the U.S. Air Force Armstrong Laboratory, Human Resources Directorate, Operational Training Division. The views contained herein are solely those of the authors and do not necessarily reflect the views of the U.S. Air Force or the Department of Defense. Both authors contributed equally to this article. Joan M. Ryder is with CHI Systems, Inc. Richard E. Redding is with Human Technology, Inc. Drs. Bernell Edwards and Thomas Killion were the technical monitors and provided valuable guidance. The authors thank John Cannon, Sharon Fisher, Wayne Zachary, and the participants of an AL/HR Workshop on Cognitive Skills Acquisition for helpful comments.     

Instructional development for teachers in higher education: impact on teaching approach

Although instructional development for teachers has become an important topic in higher education, little is known about the impact it has on daily teaching practice. The lack of systematic programme evaluation is an ongoing concern. In this study we investigate by use of a quasi-experimental design, the impact of an instructional development program for beginning university teachers on their teaching approach. Quantitative pre-test as well as post-test data were assembled from 20 experimental teachers and 20 control teachers. At the post-test qualitative data were gathered too: 17 teachers of the experimental group and 12 teachers of the control group were interviewed. Paired t-tests and analysis of covariance with the pre-test scores as a covariate showed some effect of instructional development on teaching approach. The analysis of the qualitative data sustained this result. Our results reveal the influence of instructional development on teachers’ teaching approach being slightly different for teachers belonging to distinct disciplines. Several interpretations and perspectives for further research are discussed.     

Individualizing through treatment matching: A necessary but not sufficient approach

The effects of various alternative treatments (modules), designed to compensate for student differences in precourse memory abilities (processing and retrieval skills) and motivation (anxiety, curiosity), were investigated for lessons differing in content and task requirements. Performance on each module was compared to performance on the original (mainline) instructional module for 63 to 171 students in the Air Force Advanced Instructional System’s inventory management course. Interaction analyses on lesson times-to-criterion and criterion test scores indicated that the compensating treatments were partially effective in improving the performance of low-memory or low-reading ability students, low-curious students, or high-anxious students. Additional benefits that may be expected from an individualization approach that attempts to modify students’ cognitive and affective learning strategies are discussed. This research was supported by Air Force Human Resources Laboratory Contract F33615-73-C-4004. Requests for reprints should be sent to Barbara L. McCombs, Denver Research Institute, Social Systems Research and Evaluation Division, University of Denver, Denver, CO 80208.     

Training higher education teachers for instructional design of competency-based education: Product-oriented versus process-oriented worked examples

Teachers involved in the development of competency-based higher education (CBE) are expected to fulfil a new role of instructional designer. As a consequence, they are confronted with the problem to translate abstract new curriculum principles into concrete learning tasks. Recent studies have shown that teachers can be trained to apply an instructional systems design methodology. After this training the teachers were able to design better learning tasks for CBE in comparison with their experienced-based design efforts. In order to optimize the training, this study compares an experimental condition with process-oriented worked examples with a conventional training condition with emphasis on product-oriented worked examples. After the training, the participants—25 higher education teachers—had to apply the ISD methodology to two design problems. The quality of the resulting design materials, as rated by experts, was higher in the product-oriented worked examples condition than in the process-oriented worked examples condition. The significance of this finding for training approaches to design methodology for CBE is discussed.     

The educational psychologist as instructional designer

Educational psychology, as an academic discipline, needs a model or paradigm for conceptualizing and solving educational problems. The systems approach to instructional design is proposed as a unifying process which was developed primarily by educational psychologists and has wide applicability in education. The use of the systems approach is advocated because (1) it provides a method for delivering effective service, (2) it incorporates many concepts presently familiar to educational psychologists and suggests other topics which should be of concern, (3) it suggests a wide array of research and evaluation problems, and (4) it provides an integrative framework for the design of graduate training programs. These four factors are discussed in detail.     

ISD model building: From tabula rasa to apple peel

An evidence‐based practice (EBP) model is proposed to replace the more‐art‐than‐science “rope bridge” currently spanning the defining space of a training need and the designing space of a training curriculum. The rope bridge analogy symbolizes a way to address perceived deficiencies and research gaps in the analysis‐design‐development‐implementation‐evaluation (ADDIE) model. Not to be construed as mere window dressing, a developmental model of EBP‐adherent instructional design is offered as a prototype for changing ADDIE's internal landscape and making instructional design more effective.     

Variation in I.Q. at the Preschool Level

The development of an Organizational Assessment Package (OAP) for use by Air Force management consultants is presented. Development was based on a contingency model of effectiveness with separate inventories measuring the major components of the model. Data were collected using the OAP from 4,786 Air Force military and civilian personnel. Factor analyses resulted in 22 factors which were orthogonal within each inventory. Internal consistency indices were computed using the coefficient alpha formula for the highest loading items which defined each factor. Some redundancy between inventories was identified and recommendations for items and factors for an operational OAP were made.     

基于“关联”原理的绩效导向企业培训教学设计研究

文章在绩效导向研究的基础上,以教学系统设计中"关联"教学的工作原理为依据,对企业绩效培训的内容与过程进行研究,总结出绩效导向企业培训的特点及教学设计原则。将企业培训中的学习、教学与绩效技术进行整合,设计了教学与绩效关联的企业培训教学设计过程模式及基于关联原理的绩效导向教学设计思路。通过对以上内容的详细论述分析,进一步为企业培训提供借鉴。     

论高校教师教学发展的意义和策略

高校教师教学发展是教师发展的核心成分。无论是明晰人才培养理念、目标与模式,还是设计、实施与评价课程,优化人才培养过程,或者重建学术概念、体现自己的组织存在,高校都需要教师的教学发展。为了促进高校教师教学发展,应当注意:牢固树立教师教学能够发展的信念,认真剖析教师教学发展不足的原因,科学搭建牵引教师教学成长的平台,切实制订促进并保障教师教学发展的政策与制度,正确处理高校教师教学和科研的关系,重视教师教学发展的组织性质。     

新范式教学设计的共同基础——解读《教学设计的国际观（第1册）：理论·研究·模型》

随着信息时代的到来,国际教学设计领域开始摆脱传统教学系统设计模型“专注于某种心理学取向的教学程序”的困囿,走向整合心理学、技术、文化、测量、评价、管理等多学科的开放的专业领域。《教学设计的国际观（第1册）：理论·研究·模型》汇编了欧美教育、心理、文化,特别是教学设计领域学者的理论与实践著作,从多维视角梳理了教学设计发展的历史沿革,阐述了教学设计的理论基础、研究基础和实践基础。新范式教学设计强调以一种开放的态度来认识关于教学设计的各种理论、基于教学价值和方法论取向批判性地接纳不同研究成果,注重信息技术与教育理念的整合,注重新的评估理念和方法,注重复杂系统视域中的学习环境的建构,注重跨学科研究和跨领域的应用。     

国际教学设计领域在关注什么——《教学设计中课程、规划和进程的国际观》深度解读

针对教学研究领域的三个主题：教学设计、课程设计、信息通信技术与媒体,国际教学设计研究领域的学者们主要关注如下研究热点：认知和学习心理学研究成果为教学设计模型提供了新的理论基础和创新途径,课程开发和信息通信技术需要进一步整合到教学设计模型中;课程开发及其对教学设计模型的影响;信息通信技术对教学设计和课程开发的影响,从更加整合和全面的视角来看待学习、教学、课程和技术之间的关系。《教学设计中课程、规划和进程的国际观》从以上三个方面展示了国际观点。当前教学设计研究和实践逐步形成以下发展趋势：一是教学设计研究越来越呈现出跨学科特点;二是运用技术创建有效学习环境成为教学设计研究的重要领域;三是欧美学者在教学设计研究领域既共享众多核心观点,又在研究角度、内容上各有不同,使得整体研究成果深度交融。