从用户到顾问:教学设计人员在当前在线教学中所担当的“新角色”(英文)

本文讨论了教育技术的专业实践如何影响教学设计师的角色。技术前景的动态本质使得教学设计师的工作复杂化,使得他们在应用所学的系统设计模式时有着很大的困难。目前,教学设计师在帮助机构有效采用教育技术方面担当着更加多样的、精细的角色。在假定这些新角色的基础上,教学设计人员应当跨越自身的舒适区,不仅仅是与学科专家合作的设计人员,或者是第一个吃螃蟹使用技术的人,而是真正成为绩效专家,协调机构内的不同力量。     

The priorities established among data sources when instructional designers revise written materials

The objective of the study was to investigate the priorities instructional designers establish among data sources when they revise written materials and the relationship of their practice to standard models of formative evaluation. Two modules of printed instructional material were revised by each of eight experienced instructional designers under three treatment conditions, using a counterbalanced design. Data were collected using a think-aloud procedure. The think-aloud protocol was segmented, coded and analysed. Results show that instructional designers were significantly more likely to incorporate their own knowledge into revisions than to use feedback data, and when they did use feedback data, they preferred learner comments. They did not accurately assess their use of their own knowledge or of feedback data; they thought they used feedback data more than the results showed. These instructional designers reflected some aspects of the standard instructional design model when they revised instructional materials, but their experience has caused many of the procedures to become internalized, so they are invisible in the protocols. This research was funded in part by a grant from the Social Sciences and Humanities Research Council of Canada.     

Cognitive Style FD/FI as a Learner Selection Criterion in Formative Evaluations A Qualitative Analysis

Single–subject formative evaluation appears to be a cost–effective strategy for improving instructional products. However, the criterion to use for selecting an appropriate test subject who could generate optimal feedback data for improving the instructional product remains a central concern among performance technologists. This article reports the results of a qualitative study of the effectiveness of the cognitive style construct field–dependent/independent as a student selection criterion in formative evaluation. In the study, we collected formative evaluation data from two field–dependent (FD) and two field–independent (FI) test subjects while they were individually interacting with a CAI package. We focused on four different sources of data: think–aloud protocols, researcher/subject interactions, informal observations, and debriefing interviews. Our analysis of the formative evaluation data indicates that the FI individuals were better test subjects than their FD counterparts. FI subjects showed a great deal of confidence in entering the formative evaluation process. Their feedback was abundant and precise and included specific suggestions for improving the material. They not only identified their own difficulties but also speculated about difficulties other students may encounter. In contrast, the FD subjects were anxious and demonstrated less confidence in approaching the evaluation activities. Frequent probing was necessary to trigger their reactions and generate their feedback. Their feedback data were vague, and more inferences were required for translating them into revision decisions. Both FD and FI subjects could identify major discrepancies in the presentation of the material (events of learning) as well as gross misconceptions in the processing of information. Although the FD and FI feedback data differ both qualitatively and quantitatively, no conflicting observation was made.     

Heuristic task analysis on e-learning course development: a formative research study

Utilizing heuristic task analysis (HTA), a method developed for eliciting, analyzing, and representing expertise in complex cognitive tasks, a formative research study was conducted on the task of e-learning course development to further improve the HTA process. Three instructional designers from three different post-secondary institutions in the U.S. were selected for interviews. The interviews focused on three e-learning course development cases (one from each institution), and the participants were asked to articulate their underlying thoughts and principles for designing e-learning courses. Overall, the HTA process worked well in the sense that the study could elicit procedural steps and sub-steps involved in e-learning course development and heuristic knowledge with which the instructional designers performed each step. On the surface, the e-learning course development processes that the instructional designers used looked more alike than different, entailing major steps such as meeting with faculty, developing content, monitoring courses in progress, and debriefing the instructor and students. The underlying principles and knowledge that guided each instructional designer through the processes, however, were unique in that each instructional designer constructed her own heuristics to accommodate the myriad contextual factors that arose in her work setting. The study also discussed the challenge of identifying the simplest yet most representative e-learning course development case with multiple experts and suggestions for further improving the HTA process were also presented.

The Attributes Of Instructional Materials

In this paper, we describe a comprehensive list of attributes of instructional materials. The list is of particular value for two reasons. First, it is drawn from the instructional design literature and represents a consolidation of recommendations constructed by or attributed to experts for the evaluation and revision of any type of instructional material. Second, it has been validated and found reliable through a series of studies during the past five years. We provide suggestions for how researchers and academics might use the attributes to advance knowledge in the field and how instructional designers, consultants and instructors might use the attributes to enhance their practice.     

Exploring the potential of a computer tool for instructional developers

Information and communication technology tools currently permeate almost every professional domain. Those geared toward the field of instructional development have emerged in recent years. This article explores the potential for linking the domains of computer support and instructional development.This article reports on the design and evaluation of CASCADE (Computer Assisted Curriculum Analysis, Design and Evaluation), a computer system that supports instructional developers during formative evaluation efforts. Five prototypes of CASCADE were created and evaluated on the basis of their validity (reflection of state-of-the-art knowledge and internal consistency); practicality (ability to meet the needs, wishes and contextual constraints of the target group); and effectiveness (improved user task performance).The results of this study suggest that the use of CASCADE could: (a) improve the consistency of formative evaluation plans and activities; (b) motivate developers by elevating their confidence in using formative evaluation activities; (c) save time; and (d) help to provide justifications for decisions made.     

The influence of participants in formative evaluation on the improvement of learning from written instructional materials

Formative evaluation in instructional design is widely advocated as a means to improve instructional materials through tryouts with learners and experts and revision based on this feedback. Research on formative evaluation has sought to prove its effectiveness and to provide guidance as to which methods of collecting feedback and which sources of feedback are most effective in improving learning. It is difficult to determine from the research whether aspects of tryout or of revision are responsible for improved learning. This paper describes a systematic investigation of both tryout and revision to more clearly determine the influence of different formative evaluation participants (experts, learners and revisers) on the improvement of learning from written instructional materials.It was found that revised versions incorporating learner feedback had the most impact on improving learning from the materials. It was also found that revisers have a far more powerful impact on formative evaluation outcomes than was previously supposed, in terms of how they mediate and incorporate the feedback they are given. The results refute contentions that any revision is better than none since versions revised without learner feedback did not improve learning. Implications are discussed.     

In‐Text Questions: A Note on an Experiment in Instructional Design

The use of instructional design is increasingly being discussed in the distance education and higher education literature. This paper reports an experiment in which self study questions are inserted in the text of lecture note material for external students. An evaluation revealed that students who received notes with in‐text questions were no different from those who received the standard lecture notes as judged by both performance and attitudinal criteria. Possible reasons for the perverse findings are given and it is concluded that professional instructional design assistance would be of value to subject matter experts.     

How do instructional designers evaluate? A qualitative study of evaluation in practice

This study employed a qualitative research design to investigate how instructional designers use evaluation in everyday design practice. While past research has examined how designers spend their time, how they generally make decisions, and expert-novice differences, little attention has been paid to use of context, input, process, or product evaluation, from the perspective of practicing designers. Based on interviews of practitioners, our findings included ten themes regarding how designers use evaluation to improve their products. While these results substantiate to some degree the claim that practitioners believe clients will not pay for formal evaluations, they also suggest that practitioners use evaluation in important but less formal ways. Other conclusions regarding the role of evaluation in design are provided and future directions for training and research are discussed.     

How Instructional Designers Solve Workplace Problems

This naturalistic inquiry investigated how instructional designers engage in complex and ambiguous problem solving across organizational boundaries in two corporations. Participants represented a range of instructional design experience, from novices to experts. Research methods included a participant background survey, observations of problem‐solving activities, in‐depth interviews, and analysis and evaluation of project documents and other tools. The findings revealed differences between experts and novices with regards to tolerance of ambiguity, expectations about their own roles in finding solutions for their clients, adaptability, attention to appropriate details, and management of workplace stress. The contrast between instructional design processes taught in universities and actual workplace practice was noted by both expert and novice participants. Experienced participants demonstrated adaptability in processes and communications to efficiently arrive at viable solutions for their clients. Expectation setting and relationship building emerged as techniques for creating environments supportive of instructional designers' problem‐solving activities.     

The Role of Coaching Within the Context of Instructional Design

Upon entry into the instructional design workforce, there is a need for instructional designers to continue to hone their craft and skill development. Often times novice instructional designers are paired with experts during the onboarding process. Coaching is utilized to provide novices and those less experienced with the necessary support they need to enhance their skillset. This article explores the use of coaching in the professional development of instructional designers. Specific attention is given to theoretical approaches, models, and contexts related to coaching.     

The Use of Task Analysis Procedures by Instructional Designers

The purpose of this study was to learn about the use of task analysis procedures by instructional designers. Task analysis is regarded by many to be the most integral part of the instructional design process and the most technical aspect of the instructional designer's job. It is also thought to be the most ambiguous of the instructional design, processes. A questionnaire was developed that surveyed 164 members of the International Society for Performance Improvement (ISPI) and the Association for Educational Communications and Technology (AECT) with instructional design responsibilities. This research learned what instructional design activities designers associate with task analysis and the percentage of time instructional designers devote to task analysis. It also learned what task analysis methods are most and least often used by instructional designers and identified those task analysis methods not known by instructional designers. Finally, this study determined how instructional designers learn to conduct task analyses and the factors that impact task analysis practices.     

Formative Evaluation Alternatives

Traditional formative evaluation methods and tools have a documented history of successful use. Expert, one-to-one, small group, and field-test methods have been used for decades. However, changes in design practice, technology, and lately, in theory, have resulted in alternative evaluation methods that are not as well-known in current design literature. These methods can be used to complement or replace traditional evaluation methods in a formative evaluation project. Some of these alternative methods use different groupings of experts or learners, while others use different evaluation questions or technologies. This article explains each of these formative evaluation alternatives and outlines their advantages, disadvantages, and applicable contexts.     

Knowledge Based Artificial Augmentation Intelligence Technology: Next Step in Academic Instructional Tools for Distance Learning

With augmented intelligence/knowledge based system (KBS) it is now possible to develop distance learning applications to support both curriculum and administrative tasks. Instructional designers and information technology (IT) professionals are now moving from the programmable systems era that started in the 1950s to the cognitive computing era. In cognitive computing or KBS a machine understands natural language, adapts, learns, and generates and evaluates hypotheses. A KBS system can manage data and assist instructional designers in creating tools and curricula that generate meaningful applications. As a proof of the concept, the authors conducted an exploratory case study with the input of twenty subject-matter experts (programmers, instructional designers, and content experts) for development of a proto-type KBS scholarly writing software (SWS) application that can be used for distance/online learning. Philosophical differences between the artificial intelligence and augmented intelligence approaches are also discussed. The role of instructional designers in the development and use of augmented intelligence with IBM’s Watson is also a significant part of the discussion.     

The influence of cognitive domain content levels and gender on designer judgments regarding useful instructional methods

Instructional theory is intended to guide instructional designers in selecting the best instructional methods for a given situation. There have been numerous qualitative investigations into how instructional designers make decisions and the alignment of those decisions with theoretical influences. The purpose of this research is to more quantitatively explore the question of how instructional designers actually use instructional planning theory to judge the usefulness of instructional methods. We asked 56 instructional designers to rate the usefulness of 31 instructional methods for six different cognitive domain content level conditions. The results show that content level has a statistically-significant influence on a designer’s judgments regarding the usefulness of an instructional method. A designer’s gender also has a statistically-significant influence on a designer’s judgments regarding methods, but a weak effect size limits this result. Overall, the results provide evidence that supports the core principles of instructional planning theory, specifically method generality. The results also provide instructional designers further guidance in selecting the most useful instructional methods for cognitive domain content levels.     

Validated Competencies for Distance Teaching

The International Board of Standards for Training, Performance and Instruction (IBSTPI) provides a methodology for drafting and validating teaching competencies. This study applied the IBSTPI methodology to identify and validate distance education (DE) instructor competencies. The research team's review of DE literature in the past 10 years resulted in a list of 20 competencies. The list was reviewed by 18 distance learning professionals as subject matter experts (SMEs). The SMEs' feedback and comments along with the performance statements developed for the competencies were analyzed which resulted in 54 task statements describing the instructional activities of a DE instructor. These tasks were then rated by 148 instructors in terms of importance, frequency of performance, and the perception of relative time spent on each task. The task analysis resulted in a list of 17 most frequently performed tasks that we linked back to the corresponding original competencies. Analysis of these data pointed out the significant characteristics of teaching from a distance including interaction with learners and technological and logistical requirements. This article presents the methodology and findings of this study and discusses their implications for recruitment, selection, and training of DE instructors.     

Examining Instructional Design Principles Applied by Experienced Designers in Practice

There has been ongoing debate regarding the efficacy of teaching instructional design models to novice designers given that experienced instructional designers often use principles and adapted models when they engage in the instructional design problem‐solving process. This study utilized the Delphi technique to identify a core set of guiding principles used by designers in their practice. The purpose of this study was (1) to examine and describe the principles that guided instructional designers’ practice and (2) to identify the extent to which participants’ frames of reference included components of the ADDIE instructional design model. Sixty‐one principles were ultimately identified (reached consensus among the Delphi panel members). Thirty‐two of the principles aligned with the primary components of the design process (e.g., analysis, design, development, and evaluation). Additional principles (n = 29) related to other characteristics of design such as communication, project management, and design characteristics.     

How instructional design experts use knowledge and experience to solve ill‐structured problems

This study examined how instructional design (ID) experts used their prior knowledge and previous experiences to solve an ill‐structured instructional design problem. Seven experienced designers used a think‐aloud procedure to articulate their problem‐solving processes while reading a case narrative. Results, presented in the form of four assertions, showed that experts (1) narrowed the problem space by identifying key design challenges, (2) used an amalgam of knowledge and experience to interpret the problem situation, (3) incorporated a mental model of the ID process in their problem analyses, and (4) came to similar conclusions about how to respond to the situation, despite differences in their initial conceptualizations. Implications for educating novice instructional designers are discussed.     

Read-Think-Aloud Protocols: A New Data-Source for Formative Evaluation

Traditional approaches to formative evaluation include one-to-one trials which provide some information about revisions that are needed in instructional materials, but typically fail to provide sufficient information about the learner's cognitive processing problems while studying the materials. Read-think-aloud methods are a viable alternative to traditional one-to-ones and provide valuable information about the learner's cognitive processing while reading instructional text. In this paper we describe the read-think-aloud method of formative evaluation and offer suggestions for its use.