Validation of Cognitive Sensitivity for Item Response Curves

This study sought a scientific way to examine whether item response curves are influenced systematically by the cognitive processes underlying solution of the items in a procedural domain (addition of fractions). Starting from an expert teacher's logical task analysis and prediction of various erroneous rules and sources of misconceptions, an error diagnostic program was developed. This program was used to carry out an error analysis of test performance by three samples of students. After the cognitive structure of the subtasks was validated by a majority of the students, the items were characterized by their underlying subtask patterns. It was found that item response curves for items in the same categories were significantly more homogeneous than those in different categories. In other words, underlying cognitive subtasks appeared to systematically influence the slopes and difficulties of item response curves.     

小学生乘法计算能力的认知诊断及补救教学

认知诊断通过分析被试的项目作答反应,推断被试的认知属性掌握状态,为学习困难学生设计补救教学提供了非常有价值的信息。本文作者在探讨了小学生多位数乘法计算能力的认知属性、编制了2份相同考核模式的认知诊断测验后,选择江西某小学310名高年级学生为被试,先施测第1份认知诊断测验,采用DINA模型,自编参数估计程序进行诊断,得到了每一个被试的属性掌握模式分类及全体被试在各个属性上的掌握情况。然后设计和实施补救教学,在实施补救教学后再施测第2份认知诊断测验以检验补救效果。研究发现：（1）该小学高年级学生对0XN运算法则、多位数乘以两位数的运算程序、乘法进位认知属性的掌握不太理想,特别是乘法进位。（2）属性掌握模式中属全部掌握模式的被试人数占86．47％,其余被试均分类于存在各种认知不足的掌握模式。（3）比较两份认知诊断测验报告,结果表明在认知诊断指导下的补救教学有针对性,补救后被试正确作答项目增多,属性掌握个数也有所增加,补救效果良好。     

Heuristic task analysis on e-learning course development: a formative research study

Utilizing heuristic task analysis (HTA), a method developed for eliciting, analyzing, and representing expertise in complex cognitive tasks, a formative research study was conducted on the task of e-learning course development to further improve the HTA process. Three instructional designers from three different post-secondary institutions in the U.S. were selected for interviews. The interviews focused on three e-learning course development cases (one from each institution), and the participants were asked to articulate their underlying thoughts and principles for designing e-learning courses. Overall, the HTA process worked well in the sense that the study could elicit procedural steps and sub-steps involved in e-learning course development and heuristic knowledge with which the instructional designers performed each step. On the surface, the e-learning course development processes that the instructional designers used looked more alike than different, entailing major steps such as meeting with faculty, developing content, monitoring courses in progress, and debriefing the instructor and students. The underlying principles and knowledge that guided each instructional designer through the processes, however, were unique in that each instructional designer constructed her own heuristics to accommodate the myriad contextual factors that arose in her work setting. The study also discussed the challenge of identifying the simplest yet most representative e-learning course development case with multiple experts and suggestions for further improving the HTA process were also presented.

Diagram information and its organisation in memory: Exploring the role of skill and experience

Two contrasting methods of investigation were used to characterise the basis upon which the mental representation of a scientific diagram was organised in individuals having different levels of experience and skill in the interpretation of this type of diagram. Each of these methods is described and several important methodological issues are discussed. In the first method (the Card Sort method), subjects performed a three-stage grouping of diagram elements in a card sorting task that produced an hierarchical ordering of the information constituting the diagram. In the second method (the Copy-Recall method), subjects copied then produced drawn recall of a given diagram. Measurements of the sequence in which diagram elements were produced and the time intervals between each production were used to infer the underlying cognitive structuring involved in the mental representation of the diagram information. Questions are raised concerning the way resulting data can be analysed and interpreted most effectively. Specializations: Mental representation and processing of scientific diagrams, characteristics of explanatory diagrams, visual aspects of problem solving, instructional design.     

Cognitive task analysis for instructional design: Applications in distance education

The increasing complexity of the subjectmatters taught through distance education calls for a cognitive approach to instructional design. Cognitive task analysis‐based instructional design (CTA‐BID) is especially appropriate for distance education, which requires high‐fidelity instructional materials that teach cognitive content with little or no classroom instruction. CTA‐BID organises instructional materials around the results of a cognitive task analysis that identifies the optimal knowledge structures, mental models, strategies, and skills underlying expertise in the subject matter. CTA‐BID has produced innovative instructional programmes aimed at teaching cognitive skills, accelerating the development of expertise, and improving student performance and training efficiency. This article provides an overview of CTA‐BID and its applications in the design of instructional and testing materials for distance education. I also review recent developments in education, psychology, and instructional design that complement CTA‐BID.     

Instruction-first and problem-solving-first approaches: alternative pathways to learning complex tasks

According to cognitive load theory, using worked examples is an effective and efficient instructional strategy for initial cognitive skill acquisition for novice learners, as it reduces cognitive load and frees up cognitive resources to build task competence. Contrary to this, productive failure (as well as invention learning, desirable difficulties and other problem-solving-first) frameworks suggest that scaffolded problem-solving activities (a generation phase) preceding explicit instruction enhance learners’ performance. The reported experimental study investigated the effectiveness of different levels of instructional guidance provided to students during the learning phase preceding explicit instruction in standard solution procedures in enhancing students’ engagement and transfer problem-solving skills. Specifically, the study compared partially-guided or unguided attempts at generating problem solutions as opposed to comprehensive guidance, in the form of a worked example. Levels of experienced cognitive load and learner motivation were evaluated in addition to delayed post-test performance scores. There were no differences between the three groups on the transfer post-test outcomes, even though the condition with fully-guided worked examples prior to the explicit instruction expectedly reduced cognitive load relative to the conditions without such guidance. There were also differences between the conditions on some sub-dimensions of the motivation scale. In general, the findings indicate that similar overall outcomes (delayed transfer performance) could be achieved by different sequences of instructional tasks aimed at achieving different sets of specific goals.     

The cognitive approach to training development: A practitioner's assessment

This article begins this special issue of ETR&D-Development by discussing what the cognitive approach to instructional design (ID) is and how ID practitioners can design training differently using the approach. Following some introductory comments about purpose, scope and perspective, the article is in two parts. The first part describes why the cognitive approach to ID is important and how the current approach to instructional design and training development is different from the cognitive approach. It then explains how learning occurs according to the cognitive point of view, and the different categories of learning according to one type of cognitive psychology. The second part describes a model that synthesizes and summarizes the components of a well-designed lesson, and describes what is different about this model from the current approach to ID. This model relates what learners have to do to learn to what instructional designers have to do to help them do so. It presents and briefly explains and exemplifies a general framework for instructional design based on cognitive psychology. Finally, it presents a table that can be used as a job aid to design training.     

Assessing functional effectiveness of pictorial representations used in text

Conclusion Recent literature concerning the use of illustrations in text has stressed the need for an assessment of the instructional function being served by the illustration. When an illustration is incorporated into textual matter, it should be serving a specific, pedagogically sound instructional function. Otherwise, the reader probably will not benefit from the illustration’s inclusion in the text; in fact, the reader might be distracted from the text by the illustration, with no cognitive gain being derived from this distraction. Therefore, to the traditional criteria for selection of illustrations used with text must be added the instructional function that is to be served. To determine whether an illustration will serve an intended instructional function effectively, a close examination of both the illustration’s attributes and its relationship to the text must be made. Research has shown that illustrations are composed of a variety of attributes pertaining to physical, instructional, and relational qualities of the illustrations, and that these attributes affect the way in which illustrations can be used as instructional tools. Therefore, the attributes present in an illustration will account in part for its effectiveness in serving an instructional function. The findings of this study suggest that illustrations possessing literal representation are more effective than illustrations possessing analogical representation when the instructional function to be served is identification of properties of phenomenal information, and that illustrations possessing analogical representation are more effective than illustrations possessing literal representation when the instructional function to be served is clarification of nonphenomenal information. Thus, this study is one of many needed to provide a comprehensive analysis of illustrations functioning as instructional supplements to text. Until a thorough understanding of the relationship between attributes and functions of illustrations is achieved, textbooks will in all probability continue to include illustrations which do not fulfill their instructional potential.     

Online Training: An Evaluation of the Effectiveness and Efficiency of Training Law Enforcement Personnel over the Internet

Evaluation of online training is a necessity to its future as a viable alternative method to classroom training. The purpose of Study 1 was to evaluate the effectiveness and efficiencies of online training as it compared to current classroom training with Nebraska Law Enforcement Training Center's (NLETC's) Jail Management trainees. Trainees were randomly assigned to receive either online or classroom training. Learning, motivation, and attitudes were measured for instructional effectiveness. Instructional time and cost/benefit calculations were used as measures of efficiencies. Results indicated, as predicted, that online training is as effective an instructional method as classroom training, and more efficient than classroom training. No meaningful learning differences occurred between the two groups, but online training was completed in almost half the time of classroom instruction and at a lesser cost. However, the classroom group reported higher motivation and positive feelings concerning their instruction than did the online group. Study 2 experimentally examined learning, instructional time, motivation, and attitude advantages of multimedia included in the Jail Management online training courses. Although hypothesized that video would enhance learning, motivation, and attitude, there were no differences in UNL student participants' test scores or surveys scores whether they received training with text only, audio with the text, or video with the text. Only instructional time differed among the groups.     

Formative research on the Heuristic Task Analysis process

Corporate and educational settings increasingly address more decision-making, problem-solving and other complex cognitive skills to handle complex cognitive, or heuristic, tasks, but the ever-increasing need for heuristic knowledge has outpaced the refinement of task analysis methods for heuristic expertise. Utilizing the Heuristic Task Analysis (HTA) process, a method developed for eliciting, analyzing, and representing expertise in complex cognitive tasks, a formative research study was conducted on the task of group counseling to further improve the HTA process. Implications of the findings include the need for incorporating various interview strategies and techniques, developing strategies for working with multiple experts, and considering the level of task expertise of the analyst. A revised version of the HTA process is presented based on these implications.     

Direct Instruction: a model of instructional design

Abstract Instructional design is chiefly concerned with the arrangement of the environment to optimise learning according to various criteria. While there are a number of instructional design theories and models, the Direct Instruction Model (Engelmann & Carnine, 1982) is the focus of this paper. The importance of instructional design is stressed and the Direct Instruction Model is analysed in terms of four components of a theory of instruction. A discussion of two of these components, structure and sequence, and an analysis of relevant research form the bulk of the paper. One goal of instructional training (be it for teachers or industry trainers) is the creative application of empirically verified instructional design principles so that the effectiveness of instruction, across a wide range of cognitive outcomes may be optimised.     

Teachers' intrinsic vs. extrinsic instructional goals predict their classroom motivating styles

We introduce the concept of teachers' intrinsic vs. extrinsic instructional goals and demonstrate its contribution to teachers' classroom motivating styles using independent samples across four studies. Based on self-determination theory, we hypothesized that the more teachers adopted intrinsic instructional goals the more they would rely on an autonomy-supportive motivating style, and the more they adopted extrinsic instructional goals the more they would rely on a controlling motivating style. Because no measure existed to assess intrinsic vs. extrinsic instructional goals, we created the new 4-scale, 16-item Teacher Goals Questionnaire (TGQ) in Study 1, using a pool of 72 candidate items and data from 212 fulltime K-12 teachers. In Study 2, we demonstrated the TGQ's construct and factorial validity by sampling 149 fulltime K-12 teachers. In Study 3, we tested our hypothesized model by sampling 147 fulltime K-12 teachers who reported their instructional goals on the TGQ and their motivating styles on two separate measures. Structural equation modeling analyses confirmed the hypothesized model. In Study 4, we replicated the findings from Study 3, using a multilevel sample (92 secondary teachers, 2749 students), a longitudinal research design, and student measures of teachers' motivating styles. The discussion focuses on instructional goals as key antecedents of teachers' classroom motivating styles.     

Dealing with graphic output from diagram processing tasks: Approaches to characterisation and analysis

The current relative lack of detailed knowledge about how learners interact with the types of diagrams used in instructional materials is partly due to the research difficulties in characterising the cognitive structures and processes involved in these interactions. One of these difficulties arises from the lack of research tools specifically designed for the collection and meaningful analysis of relevant data. This paper describes computer-based approaches developed to analyse and characterise graphic output produced during diagram processing tasks. Specializations: Mental representation and processing of scientific diagrams, characteristics of explanatory diagrams, visual aspects of problem solving, instructional design.     

Dynamic characteristics of mental models and dynamic visual displays

Mental models are hypothetical constructs for explaining human cognitive processes of understanding external reality, translating the reality into internal representation and utilizing it in problem solving. Three experiments were conducted to investigate important characteristics of mental models, their influence on task performance, and instructional strategies facilitating their formation. The experiments were conducted in computer-based training environments designed to teach troubleshooting electronic logic circuits. The results suggested: (a) dynamic characteristics of mental models are important for solving problems if understanding functional behaviors of the system is required to perform the task; (b) dynamic characteristics of mental models are determined primarily by subjects' understanding of the system features and functions more than by the visually presented training contents of the system; and (c) motion simulating system functions in visual displays is more effective than static visual displays in facilitating the formation of dynamic characteristics of mental models. Consequently, dynamic visual displays are more effective than static visual displays for teaching electronic troubleshooting skills. These findings provide direct implications for the development of training programs.     

Metacognition and Human Performance Improvement Originally published in 1988, PIQ 1.1

An understanding of the uses of metacognition provides a powerful tool toward developing quality and efficiency in task learning and performance. Metacognition is the executive controller of cognitive processes responsible both for self regulated learning and work performance outcomes. Comprehension monitoring and motivation are two prime components of metacognition which are responsible for commitment to, and definition of, goals, monitoring progress toward goals, and activating the appropriate thinking skills to achieve goals. Specific thinking skill categories of attention, organization, and elaboration are described with examples of how specific techniques in each category can optimize learning and job performance. Research on training of metacognitive skills through both detached and embedded skill-development programs is reviewed. In addition, ways to compensate for metacognitive skills when training is not cost-effective are described. The relationship between metacognitive skills and effective management of work performance is stressed.     

Metacognition and Human Performance Improvement

An understanding of the uses of meta-cognition provides a powerful tool toward developing quality and efficiency in task learning and performance. Metacognition is the executive controller of cognitive processes responsible both for self regulated learning and work performance outcomes. Comprehension monitoring and motivation are two prime components of metacognition which are responsible for commitment to, and definition of, goals, monitoring progress toward goals, and activating the appropriate thinking skills to achieve goals. Specific thinking skill categories of attention, organization, and elaboration are described with examples of how specific techniques in each category can optimize learning and job performance. Research on training of metacognitive skills through both detached and embedded skill-development programs is reviewed. In addition, ways to compensate for metacognitive skills when training is not cost-effective are described. The relationship between metacognitive skills and effective management of work performance is stressed.     

Video game learning dynamics: Actionable measures of multidimensional learning trajectories

Valid, accessible, reusable methods for instructional video game design and embedded assessment can provide actionable information enhancing individual and collective achievement. Cyberlearning through game‐based, metaphor‐enhanced learning objects (CyGaMEs) design and embedded assessment quantify player behavior to study knowledge discovery and application. CyGaMEs is grounded by analogical reasoning theory, cognitive task analysis and knowledge representation. A construct representation argument for validity using evidence‐centered design warrants CyGaMEs, its web‐based learning environment, Selene: A Lunar Construction GaME, its embedded assessment, and a learning dynamics approach to student, measurement and statistical models. Two studies (US volunteers, Study 1: n = 267, mean (M)_age = 15; Study 2: n = 90, M_age = 12) cross‐validate learning dynamics (learner progress, rate of progress and changes in that rate) for Selene's multidimensional goals while players learn and apply standard‐based science about fundamental geology and space science concepts. Gameplay data analyzed using regression, calculus and hierarchical linear modeling exhibit overall relatively high standardized rates of progress toward each goal statistically higher than zero. For example, adjusting for nesting within individual players, average rate of progress toward the goal of accreting lunar mass is a z‐score of 1.4 (99% confidence interval = 1.40_lower, 1.48_upper) or 1.4 standard deviations above zero.     

Resisting the sirens of temptation while studying: Using reappraisal to increase focus,enthusiasm, and performance

One of the major obstacles in the learning process is temptation, which has the power to divert students from even their most important goals (e.g. getting a degree). In two studies, we tested the hypothesis that cognitive reappraisal could be used to successfully resist temptation. Participants had to memorize tedious material while being tempted by pictures pasted on the wall (Study 1) or by funny clips on the television (Study 2). In Study 1, compared to a control group, participants who were instructed to reappraise the task as an opportunity to improve their memory (1) were less tempted by the pictures, (2) maintained their enthusiasm for the task, and (3) showed better performance in a subsequent memory test. Study 2 replicated and extended the findings from Study 1, showing that cognitive reappraisal is effective whether the target of reappraisal is the temptation itself, or the longer term goal. Taken together, our findings provide compelling evidence that cognitive reappraisal (of either the task or the temptation) may be a useful tool for increasing students' task performance and enthusiasm.     

Some Impressions of Competency-Based Training Programs

This article presents some impressions gained after two years of involvement in a competency-based training program in the pupil personnel services area, and it raises questions concerning the major features of such programs. The article includes discussion of program goals, the instructional process, assessment of student progress, and student attitudes. The danger of competency-based training becoming another fad in education is considered, and possible causes of this are noted.