TradeSmith: An Exercise to Demonstrate the Illusion of Control in Decision Making

Resource allocation decisions are a fundamental class of problems common throughout a business and therefore are found throughout business school curricula. Entrepreneurs must allocate capital, financiers must allocation cash, and production managers must create the best mix of multiple‐use resources. Within this context, a business school's curriculum, instructional materials, and learning processes must consider the implications of individual decision making. Our traditional instructional content and delivery methods may effectively teach how to set up a decision “problem” and how to obtain an optimal answer. Yet, we may not be teaching a key underlying factor: that entrepreneurs, managers, and future leaders appear to have implicit cognitive biases, which discount information and skew individual decision making. The results of this study demonstrate that a phenomenon known as the “illusion of control” presents a fundamental challenge to the efficacy of formalized educational programs on decision making. TradeSmith was designed as a problem‐based learning exercise to elicit for the illusion of control in a basic, resource allocation, decision environment. The subjects in this study demonstrate a decision‐making pattern consistent with the “illusion of control” phenomenon. Finally, by revealing individuals' implicit design‐making paradigms, TradeSmith helps them experience key issues for managerial decision making.     

Effective literacy instruction for adults with specific learning disabilities: implications for adult educators

Adults with learning disabilities (LD) attending adult basic education, GED programs, or community colleges are among the lowest performers on measures of literacy. For example, on multiple measures of reading comprehension, adults with LD had a mean reading score at the third grade level, whereas adults without LD read at the fifth grade level. In addition, large numbers of adults perform at the lowest skill levels on quantitative tasks. Clearly, significant instructional challenges exist for adults who struggle with literacy issues, and those challenges can be greater for adults with LD. In this article, the literature on adults with LD is reviewed, and evidenced-based instructional practices that significantly narrow the literacy achievement gap for this population are identified. Primary attention is given to instructional factors that have been shown to affect literacy outcomes for adults with LD. These factors include the use of explicit instruction, instructional technology, and intensive tutoring in skills and strategies embedded in authentic contexts.     

Assessment of Reading Comprehension: matching Test Purposes and Test Properties

Reading comprehension is difficult to measure because it is a multifaceted construct influenced by a variety of cognitive, social and affective variables. There are also many distinct reasons for measuring reading comprehension such as the evaluation of instructional programs, the ordering of students by ability, and the diagnosis of reading difficulties. In this article we suggest that appropriate measures of reading comprehension depend on the fit between the purposes for testing and the properties of individual tests. Three test properties that we identify are statistical: stability of individual differences, consistency of an individual's scores across testing occasions, and sensitivity of the criterion variable to treatment or growth. Three other properties are conceptual in nature and pertain to test validity: nomothetic span, construct representation, and penetration. Each of these test properties and purposes is described, as well as the perils of mismatches among them. Comparative research on reading tests, construction of tests from theories and models, and use of testing portfolios can all improve the effective measurement of reading.     

Building Pre-service Teacher Capacity to Use External Assessment Data: An Intervention Study

In recent years, the use of data to inform instructional decision making has become a particularly prominent facet of K–12 educators' professional practice. However, research estimates limited opportunities for preservice teachers to learn how to use data, including standardized test data, for such purposes. In response, this article describes the results of a pretest–posttest study of a 6-hour standardized assessment data use intervention for preservice teachers. The facilitated, collaborative, and highly structured assessment course-embedded intervention engaged preservice teachers in asking and answering four different kinds of questions (e.g., achievement strengths and weaknesses, instructional implications) at five different student levels (e.g., individual, subgroup, school) with external assessment data presented in tables, charts, and score reports. Findings—which include highly favorable preservice teacher perceptions of the intervention's impact on their data-driven decision making skills, and changes in their self-efficacy and data interpretation skills—indicate that the intervention holds promise as a preservice teacher learning mechanism.     

Transfer Evaluation of Knowledge and Skill Development: A Case Study of an Instructional Systems Development Training Program

Program evaluation involves a systematic process to collect and convert data into information which is used to improve programs, measure their effects, track quality, act on results, and improve decision making. Beginning in 1992, the LG Group of Korea was involved in an instructional systems development (ISD) training program for its human resource development professionals in cooperation with Indiana University. The LG ISD training program stressed that learning instructional design theories and practices would make LG more effective in global competition. The purposes of the research reported in this article were to evaluate the effectiveness of the ISD training program in terms of transfer of training and to gain information on how to improve future training programs. The findings of the study support the goals that were established for the ISD training program. The program participants are more confident and competent in using the ISD process for analyzing needs for performance improvement and designing and developing quality training products. The article suggests ways of solving barriers to transfer training and evaluation and improving future programs.     

Reading research for students with LD: a meta-analysis of intervention outcomes

The present article provides a meta-analysis of instructional research with samples of children and adolescents with learning disabilities in the domains of word recognition and reading comprehension. The results of the synthesis showed that a prototypical intervention study has an effect size (ES) of .59 for word recognition and .72 for reading comprehension. Four important findings emerged from the synthesis: (a) Effect sizes for measures of comprehension were higher when studies included derivatives of both cognitive and direct instruction, whereas effect sizes were higher for word recognition when studies included direct instruction; (b) effect sizes related to reading comprehension were more susceptible to methodological variation than studies of word recognition; (c) the magnitude of ES for word recognition studies was significantly related to samples defined by cutoff scores (IQ > 85 and reading < 25th percentile), whereas the magnitude of ES for reading comprehension studies was sensitive to discrepancies between IQ and reading when compared to competing definitional criteria; and (d) instructional components related to word segmentation did not enter significantly into a weighted least square hierarchical regression analysis for predicting ES estimates of word recognition beyond an instructional core model, whereas small-group interactive instruction and strategy cuing contributed significant variance beyond a core model to ES estimates of reading comprehension. Implications related to definition and instructional components that optimize the magnitude of outcomes are discussed.     

Preparing preservice teachers to make instructional decisions: An examination of data from the teacher work sample

The purpose of this American study was to examine student teachers’ ability to make instructional decisions as they engaged in teaching. We examined the narrative accounts provided by 150 student teachers within their teacher work samples (TWSs). Results indicated that most student teachers were able to implement some aspects of instructional decision making, such as noting a specific difficulty with student learning and making an on-the-spot adjustment in their instruction. However, other elements of instructional decision making were less likely to be implemented. For example, student teachers relied on a limited range of formative assessment strategies and instructional modifications, and very few provided a sound rationale for their choice of modification. These results suggest that novice teachers may benefit from more opportunities to reflect and critique upon authentic learning experiences as they complete their teacher preparation programs. Finally, the TWS appears to represent a viable method for examining the instructional decisions of student teachers.     

Customizing Instruction to Maximize Functional Outcomes for Students With Profound Multiple Disabilities

This article describes a process for customizing instruction for students with profound multiple disabilities that has been used to design instructional programs and maximize the attainment of functional outcomes for students. The process focuses on collaborative teamwork and problem solving to design and implement instructional programs by ensuring that the 5 components of the process are included. Team members determine (a) prepositioning handling procedures; (b) overall body positioning for instruction; (c) hand, arm, and head positioning; (d) instructional adaptations and materials; and (e) handling procedures to combine with systematic instructional strategies.     

A meta-analysis of single-subject-design intervention research for students with LD

This article summarizes single-subject-design intervention studies that include students with learning disabilities. Effect sizes of 85 studies were analyzed across instructional domains (e.g., reading, mathematics); sample characteristics (e.g., age, intelligence); intervention parameters (e.g., number of instructional sessions, instructional components); and methodological procedures (e.g., internal validity, treatment integrity, sample representation). The major findings were as follows: (a) All domain areas except handwriting yielded effect sizes at or above Cohen's .80 threshold for a substantial finding; (b) instructional components related to drill-repetition-practice-review, segmentation, small interactive groups, and the implementation of cues to use strategies contributed significant variance (15%) to estimates of effect size; (c) strategy instruction (SI) models better predicted effect size estimates than direct instruction (DI) models when the results were qualified by the reported intellectual and reading levels of the participants; (d) high-IQ discrepancy groups yielded lower effect sizes compared to low-IQ discrepancy groups in the domain of reading, whereas the reverse effect occurred when treatment outcomes were not reading measures; and (e) the low-IQ discrepancy groups yielded higher effect sizes for a Combined DI and SI Model when compared to competing models. The results are supportive of the pervasive influence of cognitive strategy and direct instruction models across treatment domains and of the notion that variations in sample definition moderate treatment outcomes.     

DO INTERVENTION‐EMBEDDED ASSESSMENT PROCEDURES SUCCESSFULLY MEASURE STUDENT GROWTH IN READING?

Effective intervention delivery requires ongoing assessment to determine whether students are learning at the desired rate. Intervention programs with embedded assessment procedures (i.e., assessment that occurs naturally during the process of delivering intervention) can potentially enhance instructional decisions. However, there is almost no psychometric research on this type of assessment procedure. This study was designed to examine the psychometric characteristics of three types of progress measures that are embedded within a commonly used reading intervention program. Results indicated that generalized gains across different oral reading fluency passages predict concurrent gains on common and comprehensive tests of reading fluency, and that immediate instructional gains measured during instruction were significantly different from zero and thus sensitive to intervention effects. Overall findings suggest that at least some embedded assessment procedures demonstrate predictive validity and that these types of procedures have the potential to assist educators with data‐driven instructional decisions about students’ responsiveness to intervention.     

Forecasting reading anxiety for promoting English-language reading performance based on reading annotation behavior

To reduce effectively the reading anxiety of learners while reading English articles, a C4.5 decision tree, a widely used data mining technique, was used to develop a personalized reading anxiety prediction model (PRAPM) based on individual learners' reading annotation behavior in a collaborative digital reading annotation system (CDRAS). In addition to forecasting immediately the reading anxiety levels of learners, the proposed PRAPM can be used to identify the key factors that cause reading anxiety based on the fired prediction rules determined by the developed decision tree. By understanding these key factors that cause reading anxiety, instructors can apply reading strategies to reduce reading anxiety, thus promoting English-language reading performance. To assess whether the proposed PRAPM can assist instructors in reducing the reading anxiety of learners, this study applies the quasi-experimental method to compare the learning performance of three learning groups, which are, respectively, supported by a CDRAS with individual annotations, collaborative annotations, and collaborative annotations with online instructor's support to reduce reading anxiety by the proposed PRAPM. The instructional experiment was conducted on Grade 7 students at Taipei Municipal Wan-Fang high school. Experimental results indicate that the average correct prediction rate of the proposed PRAPM in identifying the reading anxiety levels of learners was as high as 70%. Moreover, analytical results show that the collaborative annotation with online instructor's support for reducing reading anxiety by the proposed PRAPM indeed helps learners reduce reading anxiety, particularly for the male learners, showing that gender difference exists. Furthermore, based on online instructor's support for reducing reading anxiety by the proposed PRAPM, the correlation analysis also shows that the online instructor's interaction with the male learners is significantly correlated with the reading anxiety reduction. Furthermore, English-language learning performance of the three learners groups, which were given a CDRAS with different learning mechanisms, was significantly promoted.     

Relative Effectiveness of Reading Intervention Programs for Adults with Low Literacy

To compare the efficacy of instructional programs for adult learners with basic reading skills below the seventh grade level, 300 adults were randomly assigned to one of three supplementary tutoring programs designed to strengthen decoding and fluency skills, and gains were examined for the 148 adult students who completed the program. The three intervention programs were based on or adapted from instructional programs that have been shown to benefit children with reading levels similar to those of the adult sample. Each program varied in its relative emphasis on basic decoding versus reading fluency instruction. A repeated measures MANOVA confirmed small to moderate reading gains from pre- to post-testing across a battery of targeted reading measures, but no significant relative differences across interventions. An additional 152 participants who failed to complete the intervention differed initially from those who persisted. Implications for future research and adult literacy instruction are discussed.     

Choosing and using multiple information sources: Some new findings and emergent issues

This commentary highlights important contributions of four empirical investigations of how people sort through the masses of information that are now available to them and find what they need to solve some problem or make some decision. These studies demonstrate the impact of differences among information sources and the role that these differences play in people's evaluations of sources and use of the information to make decisions. Future research needs to examine changes in judgments of source credibility and information relevance as people acquire more knowledge of a topic and learn more sophisticated strategies for using multiple sources. Future studies of instructional effects on searching, sourcing and integrating multiple sources will be most informative if they collect multiple measures of processing and use of information in decision making and problem solving.     

Using silent motion pictures to teach complex syntax to adult deaf readers

This research tested whether silent motion pictures could be a source of contexts that fostered comprehension of relative clause and passive voice sentences during reading. These two syntactic structures are chronically difficult for some deaf readers. According to the instructional strategy, while subjects watched silent comedy stories, the video display intermittently focused attention on short segments of action and then called for a decision regarding which of two sentences printed in a workbook described the action segment. After this, a display on the video screen provided feedback on the accuracy of the decision. If successful here, this approach might be applied to other areas of competence in order to elevate the generally low level of reading performance by many deaf students. The study applied a single subject design in order to measure sentence comprehension accuracy before and following use of the materials. The computerized testing procedure also measured sentence reading time, an index of attention use. Thus, these data allowed an examination of whether any increases in comprehension were associated with slower, more laborious rates of reading. The instructional approach was an indirect one sharing multiple aspects of whole language methodology, and the sample included deaf subjects at a variety of reading ability levels. This permitted examination of whether an indirect instructional approach could be successful with readers demonstrating relatively low reading ability. The central research question of the study was the following 'Can this instructional method be effective with deaf readers?'.     

Instructing Struggling Older Readers: A Selective Meta‐Analysis of Intervention Research

This article synthesizes the experimental literature on reading interventions for upper elementary and middle school students identified with reading disabilities on norm‐referenced reading measures. Ten studies (12 independent samples) yielded 70 effect sizes on norm‐referenced reading measures with an aggregated mean of 0.41 (SE= .04) in favor of the experimental condition. Moderate effect sizes emerged on norm‐referenced measures of word identification (M= 0.41), decoding (M= 0.43), and comprehension (M= 0.73) and low effect sizes for fluency (M=–.29). Intervention outcomes did not significantly vary as a function of the reading skills measured, type of reading instruction, and/or variations in sample characteristics. Studies yielding low and relatively moderate effect sizes shared a number of instructional components. Overall, the magnitude of the results for experimental reading intervention studies for students with reading disabilities in the middle school age range was small to moderate. Implications of the study were discussed.     

Teachers' judgments of students' early literacy skills measured by the Early Literacy Skills Assessment: Comparisons of teachers with and without assessment administration experience

Teacher judgments about students' academic abilities are important for several reasons, including their day‐to‐day instructional decision making. Not surprisingly, previous studies have investigated the accuracy of teachers' judgments about their students' reading abilities. Previous research, however, has not investigated teachers' judgments about students' early literacy skills, nor has previous research systematically examined how teachers' training and use of an objective assessment instrument impact their judgments of students' performance on that instrument. This exploratory study offers the first investigation of teachers' judgment accuracy of pre‐kindergarten students' early literacy skills, and compares the judgment accuracy of teachers with and without opportunities to administer the Early Literacy Skills Assessment (ELSA). Findings suggest that teachers with opportunities to administer the ELSA had a significantly higher percentage of accurate judgments across half of the ELSA subtests, but their judgment accuracy was still no better than 50%–60% on all but one ELSA subtest. Implications for school psychologist practitioners and researchers are presented. © 2010 Wiley Periodicals, Inc.     

Live and Learn: Informal Learning Among Future Professionals

Although most workplace learning occurs informally, instructional design and technology professionals often focus their attention on more formal interventions, such as training. This article addresses a research study that examined the perception and use of informal learning strategies among instructional design and technology graduate students, the future members of the workforce of instructional designers and performance improvement practitioners. Results of this study may have implications for both instructional design and technology graduate programs and managers of instructional design and technology teams in organizations.     

Reading Instruction for English Learners in the Middle Grades: a Meta-Analysis

This meta-analysis synthesizes the last two decades of experimental and quasi-experimental research on reading instruction across academic contexts (e.g., social studies, science, mathematics, English language arts) for English learners (ELs) in grades 4 through 8, to determine (a) the overall effectiveness of reading instruction for upper elementary and middle school students who are ELs and (b) how the magnitude of the effect varies based on student, instructional, and study characteristics. The analysis included a total of 11 studies with 46 individual effect sizes and yielded a mean effect size of g?=?0.35 across all (i.e., standardized and unstandardized) reading measures, g?=?0.01 across standardized reading measures, and g?=?0.43 across unstandardized reading measures. For all reading, unstandardized reading, all vocabulary, and unstandardized vocabulary measures, results suggest that higher quality studies tended to have smaller effects, and these effects were even more evident for unstandardized measures (i.e., one unit increase in study quality was associated with decreased effects: g?=?0.21, g?=?0.30, g?=?0.24, g?=?0.30, respectively). For all comprehension measures, effects were larger for instruction that included both vocabulary and comprehension (g?=?0.39) than for instruction that focused on vocabulary alone (g?=?0.08). Results suggest the benefit of developing and refining high-impact approaches to reading instruction for ELs that can be delivered across content areas and grades.     

A classification of the ISIS program using Bloom's cognitive taxonomy

This article focuses on the practical use of Bloom's Taxonomy of Educational Objectives. The current status of analyzing and classifying test items and behavioral objectives was examined in this study. Specifically, the purpose of this study was to analyze and classify the ISIS minicourse performance objectives and criterion-referenced test items according to Bloom's cognitive Taxonomy in order to determine what levels of cognition the ISIS instructional materials are directed. The performance objectives and test items of thirty-three ISIS minicourses and criterion-referenced tests were collected and classified. Four research questions were posed in the study. The findings indicate that ISIS minicourse test items and performance objectives are written primarily at the Knowledge and Comprehension levels. The ISIS instructional materials reflect low percentages of upper cognitive level test items and performance objectives. Based upon the use of a chi-square analysis, twenty-four of the ISIS minicourses and tests demonstrate a positive congruence between their performance objectives and criterion-referenced test items. Nine ISIS minicourses were found to demonstrate a negative relationship between their performance objectives and test items. Implications and Recommendations based on the findings of the studies are provided.