Systematic Pilot Testing as a Step in the Instructional Design Process of Corporate Training and Development

Pilot testing assesses instructional materials with end users in the authentic learning environment prior to the implementation of the instruction and provides empirical support regarding the efficacy of the proposed instruction. Thus, subsequent decisions can be made from an informed position. This study determines the extent to which pilot tests are conducted in identified corporate training environments and ascertains reasons pilot tests were not implemented. A survey distributed to 34 individuals in positions of corporate training and development asked participants whether their design methodology and training development process systematically included pilot testing. Participants identified factors that influenced their decision to use a pilot test and described product evaluation in their organization. Survey findings indicate that in practice, designers perceive that pilot testing tends to impact the effectiveness of instructional products. Almost none of the participants pilot test materials 100% of the time, but the majority do so some of the time. A discrepancy exists between those who believe pilot testing is important and the percentage of their products which are pilot‐tested, indicating multiple factors that influence the decision to pilot test products.     

UNDERSTANDING TEST‐TYPE ASSIGNMENT: WHY DO SPECIAL EDUCATORS MAKE UNEXPECTED TEST‐TYPE ASSIGNMENTS?

We interviewed special educators (a) whose students with disabilities (SWDs) were proficient on the 2008 general education assessment but were assigned to the 2009 alternate assessment based on modified achievement standards (AA‐MAS), and (b) whose students with mild disabilities took the 2008 alternate assessment based on alternate achievement standards (AA‐AAS) and then the 2009 AA‐MAS. We explored teachers’ rationales for test‐type assignment, student characteristics, and quality of instruction to determine the test‐type decisions’ appropriateness. All teachers based their decisions on combinations of factors in the guidelines plus subjective and noninstructional factors. Findings raised concerns about the subjectivity of the assessment assignment system and the inappropriate grade‐level instruction for SWDs. Future research, implications of these findings, and limitations are discussed.     

Toward Effective Computer Use in High School Science Education: Where to from Here?

It is suggested widely that the use of computer technology holds great promise for school science education. Such a suggestion is congruent with broader beliefs that implementation of new technology is a key to improving education. However, little is known still about computer use in science classrooms and its effects on students' learning. This paper draws on major findings from recent research, integrates these with theoretical perspectives from associated education literature, and proposes guidelines for orienting computer use in high school science classrooms. It is proposed that (a) pedagogy should be strongly informed by appropriate theoretical orientations, (b) the importance of models in science should be acknowledged in pedagogy and in software development, (c) developing students' metacognition during instruction involving computers should be focussed upon, and (d) teachers' and students' beliefs and epistemologies should be recognised as key factors in educational change involving computer implementation and use.     

Systems,transfer, and fields: Evaluating a new approach to energy instruction

Energy is a central concept in science in every discipline and also an essential player in many of the issues facing people everywhere on the globe. However, studies have shown that by the end of K-12 schooling, most students do not reach the level of understanding required to be able to use energy to make sense of a wide range of phenomena. Many researchers have questioned whether the conceptual foundations of traditional approaches to energy instruction may be responsible for students' difficulties. In response to these concerns, we developed and tested a novel approach to middle school physical science energy instruction that was informed by the recommendations of the Framework for K-12 Science Education (National Research Council, 2012a) and the Next Generation Science Standards (NGSS) (NGSS Lead States, 2013). This new approach differs substantially from more traditional approaches to energy instruction in that it does not require energy forms and it emphasizes connections between energy, systems, and fields that mediate interaction-at-a-distance. We investigated student learning during this novel approach and contrasted it with student learning within a comparable unit based on a more traditional approach to energy instruction. Our findings indicate that students who learned in the new approach outperformed students who learned in the traditional approach in every quantitative and qualitative aspect considered in this study, irrespective of their prior knowledge of energy. They developed more parsimonious knowledge networks in relation to energy that focused primarily around the concept of energy transfer. This study warrants further investigation into the value of this new approach to energy instruction in both middle and high school.     

Strategic Teaching Frameworks: an Australian perspective

ABSTRACT

This paper addresses the concern that much educational multimedia is currently based upon technology rather than sound educational principles. The Strategic Teaching Frameworks (STFs) is a constructivist instructional strategy informed by two theories: anchored instruction and cognitive flexibility. This was deployed in the development of a CD‐ROM in an Australian University. The paper describes the development of a STF prototype and provides the reasoning behind decisions that had to be made. The authors outline some of the problems involved in their study and suggest implications for the use of STFs for teacher education and in the production of multimedia.     

Listening to Mothers II: Knowledge, Decision-Making, and Attendance at Childbirth Education Classes

In this column, a woman describes her concern that her childbirth classes did not provide the information she needed to make informed decisions during labor and birth. The results of the Listening to Mothers II survey suggest that this experience is not unusual. Although most women (97%) who participated in the survey wanted to know all or most of the potential risks of epidural, induction, and cesarean before consenting to have the intervention, the majority-including mothers who had experienced the intervention, women who were experienced mothers, and women who had attended childbirth classes-did not know the complications of induction or cesarean. These findings raise important questions about the outcomes of childbirth education. The factors that may contribute to these findings are discussed, and suggestions are made for insuring that women have the knowledge they need to make informed decisions about their maternity care.     

An interactional analysis of one-to-one pastoral care delivery within a primary school

Despite an interactional analysis being able to offer valuable insight into the institutional workings of pastoral care practice, pastoral care delivery remains largely unstudied. This paper will contribute new knowledge to the field of counselling and education by offering an interactional analysis of one-to-one pastoral care provision within a primary school. Much pastoral care practice is informed by theory, often accompanied by guidelines about how to deliver pastoral care activities effectively. The pastoral carer needs to convert these guidelines into talk in order to deliver the intervention as an interactional encounter. However useful these guidelines are, they cannot show what the actual delivery of those pastoral care activities might look like in real life. Using conversation analysis, we examine video recordings of pastoral care delivery to reveal the ways in which a pastoral carer supports a child’s behaviour, social and emotional well-being. The significance of the findings is that those who provide pastoral care can see in close detail what delivery might look like as a real-life encounter, imparting valuable knowledge that can then be applied alongside theory and guidelines to enhance professional practice. Of further significance is that the findings can also show how an interactional analysis of pastoral care work can be used to demonstrate social and emotional learning and that the work being done effectively supports children.     

Finding our way

Conclusion The guidelines presented here are based on my experience as a classroom teacher and a staff development coordinator for technology, and upon my study of the literature regarding the use of technology for instruction. I do not present these guidelines as a recipe for success in integrating the use of computers into instruction, nor as an exhaustive list of what must be done for integration to occur. To do so would be an oversimplification of a complex issue. There are many difficult decisions that schools need to make, and I offer these guidelines as a means for drawing upon the experience of those of us who are in the process of integrating technology into instruction. Even those of us who are heavily involved in this work feel as though we are just finding our way. I hope that what I have shared in this article will help others to find their way as well. He has taught math, science, and computer programming at the secondary level in the North Syracuse (New York) School District for the past nine years. During 1999–2000 school year, he coordinated the district’s staff development program for technology in addition to his teaching duties     

Accommodations for Students Who Are Deaf or Hard of Hearing in Large-Scale, Standardized Assessments: Surveying the Landscape and Charting a New Direction

Students who are deaf or hard of hearing (SDHH) often use test accommodations when they participate in large-scale, standardized assessments. The purpose of this article is to present findings from the Third Annual Survey of Assessment and Accommodations for Students who are Deaf or Hard of Hearing. The "big five" accommodations were reported by at least two-thirds of the 389 participants: extended time, small group/individual administration, test directions interpreted, test items read aloud, and test items interpreted. In a regression analysis, language used in instruction showed the most significant effects on accommodations use. The article considers these findings in light of a more proactive role for the National Survey in providing evidence for the effectiveness of accommodations with SDHH.     

“We are only looking for the right answers”: the challenges of controversial issue instruction in South Korea

This study explores the perceptions of South Korean moral and social studies education teachers, teacher educators, and students on the promises and challenges of introducing and discussing controversial issues within their classrooms. The findings suggest that college entrance exams, textbooks, didacticism, dualism, administrators, textbooks, standards, fear of bias, scant instructional time, and reticent classroom climates collectively undermine controversial instruction in South Korea The few pathways that counter these challenges include student-generated curriculum topics and tying lessons to current events. Similar to other democratic societies, South Korea's paucity of reflective thinking on controversial issues represents a clear hazard to the development of citizens who make informed and reasoned decisions for the common good.     

Student characteristics for online learning success

Higher education has been impacted significantly by the proliferation of online instruction. Despite this trend, empirical investigations of this delivery method have lagged. Many have documented the need for research on online student characteristics, yet current literature consists of personal teaching experiences and anecdotal observations. To address these issues the authors conducted three studies. The goals were to describe how the new measure of online student characteristics (i.e., TOOLS — Test of Online Learning Success) was constructed and validated, outline the contributions the measure has made to our knowledge of online learners, and review the research literature that supports the findings. The results demonstrated the measure's simple and stable structure, construct validity, criterion validity, internal reliable, and test–retest reliability. The measure, administration guidelines, scoring procedures, and outcome interpretations are provided in the public domain. The different purposes for which the measure may be used are discussed. The paramount student characteristics that emerged are presented and discussed in regards to other research findings.     

Teacher learning in Lesson Study: Affordances,disturbances, contradictions,and implications

Lesson Study is a form of professional development where teachers collaboratively design research lessons and improve instruction using the evidence they have gathered on student learning and development. This article reports on a professional learning team’s enactment of Lesson Study in an elementary school. A methodological approach informed by Cultural-Historical Activity Theory was adopted to study teacher learning practices that provided affordances to teacher learning, practices that produced disturbances to teacher learning, and underlying systemic contradictions revealed by the disturbances. The findings suggest implications for enhancing school-based professional development through Lesson Study.     

Promoting Metacognitive Decision-Making in Teacher Education

Effective teachers are characterized by their abilities to make thoughtful, deliberate, and informed adaptations while teaching (Hoffman & Pearson, 2000). These in-the-moment teaching decisions are guided by a complex web of teacher knowledge. Raising teachers' awareness of the decisions they make on a moment-by-moment basis may aid in thoughtful, more deliberate decision-making and more robust instruction. We charge that preservice teachers can acquire this skill, as well. This article highlights practices employed in preservice teacher education courses as a means to promote awareness of, and engagement with, in-the-moment teaching decisions. Using examples from mathematics, social studies, and literacy methods courses, we describe our processes to intentionally cultivate preservice teachers' adaptive teaching skills through metacognitive decision-making.     

Effect of adaptive advisement on perception in learner-controlled,computer-based instruction using a rule-learning task

The proposition that learner control of instruction can be facilitated by directly affecting the student’s perception of learning need was investigated. Students in one experimental condition were continuously advised during instruction of their learning performance and needs (amount and sequence of content) in relationship to the desired acquisition of rules at a given mastery criterion and were allowed to make instructional decisions. It was hypothesized that this condition would prove more effective on posttest performance than conditions without advisement — either partial learner control or conventional learner control. Results are discussed in reference to an information-processing approach to the design of computer-based instruction.     

Exploring Middle School Students’ Understanding of Three Conceptual Models in Genetics

Genetics is the cornerstone of modern biology and a critical aspect of scientific literacy. Research has shown, however, that many high school graduates lack fundamental understandings in genetics necessary to make informed decisions about issues and emerging technologies in this domain, such as genetic screening, genetically modified foods, etc. Genetic literacy entails understanding three interrelated models: a genetic model that describes patterns of genetic inheritance, a meiotic model that describes the process by which genes are segregated into sex cells, and a molecular model that describes the mechanisms that link genotypes to phenotypes within an individual. Currently, much of genetics instruction, especially in terms of the molecular model, occurs at the high school level, and we know little about the ways in which middle school students can reason about these models. Furthermore, we do not know the extent to which carefully designed instruction can help younger students develop coherent and interrelated understandings in genetics. In this paper, we discuss a research study aimed at elucidating middle school students’ abilities to reason about the three genetic models. As part of our research, we designed an eight-week inquiry unit that was implemented in a combined sixth- to eighth-grade science classroom. We describe our instructional design and report results based on an analysis of written assessments, clinical interviews, and artifacts of the unit. Our findings suggest that middle school students are able to successfully reason about all three genetic models.     

Boosting financial literacy: benefits from learning study

This paper reports on a study that employed a theory-based approach in the form of a learning study to enhance a domain-specific generic capability, financial literacy, of Grade 12 students to empower them to make informed and independent financial decisions. Financial literacy is seen in this study as a function of student understanding of a limited set of interrelated economic concepts that can be used as tools to assess financial situations and make sound financial decisions. Twelve teachers participated in the study. Six worked together in a learning study group and drew on a particular theory of learning in planning and carrying out lessons designed to serve the learning aims, whereas the other six worked collaboratively in a lesson study group. To evaluate the effectiveness of the two learning conditions created by the two groups, 193 students answered questions on complex, everyday financial situations in four tests: a pretest, a posttest following the research lessons, and delayed posttests 6 weeks and 6 months after instruction. The results showed that students in the learning study group outperformed their counterparts in the lesson study group in all three post-lesson tests, and that the inter-group performance gap was maintained or widened over time.     

Science Teachers' Elicitation Practices: Insights for Formative Assessment

In evaluating teachers' instructional decisions during instruction, it is clear that the nature of their elicitation is crucial for student learning. When instructional decisions are informed by information about students' conceptual understanding, significant learning is possible. This article examined the elicitation practices of two high school science teachers who indicated that they made instructional decisions based on the elicited evidence of students' knowledge but whose elicitation practices were characteristic of low-level elicitation. The teachers focused on students' responses that used canonical terms and expressed acceptable knowledge. The teachers demonstrated low-level responsiveness because they did not have full access to students' knowledge. The elicited evidence of students' knowledge that was used in making instructional decisions was not representative of students' conceptual understanding. There was, thus, a mismatch between the teachers' perspectives about their formative assessment practice and what is considered effective formative assessment.     

CBT OR IVD?—That’s the question

Summary Computer-Based Training is usually based upon modular branching paradigms familiar in print-based programmed learning materials. It successfully involves students actively in the instruction, but due to inherent limitations lacks the realism and immediacy of television. Interactive video is a recent development in instructional technology, merging of computers and television. This marriage has produced a hybrid technology which can provide the structure and involvement of computerassisted instruction and the realism of television. For interactive video to realize its potential, rigorous design and technical requirements must be satisfied. Educators and trainers can make informed decisions about the use of CBT versus IVD by studying the large body of research that deals with the instructional effectiveness of these technologies. Also, answering questions regarding learning outcomes, budget constraints, and other issues is critical before committing to one medium over the other. Knowing the advantages and disadvantages of each instructional system is key to selecting an appropriate and cost-effective instructional approach.     

Cognitive Activity‐Based Design Methodology for Novice Visual Communication Designers

The notion of design thinking is becoming more concrete nowadays, as design researchers and practitioners study the thinking processes involved in design and employ the concept of design thinking to foster better solutions to complex and ill‐defined problems. The goal of the present research is to develop a cognitive activity‐based design methodology for novice visual communication designers, which will be achieved by mapping the findings from a comparative analysis of novice and expert visual communication designers' thinking processes onto the prospective methodology. Under the proposed methodology, activity modes take place in a chronological flow under specific guidelines involving various forms of design cognition. The guidelines correlate to design phases from problem structuring to detailed design and to the cognitive processes of divergent and convergent thinking. The methodology gives open‐ended instruction to novices endeavouring to proceed with the design process, solve complex design problems and make better design decisions. This research has value for its unique approach to methodology development. Furthermore, the proposed methodology provides guidance for more effective cognitive activities during the design process and holds potential for implementation in design education due to its focus on the needs of novice designers.     

Student contribution in asynchronous online discussion: a review of the research and empirical exploration

The increasingly prevalent use of Internet in schools and homes has resulted in asynchronous online discussion becoming an increasingly common means to facilitate dialogue between instructors and students, as well as students and students beyond the boundaries of their physical classrooms. This article is organized into two main sections. In the first section, we review 50 empirical studies in order to identify the factors leading to limited student contribution. Limited student contribution is defined as students making few or no postings, or students exhibiting surface-level thinking or low-level knowledge construction in online discussions. We then identify the various empirically based guidelines to address the factors. In the second section, we discuss three potential guideline dilemmas that educators may encounter: (a) use of grades, (b) use of number of posting guideline, and (c) instructor-facilitation. These are guidelines where previous empirical research shows mixed results when they are implemented. Acknowledging the dilemmas is essential for educators and researchers to make informed decisions about the discussion guidelines they are considering implementing. Finally, we report two exploratory case studies on student-facilitation that we conducted. Using students as facilitators may be an alternative solution to educators who wish to avoid the instructor-facilitation guideline dilemma.