Instructional Sensitivity of a State's Standards-Based Assessment

Abstract

The accuracy of achievement test score inferences largely depends on the sensitivity of scores to instruction focused on tested objectives. Sensitivity requirements are particularly challenging for standards-based assessments because a variety of plausible instructional differences across classrooms must be detected. For this study, we developed a new method for capturing the alignment between how teachers bring standards to life in their classrooms and how the standards are defined on a test. Teachers were asked to report the degree to which they emphasized the state's academic standards, and to describe how they taught certain objectives from the standards. Two curriculum experts judged the alignment between how teachers brought the objectives to life in their classrooms and how the objectives were operationalized on the state test. Emphasis alone did not account for achievement differences among classrooms. The best predictors of classroom achievement were the match between how the standards were taught and tested, and the interaction between emphasis and match, indicating that test scores were sensitive to instruction of the standards, but in a narrow sense.     

Approaches to teaching in mainstream and separate postsecondary classrooms

The study examined attitudes toward teaching reported by university instructors who normally teach hearing students (with the occasional deaf or hard of hearing student) and by instructors who normally teach deaf and hard of hearing students at the same institution. Overall, a view of instruction as information transmission was associated with a teacher-focused approach to instruction, whereas viewing instruction as a means of promoting conceptual change was associated with a student-focused approach. Instructors in mainstream classrooms were more oriented toward information transmission than conceptual change, whereas instructors experienced in separate classrooms for deaf and hard of hearing students reported seeking to promote conceptual change in students and adopting more student-focused approaches to teaching. These results are consistent with previous findings concerning instructors' approaches to teaching and deaf and hard of hearing students' approaches to learning, and may help explain recent findings regarding student outcomes in separate versus mainstream secondary classrooms.     

Factors impacting teachers' argumentation instruction in their science classrooms

Science education research, reform documents and standards include scientific argumentation as a key learning goal for students. The role of the teacher is essential for implementing argumentation in part because their beliefs about argumentation can impact whether and how this science practice is integrated into their classroom. In this study, we surveyed 42 middle school science teachers and conducted follow-up interviews with 25 to investigate the factors that teachers believe impact their argumentation instruction. Teachers responded that their own learning goals had the greatest impact on their argumentation instruction while influences related to context, policy and assessment had the least impact. The minor influence of policy and assessment was in part because teachers saw a lack of alignment between these areas and the goals of argumentation. In addition, although teachers indicated that argumentation was an important learning goal, regardless of students' backgrounds and abilities, the teachers discussed argumentation in different ways. Consequently, it may be more important to help teachers understand what counts as argumentation, rather than provide a rationale for including argumentation in instruction. Finally, the act of trying out argumentation in their own classrooms, supported through resources such as curriculum, can increase teachers' confidence in teaching argumentation.     

Evaluating the impact of differentiated instruction on literacy and reading in mixed ability classrooms: Quality and equity dimensions of education effectiveness

This article presents the findings of a quasi-experimental study that evaluated the effect of differentiated instruction on students’ learning, in mixed ability classrooms. Participants in the study were 24 teachers and 479 grade-four elementary students. Results indicate that in classrooms where differentiated instruction methods were systematically employed, students made better progress compared to students in classrooms where differentiated instruction methods were not employed, the family's socioeconomic status did not lead to differentiation in students’ achievement and the quality of differentiated teaching had a corresponding effect on students’ achievement. Based on these findings, the article discusses the significance of the systematic employment of differentiated instruction methods in mixed ability classrooms for promoting equity, optimization of quality and effectiveness in teaching.     

Including the kids across the hall: collaborative instruction of hearing, deaf, and hard-of-hearing students

This study follows graduate interns and their cooperating teachers through a collaborative student teaching experience in D/HH and hearing classrooms at the same grade level. Teacher teams, enrolled in a graduate course on collaboration, designed and conducted collaborative instruction (K-junior high school level), which focused on cooperative learning activities. Measures included student evaluations and separate intern and teacher evaluations of student performance, the integrated instructional units, and professional collaboration. Student evaluations included positive responses to integrated instruction by both groups, though there was more trepidation expressed by some D/HH students. Teachers reported very positive outcomes for all students including increased motivation to learn about and to interact with the other group and, for D/HH students, an increase in socially and academically appropriate behaviors. Specific student needs for training prior to integrated experiences were noted. Teachers stressed the importance of topic selections that would involve the expertise of all participants equally, needs for teacher in-service training and, administrative support for collaborative planning.     

Practices more likely to occur in kindergarten mathematics teachers’ instruction when English learners are present

Considering the rising proportion of English learners (ELs) in general education, little is known about how this change in classroom composition relates to the frequency of instructional activities and skills. This study uses a multimodal perspective on learning mathematics in a multilingual context, which recognizes the diverse modes of communication that students draw on in mathematics classrooms. We examined national data using quantitative analyses and investigated the mathematics practices that were more likely to occur in kindergarten classrooms that did and did not have ELs. We found teachers reported greater frequency of manipulatives and count/calendar activities in mathematics instruction when there were EL students in the classroom. Teachers in classrooms with ELs also reported a greater frequency of mathematics content that involved teaching shapes and practical skills. The findings indicate that teachers tended towards a supportive multimodal approach to instruction when ELs were present.     

Assessment Data-Informed Guidance to Individualize Kindergarten Reading Instruction: Findings from a Cluster-Randomized Control Field Trial

The purpose of this cluster-randomized control field trial was to was to examine the extent to which kindergarten teachers could learn a promising instructional strategy, wherein kindergarten reading instruction was differentiated based upon students' ongoing assessments of language and literacy skills and documented child characteristic by instruction (CXI) interactions; and to test the efficacy of this differentiated reading instruction on the reading outcomes of students from culturally diverse backgrounds. The study involved 14 schools and included 23 treatment (n = 305 students) and 21 contrast teacher (n = 251 students). Teachers in the contrast condition received only a baseline professional development that included a researcher-delivered summer day-long workshop on individualized instruction. Data sources included parent surveys, individually administered child assessments of language, cognitive, and reading skills and videotapes of classroom instruction. Using Hierarchical Multivariate Linear Modeling (HMLM), we found students in treatment classrooms outperformed students in the contrast classrooms on a latent measure of reading skills, comprised of letter-word reading, decoding, alphabetic knowledge, and phonological awareness (ES = .52). Teachers in both conditions provided small group instruction, but teachers in the treatment condition provided significantly more individualized instruction. Our findings extend research on the efficacy of teachers using Individualized Student Instruction to individualize instruction based upon students' language and literacy skills in first through third grade. Findings are discussed regarding the value of professional development related to differentiating core reading instruction and the challenges of using Response to Intervention approaches to address students' needs in the areas of reading in general education contexts.     

Professional concerns of beginning teachers of deaf and hard of hearing students

The professional concerns of beginning teachers of students who are deaf or hard of hearing were examined. Five first-year teachers of deaf and hard of hearing students served as participants. Two of the participants were itinerant teachers; three taught in self-contained classrooms. Participants were selected from programs serving deaf and hard of hearing students in rural and urban areas of the midwestern and southwestern United States. To interview the study participants, the researcher used an in-depth phenomenological method employing semi-structured questions and guided by a constructivist paradigm. Data were analyzed using qualitative analysis strategies (Bogdan & Biklen, 1992; Miles & Huberman, 1994). Results showed that concerns of beginning teachers of deaf and hard of hearing students are specific to service delivery models and geography. Participants provided specific recommendations for addressing the concerns of beginning teachers of deaf and hard of hearing students.     

Successful students who are deaf in general education settings

The study identified successful students who were deaf and were receiving most of their educational services in general education settings, in order to examine factors contributing to their success. Teachers in a western state were asked to nominate students who were deaf who were in the upper elementary through high school grades and were receiving most of their educational services in general education classrooms. Qualitative procedures were used to gather information on 20 successful students who were deaf. Inquiry focused on observation of the students in general education settings and interviews to gather perceptions of (a) the successful students themselves, (b) deaf education teachers, educational interpreters, and paraprofessional note takers serving these students, (c) general education teachers working with these students, and (d) parents. The students' primary communication modes were closely divided between sign language and spoken English; communication mode did not seem to be a salient factor in success. Results of the interviews with each group, a summary of observations, and themes that emerged across groups are provided.     

Integration of computers and related technologies into deaf education teacher preparation programs

The study examined how computer technology is used, modeled, and taught in programs that prepare future teachers of students who are deaf or hard of hearing. Data were collected through two self-reported survey instruments sent to participants from two different groups: (a) deaf education faculty (n = 233) and (b) administrators of programs serving students who are deaf or hard of hearing (n = 100). Among the faculty who responded to the survey, 99% said they used computers and printers for instruction during class, 84% said they used VCRs for instruction during class, and 56% said they used video cameras for instruction during class. The information that was reported on how technology was being used for instruction and how its use was being taught to students in deaf education teacher preparation programs indicates that the need persists for integration of technology into these programs. A need likewise continues for improved instructional strategies relative to computers and technology.     

Using online glossing lessons for accelerated instruction in ASL for preservice deaf education majors

Teachers of deaf and hard of hearing students must serve as language models for their students. However, preservice deaf education teachers typically have at most only four semesters of American Sign Language (ASL) training. How can their limited ASL instructional time be used to increase their proficiency? Studies involving deaf and hard of hearing students have revealed that glosses (written equivalents of ASL sentences) can serve as "bridges" between ASL and English. The study investigated whether glossing instruction can facilitate hearing students' learning of ASL. A Web site was developed in which ASL glossing rules were explained and glossing exercises provided. Posttest scores showed the experimental group improving from 39% to 71% on ASL grammar knowledge. These findings indicate that online glossing lessons may provide the means to obtain ASL skills more readily, thus preparing deaf education teachers to serve as ASL language models.     

Suggestions for preparing itinerant teachers: a qualitative analysis

Increasing numbers of students who are deaf or hard of hearing receive educational services in general education classrooms. This placement shift has altered the way teachers of students who are deaf or hard of hearing work, causing an increase in the number of itinerant teachers. As placement trends for students who are deaf or hard of hearing and teachers' job responsibilities have changed, the field of deaf education has only slightly modified professional standards for licensed teachers of students who are deaf or hard of hearing. Most teacher preparation programs continue training preservice teachers to work in self-contained classrooms, leaving itinerant teachers feeling underprepared. Interviews were conducted with 25 experienced itinerant teachers to determine which content and experiences should be included in preparation programs for preservice teachers of students who are deaf or hard of hearing who plan to become itinerant teachers. Results indicate that changes in course work and practical are necessary to best prepare these teachers.     

Time engaged in reading. A critical factor in reading achievement.

The reading achievement of deaf children may be low not only as a result of factors related to the hearing loss, such as a lag in language development. Environmental factors such as the quantity and quality of reading instruction, for example, may also cause low reading achievement. This study looked at the amount of time spent reading and the types of teacher interactions during reading instruction in classrooms at a school for deaf children and associated satellite classes in New Zealand. It was found that the deaf children spent very low levels of time engaged in reading and were subjected to teacher interactions that may inhibit the development of meaning-based reading skills. The quantity and quality of reading instruction for deaf children may differ from that experienced by most hearing children in New Zealand.     

Benefits of sign language interpreting and text alternatives for deaf students' classroom learning

Four experiments examined the utility of real-time text in supporting deaf students' learning from lectures in postsecondary (Experiments 1 and 2) and secondary classrooms (Experiments 3 and 4). Experiment 1 compared the effects on learning of sign language interpreting, real-time text (C-Print), and both. Real-time text alone led to significantly higher performance by deaf students than the other two conditions, but performance by deaf students in all conditions was significantly below that of hearing peers who saw lectures without any support services. Experiment 2 compared interpreting and two forms of real-time text, C-Print and Communication Access Real-Time Translation, at immediate testing and after a 1-week delay (with study notes). No significant differences among support services were obtained at either testing. Experiment 3 also failed to reveal significant effects at immediate or delayed testing in a comparison of real-time text, direct (signed) instruction, and both. Experiment 4 found no significant differences between interpreting and interpreting plus real-time text on the learning of either new words or the content of television programs. Alternative accounts of the observed pattern of results are considered, but it is concluded that neither sign language interpreting nor real-time text have any inherent, generalized advantage over the other in supporting deaf students in secondary or postsecondary settings. Providing deaf students with both services simultaneously does not appear to provide any generalized benefit, at least for the kinds of materials utilized here.     

Teachers' learning to facilitate high‐level student thinking: Impact of a video‐based professional development

Attaining the vision for science teaching and learning emphasized in the Framework for K‐12 Science Education and the next generation science standards (NGSS) will require major shifts in teaching practices in many science classrooms. As NGSS‐inspired cognitively demanding tasks begin to appear in more and more science classrooms, facilitating students' engagement in high‐level thinking as they work on these tasks will become an increasingly important instructional challenge to address. This study reports findings from a video‐based professional development effort (i.e., professional development [PD] that use video‐clips of instruction as the main artifact of practice to support teacher learning) to support teachers' learning to select cognitively demanding tasks and to support students' learning during the enactment of these tasks in ways that are aligned with the NGSS vision. Particularly, we focused on the NGSS's charge to get students to make sense of and deeply think about scientific ideas as students try to explain phenomena. Analyses of teachers' pre‐ and post‐PD instruction indicate that PD‐participants began to adopt instructional practices associated with facilitating these kinds of student thinking in their own classrooms. The study has implications for the design of video‐based professional development for science teachers who are learning to facilitate the NGSS vision in science classrooms.     

Potential or problem? An investigation of secondary school teachers’ attributions of the educational outcomes of students with specific learning difficulties

Despite strong support for inclusive education in principle, many teachers and administrators still demonstrate mixed responses to the inclusion of certain students in their classrooms. Students with specific learning difficulties (SpLD) form a large group of students in inclusive classrooms yet some provincial, state and national jurisdictions fail to acknowledge the existence of these students. Not acknowledging and understanding these students can deny them the recognition and resources necessary for their genuine participation in education and, in turn, society. The aim of this study was to examine British in-service secondary teachers’ attributional responses to students with and without specific learning difficulties. The participants included 122 British secondary school teachers who were surveyed in response to vignettes of hypothetical male students who had failed a class test. The study found that while teachers attributed more positive causes towards students without SpLD, they exhibited more negative causes towards students with SpLD. Teachers’ causal attributional outcomes of students’ level of achievement can impact upon the students’ own attributions, with teachers’ responses for students with SpLD having the potential to, unintentionally, influence students’ own sense of self-efficacy and motivation. The paper concludes with a consideration of the implications of the research and recommendations for practice.     

Discrete mathematics in deaf education: a survey of teachers' knowledge and use

The study documents what deaf education teachers know about discrete mathematics topics and determines if these topics are present in the mathematics curriculum. Survey data were collected from 290 mathematics teachers at center and public school programs serving a minimum of 120 students with hearing loss, grades K-8 or K-12, in the United States. Findings indicate that deaf education teachers are familiar with many discrete mathematics topics but do not include them in instruction because they consider the concepts too complicated for their students. Also, regardless of familiarity level, deaf education teachers are not familiar with discrete mathematics terminology; nor is their mathematics teaching structured to provide opportunities to apply the real-world-oriented activities used in discrete mathematics instruction. Findings emphasize the need for higher expectations of students with hearing loss, and for reform in mathematics curriculum and instruction within deaf education.