Teachers' judgments of students' early literacy skills measured by the Early Literacy Skills Assessment: Comparisons of teachers with and without assessment administration experience

Teacher judgments about students' academic abilities are important for several reasons, including their day‐to‐day instructional decision making. Not surprisingly, previous studies have investigated the accuracy of teachers' judgments about their students' reading abilities. Previous research, however, has not investigated teachers' judgments about students' early literacy skills, nor has previous research systematically examined how teachers' training and use of an objective assessment instrument impact their judgments of students' performance on that instrument. This exploratory study offers the first investigation of teachers' judgment accuracy of pre‐kindergarten students' early literacy skills, and compares the judgment accuracy of teachers with and without opportunities to administer the Early Literacy Skills Assessment (ELSA). Findings suggest that teachers with opportunities to administer the ELSA had a significantly higher percentage of accurate judgments across half of the ELSA subtests, but their judgment accuracy was still no better than 50%–60% on all but one ELSA subtest. Implications for school psychologist practitioners and researchers are presented. © 2010 Wiley Periodicals, Inc.     

Assessment of mathematics and reading performance: An examination of the correspondence between direct assessment of student performance and teacher report

Teacher judgments have been identified as a primary source of information regarding student academic achievement. Research examining the accuracy of teachers' judgments in assessing students' academic abilities has shown relatively high accuracy. However, previous studies have relied primarily on norm‐referenced measures to obtain estimates of students' achievement in reading and mathematics. Recent developments in the assessment of students' academic skills, such as Curriculum‐Based Measurement (CBM), provide a direct estimate of students' skill levels in basic areas such as reading and mathematics. The purpose of the present study was to examine the extent to which teachers' perceptions of students' reading and mathematics skills corresponded to direct estimates of students' reading and mathematics skills. Two second‐grade teachers estimated the reading and mathematics skills of 33 second‐grade students. Results of this study indicated that teachers were not accurate in assessing their students' mathematics functioning. Teachers were more accurate in assessing the occurrence of Mastery mathematics levels in basic addition, but were very inaccurate in assessing the occurrence of Mastery, Instructional, or Frustrational mathematics levels in all other skills assessed. In reading, teachers' judgment accuracy varied as a function of grade‐level material and instructional level. Specifically, teachers experienced considerable difficulty accurately identifying students who were reading at a Mastery level in grade‐level or above‐grade‐level material. © 2006 Wiley Periodicals, Inc. Psychol Schs 43: 247–265, 2006.     

Examining the accuracy of teachers' judgments of DIBELS performance

The purpose of this investigation was to examine the accuracy of teachers' judgments of students' early literacy skills and to determine if students' achievement levels influenced teachers' judgments. Typical and lower‐achieving kindergarten and first‐grade students' scores on the Nonsense Word Fluency and Phoneme Segmentation Fluency measures of the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) were compared to the predicted performance by their teachers. Results showed a moderately strong correlation between teachers' judgments and students' performance across all of the students, which is consistent with past research. Teachers' judgments, however, consistently and significantly overestimated the actual performance of students, particularly those who were typically performing students. The findings of the investigation suggest that relying on teachers' judgments of students' early literacy skills alone may be insufficient to accurately identify students at risk for reading difficulties. © 2011 Wiley Periodicals, Inc.     

Sex,School, and Society

Accurate teacher judgment of students' reading ability is critical for identification of students in need of intervention. Little research is available that examines the accuracy of teachers' judgments in identifying students at risk for early reading difficulty as students begin school. Findings of this study indicate that teachers are fairly accurate in their judgments, but the fact that reading ability provides the foundation for academic success suggests the need for a high degree of accuracy in their judgments of students' reading ability.     

Should teachers be accurate or (overly) positive? A competitive test of teacher judgment effects on students’ reading progress

Previous findings on effects of teachers' judgments on student learning have been contradictory leading to the question of what kinds of judgments are most beneficial: accurate or (overly) positive ones? In this study, we provide the first competitive test of prominent but contradictory hypotheses regarding the consequences of teachers' judgments in the context of reading proficiency using reading fluency and reading comprehension performance judgments from 145 teachers and measures of real performance and learning progress across eight points of measurement from 2880 students. Response Surface Analyses combined with an information-theoretic approach for model comparison revealed no evidence of positive effects of judgment accuracy or overestimation of student performance by teachers. Instead, progress in reading fluency and reading comprehension was best predicted by students' prior achievement. For reading comprehension, the positivity of teachers' judgments was additionally beneficial: The higher a teacher judged a student's performance, the more the student learned.     

Comparing global judgments and specific judgments of teachers about students' knowledge: Is the whole the sum of its parts?

Teachers' judgments about students' knowledge and skills can be global or specific depending on the diagnostic situation during teaching. We test the relationship between these judgments, their accuracy, and whether global judgment (GJ) accuracy can be measured by aggregating specific judgments (SJ). Judgments of 52 primary school teachers about their students' achievement in a standardized mathematics test were assessed. SJs and GJs correlated high. However, SJs were slightly more accurate than GJs. Additionally, teachers' GJ accuracy is not similar to the accuracy of aggregated SJs. We conclude that teachers use different judgment strategies for GJs and SJs.     

General education teacher judgments of their low‐performing students' short‐term reading progress

This study investigated the accuracy of classroom teachers' judgments of the reading progress of their low‐performing students. Participants were 36 second grade teachers and students in their lowest reading groups (n = 150). Student progress was monitored weekly using reading‐curriculum‐based measurement (R‐CBM) procedures. After 6 weeks, teachers were asked to rate their students' progress. Expert judges later reviewed the teachers' R‐CBM graphs and rated the individual and group progress based on the graphs. Teacher ratings did not correlate with expert ratings or the R‐CBM slope estimates. Expert ratings correlated highly with slope estimates. Teachers' estimates of progress were significantly higher than expert judges' ratings, indicating that teachers may overestimate student progress. Implications for practice and future research are discussed. © 2008 Wiley Periodicals, Inc.     

Progress in International Reading Literacy Study 2006 (PIRLS): pedagogical correlates of fourth‐grade students in Hong Kong

Reading literacy of fourth‐grade students in Hong Kong showed a remarkable improvement from 2001 to 2006 as shown by international PIRLS studies. This study identified various aspects of the teacher factor contributing to the significant improvement among students. A total of 4,712 students and 144 teachers from 144 schools were randomly selected using probability proportional‐to‐size technique to receive the Reading Assessment Test and complete the Teacher's Questionnaire, respectively. A number of items pertaining to teachers' instructional strategies and activities, opportunities for students to read various types of materials, practices on assessment, and professional preparation and perception, were found to be significantly correlated with the outcome of students' reading literacy. Stepwise regression procedure revealed four significant predictors for students' overall reading achievement. The most powerful predictor was the use of materials from other subjects as reading resources. Suggestions to improve quality of teaching of reading and further studies are made.     

Toward a general model of instructional communication

A general model of instructional communication is advanced and the linkages of four components in the model are tested. Data were drawn from college teachers and college students through a split‐class design which permitted collection of unique data from teachers and two sets of their students. Using by‐class analyses of data it was determined that teacher self‐reported temperament is significantly correlated with students' perceptions of the teachers' communication behaviors and the students' evaluation of teachers' source credibility and task attractiveness. All of these measures were also found to predict learning outcomes and teacher evaluations of another group of students. The results support the general model and provide a foundation for future research in instructional communication.     

Enhancing student access to science curricula through a reading intervention

The current study explored whether a reading intervention combining flexibly applied multisyllabic word‐decoding strategies with evidence‐based fluency strategies was effective in improving the science text reading skills of upper‐elementary struggling readers. Four students, three in fourth and one in fifth grade, participated in the study. A delayed multiple baseline design was utilized, with a staggered 3‐week baseline followed by 8 weeks of reading intervention. Three students demonstrated small to moderate gains in reading fluency on science instructional passages, but no generalized gains in reading fluency on standardized passages. All students demonstrated direct gains in multisyllabic word‐decoding accuracy on science instructional passages, but no generalized gains in decoding accuracy on standardized passages. Participating students rated the intervention favorably and perceived gains in their reading skills. These findings support the use of science curricular passages when implementing reading interventions to enhance students’ ability to access the curriculum.     

Embracing the Complexity of Instruction: A Study of the Effects of Teachers' Instruction on Students' Reading Comprehension

The purpose of this study was to determine whether third-grade teachers' instructional actions during reading comprehension lessons contributed to their students' reading comprehension achievement. Our framework focused on teachers' emphasis on three dimensions of instruction (pedagogical structure, teacher-directed instruction, and support for student learning), as observed in comprehension lessons across a year. Third-grade teachers' instruction was analyzed first by measuring their latent propensity to engage in instructional actions in the three dimensions and then by using these latent variables in a multilevel model to examine their students' gains in reading comprehension. Results provided support for the theoretical dimensions, taking into account contextual variables including lesson, student, and teacher characteristics; teachers' engagement in teacher-directed instruction and their support for student learning significantly contributed to their students' reading comprehension. Results suggest that analysis of teachers' instructional actions within and across lessons is a promising approach for the study of effective reading instruction.     

Relationship of Online Teacher Professional Development to Seventh-Grade Teachers' and Students' Knowledge and Practices in English Language Arts

Online professional development (OPD) has potential to improve teacher quality by improving teachers' knowledge and instructional practices. These changes, in turn, have potential to improve student achievement. Unfortunately, there is a dearth of scientific research on the effects of OPD on teachers and, more importantly, on students. This article presents the results of a randomized controlled trial exploring the effects of a series of three learning-community model OPD workshops on teachers' content knowledge, teachers' instructional practices, students' content knowledge, and students' practices in the context of seventh-grade English language arts. There were statistically significant effects on teachers' vocabulary and overall English language arts content knowledge and on vocabulary and writing practices. The effect sizes of these changes ranged from small to medium. There were statistically significant effects on students' reading comprehension practices.     

The Relationship of Teachers' Instructional Decisions and Beliefs about Alternate Assessments to Student Achievement

In the decade since alternate assessments based on alternate achievement standards (AA-AAS) were first required, special education teachers have had to develop content knowledge and pedagogical skills to teach academics to students with significant cognitive disabilities. The purpose of this study is to examine the impact that teachers' instructional decisions and beliefs about AA-AAS have on the academic achievement of students with significant cognitive disabilities. A survey of nearly 400 teachers across three states provided evidence of teachers' beliefs, as well as their instructional planning processes for students. Students' learning characteristics were also identified. Survey responses were analyzed in conjunction with students' 2010–2011 scores on their statewide AA-AAS. The results indicated that students with presymbolic communication systems had a lower likelihood of achieving proficiency in English language arts/reading and math than students with abstract symbolic communication. Students with concrete symbolic communication systems had lesser odds of proficiency in ELA/reading but not in math. Teachers who agreed that AA-AAS had an impact on instruction tended to have students with higher achievement proficiency. However, more negative views of the validity of AA-AAS scores were associated with greater odds of proficiency.     

Opening up the black box: Teacher competence,instructional quality,and students’ learning progress

Existing research indicates inconsistent or at best weak predictive effects of teacher knowledge on student achievement. Data from Germany were used to examine the relation between teachers' content and pedagogical content knowledge, their perception, interpretation, and decision-making skills, the instructional quality implemented in class, and students' learning progression in mathematics. Rather than direct effects of teacher knowledge on students, we hypothesized an effect chain with multiple mediation processes while controlling for school type and student background. Multi-level modeling with 3496 students from 154 classrooms revealed a mediating role of teachers' skills and their instructional quality for the relation between teacher knowledge and students' learning progress. Effect sizes were medium to strong, and the model explained a large amount of variance. No direct effects of teachers’ knowledge on student progress were found. We discuss our findings with respect to the teacher-competence-as-a-continuum model and with respect to future research.     

Reading and spelling gains following one year of Orton‐Gillingham intervention in Singaporean students with dyslexia

Despite the widespread use of Orton‐Gillingham (OG) based approaches to dyslexia remediation, empirical support documenting its effectiveness is lacking. Recently, Chia and Houghton demonstrated the effectiveness of the OG approach for remediation of dyslexia in Singapore. As a conceptual replication and extension of that research, we report results of 39 students with dyslexia aged between six and 14 years enrolled in an OG intervention programme over a period of one year in a single‐subject research (pre‐test/post‐test) design. Analyses of variance showed that students significantly improved in standardised tests of reading and spelling with moderate effect sizes (Cohen's d = 0.52–0.58). Additionally, an inverse relationship was found between students' ages when they began intervention and gains made during the intervention. Results thus indicate the effectiveness of an OG approach in remediating literacy difficulties in students with dyslexia and, taken together with previous studies, further suggest the importance of early identification and intervention.     

The effects of cooperative learning on enhancing Hong Kong fifth graders' achievement goals,autonomous motivation and reading proficiency

Research indicates that cooperative learning with teacher‐guided instruction is more effective in helping young children to learn than cooperative learning with minimal guidance. In the present study, two different cooperative learning activities (jigsaw and drama) and a control condition (a traditional teacher‐led approach) were compared. The participants were 279 Grade 5 Hong Kong students located in nine classrooms. The two experimental conditions emphasised teachers' cognitive support in helping students to understand a text through both teacher‐led and student‐led activities. Post‐test data included a reading comprehension test and three questionnaires that investigated students' goal orientations, initial level of relative autonomy and perceptions of instructional practices. Findings indicate that students in the jigsaw group outperformed students in both the drama group and the control group in the reading comprehension test. The implications of conducting cooperative learning activities with well‐planned teacher guidance are discussed.     

The importance of teachers' emotions and instructional behavior for their students' emotions – An experience sampling analysis

The present study focuses on the relationship between teachers' emotions, their instructional behavior, and students' emotions in class. 149 students (55% female, M age = 15.63 years) rated their teachers' emotions (joy, anger, anxiety) and instructional behavior, as well as their own emotions in an experience-sampling study across an average of 15 lessons in four different subject domains. Intraindividual, multilevel regression analyses revealed that perceived teachers' emotions and instructional behavior significantly predicted students' emotions. Results suggest that teachers' emotions are as important for students' emotions as teachers' instructional behavior. Theoretical implications for crossover theory and practical recommendations for teachers are discussed.     

Promoting classroom adjustment and social skills for students with mental retardation within an experimental and control group design

Abstract

This study aimed to evaluate the impact of the “I Found a Solution” computer‐assisted social skills intervention program on students with mild mental retardation. Teachers randomly divided students from two Israeli special schools (58 males, 29 females; aged 10.6 to 17.11 years) into an experimental and a control group. Teachers trained the experimental group using a social skills package including computerized social conflict scenarios and adventure games, group discussions, and homework tasks. Within the same time frame, teachers trained the control group to use various academic computer software programs. Following the intervention process, the research team collected information on students' self‐reported social skills and teachers' ratings of the students' adaptive classroom behavior. The experimental group performed significantly better than controls on posttest criteria. Teachers rated trained students as demonstrating better task orientation and less aggression and behavior difficulties. On subjective self‐reports, trained students reported more cooperation and assertion than the control group, but self‐control and empathy measures did not differentiate between groups. The use of a naturalistic mini‐environment for experimenting and rehearsing effective social strategies appears promising but requires further exploration.     

Examining the Relations Between Spatial Skills,Spatial Anxiety,and K‐12 Teacher Practice

Spatial skills are important for student success in STEM disciplines at the K‐12 educational level. Teachers' spatial skills and feelings about completing spatial tasks influence their teaching as well as their students' spatial learning. However, the relation between teachers' spatial skills and their spatial anxiety is not well understood. Here we investigated if teachers' spatial skills influence two kinds of small‐scale spatial anxiety: (a) anxiety for tasks involving visual imagery and (b) anxiety for tasks involving mental manipulations. In addition, we investigated if teachers' spatial skills in conjunction with their small‐scale spatial anxiety influence the integration of spatial practices, such as gestures and diagrams, into their teaching. Eighty‐two K‐12 teachers completed two subscales of small‐scale spatial anxiety, a measure of spatial skills, and a teaching activities questionnaire. Results indicate that teachers' spatial skills are negatively associated with their spatial anxiety for mental manipulation tasks, and positively associated with their use of spatial practices. These findings highlight the need to account for teachers' spatial skills when considering how to improve students' spatial learning.     

Identifying determinants of teachers' judgment (in)accuracy regarding students' school-related motivations using a Bayesian cross-classified multi-level model

Teachers differ considerably in their judgment accuracy of motivational student characteristics. Thus far, only few investigations have focused on explaining these differences. In this study, we investigated to what extent groups of characteristics (i.e., student, information, teacher, and class characteristics) derived from the Realistic Accuracy Model (Funder, 1995) are relevant for explaining differences in teachers' judgment accuracy regarding students' school-related self-concept and autonomous motivation. Data from 1239 students and 341 teachers were analyzed using a Bayesian cross-classified multi-level modeling approach. Our analyses showed that variance in teacher judgments is largely due to variation at the level of judgments and less due to variation in the slope (i.e., the accuracy of teacher judgments). Teachers' judgment accuracy varied to a comparable degree across teachers and classes. Significant determinants for these differences were teachers' subject and students' grade point average.