Evaluating the Instructional Sensitivity of Four States' Student Achievement Tests

As state tests of student achievement are used for an increasingly wide array of high- and low-stakes purposes, evaluating their instructional sensitivity is essential. This article uses data from the Bill and Melinda Gates Foundation's Measures of Effective Project to examine the instructional sensitivity of 4 states' mathematics and English language arts assessments to 5 measures of pedagogical quality. The results indicate modest overall levels of sensitivity, with some state assessments insensitive to most or all measures of pedagogical quality. Furthermore, there are sometimes substantial differences across states in the sensitivity of their assessments. The article calls for more research on what makes state tests instructionally sensitive and cautions against the use of insensitive assessments for consequential decisions.     

Classroom instructional quality,exposure to mathematics instruction and mathematics achievement in fifth grade

This study examined the quality of teacher–child interactions and exposure to mathematics instruction as predictors of 5th grade student’s mathematics achievement. The sample was a subset of the children involved in the NICHD–SECC longitudinal study (N = 657). Results indicate that, even after controlling for student demographic characteristics, more exposure to mathematics instruction was related to increased fifth grade mathematics achievement for both calculations and applied problems assessments, but there was no main effect for improved instructional quality. Findings also indicate that, in classrooms where lower instructional quality was observed, greater exposure to mathematics instruction predicted improved mathematics achievement. Findings are discussed in terms of differing aspects of mathematics instruction and the possible compensatory role of exposure to instruction in classrooms of lower quality.     

Teacher effects on student achievement in first grade: which aspects matter most?

The present study investigates the effects of teachers (background qualifications, attitudes and beliefs, and instructional practices) on student achievement in mathematics, reading, and spelling in 1st grade. Its theoretical framework and methodology are based on recent work by Palardy and Rumberger (2008). Data from the SiBO Project, a longitudinal study in Flemish primary education, were analyzed for the present study using 2-level regression analysis. The results showed that teacher background had the largest effect on mathematics achievement, whereas instructional practices had the largest effects on both reading and spelling achievement. Moreover, the size of teacher effects depended upon the specific learning domain: Larger teacher effects were found for mathematics than for spelling and reading. Overall, the results suggest that teachers had a modest to strong effect on student achievement in first grade.     

The impact of alternative authentic assessment outcomes in Palestinian fourth grade math classrooms

Focusing on evaluating students’ performance in basic math classrooms, the researchers in this study examined the impact of alternative assessments on learning outcomes in fourth grade Palestinian classrooms. Representing a large sector of education in the Palestinian territories, fourth grade students were randomly selected to participate in the study in which they participated in various math instructional and assessment activities throughout multiple lessons and cycles. Alternative assessment approaches were used including student self- assessment, peer assessment, and teacher assessment in terms of three main achievement levels to measure the extent to which students learned the math concepts by recall and remembrance, ability to apply, and making inferences. Mixed Design ANOVA measures were used to analyze and interpret the results which showed significant trends and correlations across the four cycles of math instruction and indicated that alternative authentic assessment methods have had a positive impact on students’ learning and application of math. Implications for integrating authentic assessment measures as well as peer and self- assessments were drawn in light of the promising outcomes of augmenting student motivation and developing critical life-long skills.     

Project Proficiency: Assessing the Independent Effects of High School Reform in an Urban District

ABSTRACT.

The authors sought to determine if an urban school district's effort to fundamentally change teaching, assessment, and intervention practices increased student achievement and decreased achievement variation among classrooms in 11 high schools. They examined Grade 11 mathematics achievement data from the 2009–2010 and 2010–2011 Kentucky Core Content Test. The sample consisted of 2,451 students in 11 high schools that implemented an initiative called Project Proficiency. The study used hierarchical linear modeling to determine if changes in instructional practices yielded academic gains, while controlling for individual and school socioeconomic status (SES) and prior student achievement. The analysis revealed that Project Proficiency had a significant impact on state mathematics achievement and reduced classroom variation by 55% in the final model, as opposed to 16% in the control group. Results also indicated that a school's SES had a greater influence on student achievement than an individual student's SES.     

Mathematics achievement,language and cognitive development: Classroom practices in Papua New Guinea

A case study was conducted of mathematics instruction at five sites in Papua New Guinea. Trial instructional materials were developed, teachers received inservice training and implementation was carefully monitored for a six-week period. English and mathematics achievement instruments and eight measures of cognitive development were administered to a sample of 201 students in grades two, four and six. Mathematics and language achievement varied significantly among the five sites. Students at all grade levels exhibited poor application of problem solving skills while showing computation, measurement and mathematical language abilities superior to results previously reported. English reading and measures of conservation and classification competence were highly correlated with mathematics achievement. Correlations between memory measures and mathematics achievement decreased from grade two to grade six while correlations between measures of language and cognitive development and mathematics achievement tended to increase. The results indicated that the non-consumable student textbooks ameliorated to some extent the problems associated with non-native language instruction for the students involved in the study. It was concluded that locally developed textbooks with an appropriate language load would enhance mathematics learning, particularly in remote-rural areas where current teaching conditions and lack of materials interact with poor English skills to create an imbalance in achievement.     

Parental Involvement,Instructional Expenditures,Family Socioeconomic Attributes,and Student Achievement

Abstract

The influence of parental involvement, socioeconomic status of parents, and instructional supplies expenditures on mathematics achievement scores of Grade 4 students in a low-income county in North Carolina were examined. An educational production function framework was used to analyze the influence of educational resources on mathematics achievement scores. Pearson product-moment correlation and ordinary least squares regression were used to determine the overall strength of each relation and the variables with the greatest impact on mathematics achievement. Results indicated that instructional supplies expenditures per pupil and parental volunteer hours were not statistically significant in explaining mathematics test scores. Furthermore, results showed that the percentage of students in free/reduced-price lunch programs was related negatively to students' academic performance in mathematics. This finding supports the notion that economic circumstances are correlated with academic achievement.     

Multifaceted Assessment of Inquiry-Based Science Learning

To improve student science achievement in the United States we need inquiry-based instruction that promotes coherent understanding and assessments that are aligned with the instruction. Instead, current textbooks often offer fragmented ideas and most assessments only tap recall of details. In this study we implemented 10 inquiry-based science units that promote knowledge integration and developed assessments that measure student knowledge integration abilities. To measure student learning outcomes, we designed a science assessment consisting of both proximal items that are related to the units and distal items that are published from standardized tests (e.g., Trends in International Mathematics and Science Study). We compared the psychometric properties and instructional sensitivity of the proximal and distal items. To unveil the context of learning, we examined how student, class, and teacher characteristics affect student inquiry science learning. Several teacher-level characteristics including professional development showed a positive impact on science performance.     

Principals’ Instructional Leadership Within a Teacher Performance Appraisal System: Enhancing Students’ Academic Success

The purpose of this study was to identify principals’ instructional leadership actions within a comprehensive teacher evaluation system in successful schools rated as recognized or exemplary by the accountability measures in place. The study followed a multiple case study approach. Participants included six school administrators within the same school district. From these three were principals and three assistant principals, representing all three schools levels—elementary school, middle school, and high school. The main data sources consisted of interviews, observations and journaling. Data were analyzed inductively to discern emerging themes. Findings of the study revealed that principals’ instructional leadership actions at all three levels included setting clear expectations, monitoring instruction through walk-through observations, and providing professional development opportunities according teachers’ needs. Furthermore, the cross site analysis revealed that principals tend to rely on instructional leadership actions that are pertinent to each grade level’s needs. Findings also suggest that principals, indeed, apply the teacher performance appraisal system as a basis to enhance instruction and improve student achievement.     

Effects of a Learning Information System on Mathematics Achievement and Classroom Structure

Abstract

The authors examined the effects of implementing an instructional system that automates application of evidence-based components of effective instruction on student mathematics achievement and on classroom behaviors known to be related to overall student achievement outcomes. A treatment group of 157 4th- and 5th-grade students used the intervention in conjunction with the Everyday Math curriculum. Student performance was compared with a within-school control group of 61 4th- und 5th-grade students, as well as all 4th- and 5th-grade students in the district (N = 6,385). The students in the control groups received only the Everyday Math curriculum. Results indicate that the implementation of the instructional management system as an enhancement to Everyday Math resulted in an increase in the amount of time spent on classroom activities that researchers have identified as contributing to positive academic outcomes. Furthermore, students who used the Accelerated Math program demonstrated greater mathematics achievement gains than did the control groups.     

Instructional Tasks and the Development of Student Capacity to Think and Reason: An Analysis of the Relationship between Teaching and Learning in a Reform Mathematics Project

Abstract

In the present study the relationship between teaching and learning was examined using a conceptual framework that links dimensions of instructional tasks with gains in student learning outcomes. The greatest student gains on a performance assessment consisting of tasks that require high levels of mathematical thinking and reasoning were related to the use of instructional tasks that engaged students in the “doing of mathematics” or the use of procedures with connections to meaning. In addition, student performance gains were greater for those sites whose tasks were both set up and implemented to encourage the use of multiple solution strategies, multiple representations, and explanations. Whereas, student performance gains were relatively small for those sites whose tasks tended to be both set up and implemented in a procedural manner and that required a single solution strategy, single representations, and little or no mathematical communication.     

The Sensitivity of Value-Added Teacher Effect Estimates to Different Mathematics Achievement Measures

Using longitudinal data from a cohort of middle school students from a large school district, we estimate separate "value-added" teacher effects for two subscales of a mathematics assessment under a variety of statistical models varying in form and degree of control for student background characteristics. We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers' performance based on value-added models can be sensitive to the ways in which student achievement is measured.     

Supporting mathematics vocabulary instruction through mathematics curricula

Abstract

This study examines four of the most commonly-used core mathematics curricula in the USA for evidence of support for research-based instructional strategies for mathematics vocabulary in first and second grade. Content analyses of the teachers’ editions of two units for each grade level were analyzed per curriculum (n?=?16). Statistically significant differences among curricula were found for number of target words (range 6–51 per unit), level of difficulty of terms (basic to technical), and number of support strategies per word. Multiple means of representation varied in terms of verbal and non-verbal strategies for target terms. These differences indicate children are experiencing substantially different mathematics vocabulary learning opportunities, which may impact later mathematics achievement. Implications for practice, curriculum development, and future research are addressed.     

Introduction to the Special Issue: Effective Practices for Children and Youth with Autism Spectrum Disorders

This article focuses on the development and use of alternative reading measures that are performance based (authentic and direct), technically adequate, and help teachers make instructional decisions. Although performance assessments are promulgated as better than many traditional, published measures of achievement, few studies on technical adequacy have been conducted. We present 2 studies that focus on concurrent criterion validity and instructional decision making; the studies are framed as examples of reading measurement within the debate on whole language programs. The first study focuses on the relation of the performance measure of oral reading with several other reading measures; the second study focuses on teacher decision making, using quantitative and qualitative outcomes reflecting individual student change in performance (in reading fluency and prosody) over time. In this article, both studies are described briefly, and the emphasis of discussion is placed on appropriately evaluating programs so that the measures match the interventions (are systemically valid) and support their effec- tiveness for use with individual students.     

Assessment of mathematics and reading performance: An examination of the correspondence between direct assessment of student performance and teacher report

Teacher judgments have been identified as a primary source of information regarding student academic achievement. Research examining the accuracy of teachers' judgments in assessing students' academic abilities has shown relatively high accuracy. However, previous studies have relied primarily on norm‐referenced measures to obtain estimates of students' achievement in reading and mathematics. Recent developments in the assessment of students' academic skills, such as Curriculum‐Based Measurement (CBM), provide a direct estimate of students' skill levels in basic areas such as reading and mathematics. The purpose of the present study was to examine the extent to which teachers' perceptions of students' reading and mathematics skills corresponded to direct estimates of students' reading and mathematics skills. Two second‐grade teachers estimated the reading and mathematics skills of 33 second‐grade students. Results of this study indicated that teachers were not accurate in assessing their students' mathematics functioning. Teachers were more accurate in assessing the occurrence of Mastery mathematics levels in basic addition, but were very inaccurate in assessing the occurrence of Mastery, Instructional, or Frustrational mathematics levels in all other skills assessed. In reading, teachers' judgment accuracy varied as a function of grade‐level material and instructional level. Specifically, teachers experienced considerable difficulty accurately identifying students who were reading at a Mastery level in grade‐level or above‐grade‐level material. © 2006 Wiley Periodicals, Inc. Psychol Schs 43: 247–265, 2006.     

Grade Placement of High-School Biology

Abstract

Using 35 elementary schools (3,350 fourth and sixth grade students), 10 secondary schools (3,613 eight and eleventh grade students), and 1,145 teachers, this study presents data summarizing the relationships between student' perceptions of "verified" principal competencies and selected school climate indices and outcome variables. The results indicated that there is a general tendency for positive teacher attitudes towards various dimensions of the school and working environment and higher student standardized achievement test performance to be associated with students' reports of a low frequency of interaction with die principal. A student "independence factor" was hypothesized to account for these results, with the implication being that principal/student interaction is minimized in schools where teacher and student attitudes are positive and student achievement is high. In addition, effective principal performance in dealing with student misbehavior was highly and positively associated with school average daily attendance at the secondary level. Supplementary analyses indicated that teacher and student attitudes "mediating" the school environment were relatively independent for both elementary and secondary samples. General support was found for higher correlations between student assessments of principal competencies and school environment measures than with student performance measures.     

Monitoring Student Achievement for Accountability: The Demonstration of a Model

Abstract

This study demonstrated a procedural model that can be applied by any school to assess, guide, and account for the progress of its students as well as to analyze its own effectiveness. The model uses equivalent achievement tests to monitor student achievement in subject areas at grade levels, between grade levels, and across subgroups of students. Multiple regression analyses of test scores between grades identify factors associated with achievement Using sixth and eighth grade Comprehensive Tests of Basic Skills scores in a matched longitudinal sample of 208 students, the study found small differences in average achievement between boys and girls. Differences between corresponding sixth and eighth grade test means were higher in mathematics than in language. From the sixth grade to the eighth, there was a widening gap in average achievement between high and low I.Q. groups. In multiple regressions of eighth grade test scores on sixth grade measures, I.Q., study skills, and reading were prevalent in the regression equations, but clusters of measures associated with achievement differed between high and low’ LQ. groups. The results of the study have implications for developing and evaluating the achievement of students with varying mental abilities.