Policy issues: Student Performance and Congressional Support

In the first issue of this journal, I wrote about policy issues with which all stakeholders associated with at-risk children and youth should be involved (Carroll, 1996). Continuing in the policy arena, I now speak to student results. The Title I program serves more than 5 million children with a $7 billion appropriation, and school districts need only report to the state the achievement of Title I participants who are tested as part of the annual state assessment program at three grade groupings--Grades 3 to 5, 6 to 8, and 10 to 12. Districts and states are no longer required to conduct pretest and posttest assessments that show the normal curve equivalent growth of children. Instead, adequate yearly progress toward meeting the states' definitions of advanced, proficient, and partially proficient student performance measures is the new yardstick of accountability and program success. These definitions apply no later than the year 2000-2001, when the states must have their student assessments aligned with their content and student performance standards. Even though the new Title I regulations ease up on frequency and coverage of assessment, Title I schools and programs should not. Schools must assess the performance of all their students and show results if we are to garner continued financial and program support from members of Congress and out constituencies at the state and local levels.     

The Student Risk Screening Scale: Exploring Dimensionality and Differential Item Functioning

This research attempted to discover the degree to which performance on assessments that have no direct consequences for individual students can be attributed to students' motivational states in addition to their underlying achievement in the domains measured. If such findings were reliable, it would allow us to correct underestimates of domain performance due to lack of motivation. Such corrections would be extremely interesting in explaining differential subgroup performance in state or national assessments that have institutional but not individual consequences. Recent information on international assessments (e.g., the Third International Mathematics and Science Study [TIMSS]) indicates that 12th-grade students in the United States are doing poorly on such assessments compared with their peers in other countries. These poor results are usually attributed to cognitive factors such as students' opportunities to learn, etc. However, a partial explanation may be motivational. Because the low-stakes tests were administered in these 12th-graders' final year in high school, this timing may have negatively affected motivation, and thus performance. Using TIMSS released math items, we provided $10.00 per item correct so as to increase a motivational effect and thus increase performance. However, the monetary incentive was not effective in improving performance.     

The Role of Student Growth Percentiles in Monitoring Learning and Predicting Learning Outcomes

Most formative assessments rely on the performance status of a student at a particular time point. However, such a method does not provide any information on the propensity of the student to achieve a predetermined target score or whether the student is performing as per the expectations from identical students with the same history of academic achievement. Although such a trend has been proposed via the use of the student growth percentiles (SGPs), most of the work associated with SGPs has focused mainly on teacher and school accountability. In this article, however, we illustrate examples in the use of SGPs and entry growth percentile scores to monitor students' learning progress. Monitoring students' learning trends, evaluated in the context of their SGPs and their propensity to achieve the target score, would likely yield meaningful differential information even for students with the same score at a particular point in time.     

Grade Placement of High-School Biology

Abstract

Using 35 elementary schools (3,350 fourth and sixth grade students), 10 secondary schools (3,613 eight and eleventh grade students), and 1,145 teachers, this study presents data summarizing the relationships between student' perceptions of "verified" principal competencies and selected school climate indices and outcome variables. The results indicated that there is a general tendency for positive teacher attitudes towards various dimensions of the school and working environment and higher student standardized achievement test performance to be associated with students' reports of a low frequency of interaction with die principal. A student "independence factor" was hypothesized to account for these results, with the implication being that principal/student interaction is minimized in schools where teacher and student attitudes are positive and student achievement is high. In addition, effective principal performance in dealing with student misbehavior was highly and positively associated with school average daily attendance at the secondary level. Supplementary analyses indicated that teacher and student attitudes "mediating" the school environment were relatively independent for both elementary and secondary samples. General support was found for higher correlations between student assessments of principal competencies and school environment measures than with student performance measures.     

Sample Exclusion and Student Attrition Effects in the Longitudinal Study of Middle School Mathematics Performance

Achievement data from a longitudinally matched student cohort from a large school district in the southwestern United States were analyzed to investigate sample exclusion and student attrition effects on estimates of student, school, and district mathematics performance. Use of 2- and 3-level longitudinal growth models to estimate the growth trajectories of middle school students revealed that mathematics performance differed across 2 sample conditions. Relative to the achievement outcomes associated with a sample that included all students from the longitudinal cohort, district and school achievement were generally higher and student group performance more similar in the smaller, more advantaged student sample used for district accountability reporting. Further investigation of the school performance estimates showed that cross-sample changes in student achievement outcomes were closely related to the proportion of students from special student populations who were excluded from the district accountability sample. The achievement differences and the differential patterns of association demonstrated in this study suggest that conclusions drawn about district and school performance and relationships between student characteristics and student achievement outcomes may depend to some degree on which students are included in an analytic sample. Investigators seeking to take advantage of longitudinal designs in school effectiveness research are cautioned to closely examine their data for nonrandom student attrition and document the impact of sample exclusion and student attrition effects in the research and accountability reports that are produced from longitudinal data sets.     

Multiple Sources of Evidence: An Analysis of Stakeholders' Perceptions of Various Indicators of Student Learning

This study compared different stakeholders' perceived validity of various indicators of student learning used to judge the quality of students' academic performance. Data were gathered from the questionnaire responses of 314 educators in three states that have implemented comprehensive state-wide assessment programs with high-stakes consequences both for educators and for students. MANOVA results showed that while educators generally hold similar perceptions, significant differences exist between school administrators and teachers. Administrators perceived the results from nationally normed standardized assessments, state assessments, and district assessments to be more valid indicators of student achievement than did teachers. In contrast, teachers granted more validity to classroom observations and homework completion and quality than did administrators. The implications of these differences for reform initiatives are discussed, particularly with regard to teachers' motivation to improve results.     

Effects of Student Displacement in Louisiana During the First Academic Year After the Hurricanes of 2005

The combined impact of Hurricanes Katrina and Rita in 2005 left the education system in the Gulf Coast region of the United States with tremendous challenges to rebuild infrastructure, reestablish services, and accommodate the movements of students that occurred as result of the storms. This article focuses on Louisiana's public school students, of whom 1/4 were displaced as a result of the storms. It explores the experiences of students and the effects of student movements on student achievement and on the state's public education system during the first academic year following the hurricanes. Results suggest that, overall, the education system handled the disruptions of the disaster relatively well. However, more detailed examinations show that some displaced students had problems such as nonenrollment or poor attendance, mental health or behavioral problems, and academic setbacks. Negative achievement effects, which were small overall, were most pronounced among students who remained displaced for the duration of the academic year, and appeared to be mitigated by students' tendency to enroll in schools with higher student performance than their original schools. Additional results show that the negative effects were associated with both the number of schools attended and the amount of school time lost. However, these analyses do not include the displaced students who did not reenroll in a Louisiana public school. These students constitute more than a 1/4 of displaced students and tended to be those who are most at risk for poor academic outcomes.     

Classroom Assessment Practices,Teacher Judgments,and Student Achievement in Mathematics: Evidence from the ECLS

In this study we use data from the Early Childhood Longitudinal Survey third- and fifth-grade samples to investigate teacher judgments of student achievement, the extent to which they offer a similar picture of student mathematics achievement compared to standardized test scores, and whether classroom assessment practices moderate the relationship between the two measures. Results indicate that teacher ratings correlate strongly with standardized test scores; however, this relationship varies considerably across teachers, and this variation is associated with certain classroom assessment practices. Furthermore, the evidence suggests that teachers evaluate student performance not in absolute terms but relative to other students in the school and that they may adjust their grading for some students, perhaps with basis on perceived differences in need and/or ability.     

Interpersonal Behavior,Laboratory Learning Environments,and Student Outcomes in Senior Biology Classes

The purpose of this study was to investigate associations between students' perceptions of their biology teachers' interpersonal behavior and their laboratory learning environments and their attitudinal, achievement, and performance outcomes. A sample of 489 students from 28 senior biology classes in eight schools in Tasmania, Australia completed the Questionnaire on Teacher Interaction (QTI) and the Science Laboratory Environment Inventory (SLEI). Responses to two attitude questionnaires, achievement on an external written examination, and performance in laboratory practical tests were used as student outcome measures. Statistical analyses supported the reliability and validity of the QTI and the SLEI when used with senior secondary biology students. We investigated associations between students' perceptions of teacher behavior and their laboratory learning environment with student outcomes, including the unique and common contributions of the QTI and SLEI to variance in student outcomes. Associations with students' perceptions of the learning environment were stronger for the attitudinal outcomes than for the cognitive or practical skills outcomes. Some commonality between the QTI and SLEI scales was found in their contributions to the variance in attitudinal outcomes, but not in their contributions to variance in cognitive and practical skill outcomes. © 1999 John Wiley & Sons, Inc. J Res Sci Teach 37: 26–43, 2000     

The Relationship Between Undergraduate Attendance and Performance Revisited: Alignment of Student and Instructor Goals

We revisit the relationship between attendance and performance in the undergraduate university setting and apply agency theory in the instructor–student context. Building on agency theory propositions in the educational setting advanced by Smith, Zsidisin, and Adams (2005) , we propose that the student and instructor must align goals to promote the student's achievement of performance learning outcomes, and attendance functions as a behavior‐based alignment mechanism to encourage the convergence of faculty and student interests. Further, we propose that attendance does not equally affect lower‐ and higher‐performing students and that absences are also negatively related to students' cumulative grade point average. We test these hypotheses with data from undergraduates enrolled in management courses at a state university in the southeast. Our results show that attendance is positively related to exam performance, there are more pronounced negative effects of an absence for lower‐performing students than for higher performers, and absences are negatively related to a student's cumulative grade point average. We discuss the implications of our findings for students, instructors, and universities as well as practice in teaching and learning.     

Focusing on the Whole Student: An Evaluation of Massachusetts's Wraparound Zone Initiative

We used a comparative interrupted time series design to examine the impact of the Massachusetts' Wraparound Zone Initiative on student achievement, attendance, retention, and suspension. This initiative was designed to create coordinated district systems that allow schools to proactively and systematically address students' nonacademic needs. The state education agency awarded grants to seven high-need districts, each of which used the funds to support a select group of schools in implementing strategies that focused on school climate and culture, family engagement, systems to identify and address student needs, and community partnerships. Results from this study showed that students in Wraparound Zone schools performed better on state academic assessments than students in comparison schools, when considering prior achievement trends. Effects were statistically significant in the 3rd year of implementation. The impact on academic achievement was greatest for third- and fourth-grade students, and was particularly strong for English learners. There was no statistically significant overall impact on attendance, retention, or suspension.     

Student application for special consideration for examination performance following a natural disaster

Universities have a long-established tradition of granting students special consideration when circumstances beyond their control negatively affect performance in assessments. Typically, such situations affect only one student (e.g. medical emergencies) but we consider the impact of a natural disaster that led to all students being eligible for special consideration on a single assessment. Students did not have to get applications for special consideration endorsed by a qualified professional but were able to rate their own level of impairment. Our findings indicated that students were impaired in their performance and accordingly application for special consideration was warranted. Those few students who did not submit an application were disadvantaged relative to their peers. There was little relationship between the students’ self-reported levels of impairment and their performance, but those who considered themselves seriously impaired were disproportionately unlikely to complete the assessment. Those with poorer grades leading into the final assessment were no more likely to request special consideration. Although our observations were with an unusual example, our overall findings support the need for a special consideration policy, and indicate that students can treat such a policy appropriately and not exploit the opportunity to obtain unmerited advantage.     

The Relationship of Teachers' Instructional Decisions and Beliefs about Alternate Assessments to Student Achievement

In the decade since alternate assessments based on alternate achievement standards (AA-AAS) were first required, special education teachers have had to develop content knowledge and pedagogical skills to teach academics to students with significant cognitive disabilities. The purpose of this study is to examine the impact that teachers' instructional decisions and beliefs about AA-AAS have on the academic achievement of students with significant cognitive disabilities. A survey of nearly 400 teachers across three states provided evidence of teachers' beliefs, as well as their instructional planning processes for students. Students' learning characteristics were also identified. Survey responses were analyzed in conjunction with students' 2010–2011 scores on their statewide AA-AAS. The results indicated that students with presymbolic communication systems had a lower likelihood of achieving proficiency in English language arts/reading and math than students with abstract symbolic communication. Students with concrete symbolic communication systems had lesser odds of proficiency in ELA/reading but not in math. Teachers who agreed that AA-AAS had an impact on instruction tended to have students with higher achievement proficiency. However, more negative views of the validity of AA-AAS scores were associated with greater odds of proficiency.     

Effect of Student Characteristics on Teachers' Predictions of Student Success

Abstract

To determine teachers' perceptions of the likelihood of students' academic and social success in general education settings, 384 teachers responded to 1 of 32 possible case studies describing a student, in which gender, reading achievement, social behavior, and attentiveness were manipulated experimentally. Results showed that, in some instances, teachers used irrelevant information in their predictions (e.g., social behavior influenced academic predictions, reading achievement influenced social predictions). Furthermore, teachers demonstrated gender bias in some instances and appeared to make negative predictions when an ambiguous label explained students' reading problems. The author discusses implications for the inclusion of students with special needs in general education classes.     

Patterns of Teacher‐Student Interaction in Inclusive Elementary Classrooms and Correlates with Student Self-Concept

The efficacy of placing students with special needs in inclusive classrooms may depend in part on how instructional factors contribute to student outcomes. Differences in frequencies and levels of cognitive engagement of interactions among nine teachers in inclusive elementary classrooms were related to three other variables: teachers' ratings on the Pathognomonic-Interventionist (PATH/INT) Scale, students' designation either as exceptional or at-risk (EX/AR) or as typically achieving (TA), and students' scores on the Piers Harris Children's Self-Concept Scale. Teachers with PATH beliefs, who attribute learning difficulties to permanent characteristics of the student that are beyond the teacher's mandate, interacted infrequently with their EX/AR students at low levels of cognitive engagement. Teachers with INT beliefs, who see themselves as responsible for the achievement of all their students irrespective of their disabilities, interacted with all students more frequently, and at higher levels of cognitive engagement. In contrast to the PATH teachers, their EX/AR students received more instructional interactions than their TA students. As expected, the EX/AR students in all classrooms had lower Piers Harris Self-Concept Total Scale scores than typically achieving students. However, both TA and EX/AR students had lower Self-Concept Total Scale scores in the classrooms of teachers with PATH beliefs, compared to students in the classrooms of INT teachers. The relationship is discussed between teachers' beliefs, their different patterns of instructional interactions with students with and without disabilities in inclusive classrooms, and the possible impact of instructional interventions on students' self-concept.     

Understanding When Teachers Disagree about Student Disability

The persistent racial and ethnic disparities in special education in the United States raise concerns about the potential misidentification of students. While previous studies have focused on how various student and school factors influence teacher decisions, there is less attention on when teachers disagree about student disability or special education. The current study uses national data from the Educational Longitudinal Study of 2002 to examine when teachers disagreed in their perception of student disability for more than 10,000 high school students. A unique feature of the data is each student is observed in two different subject classrooms, providing an opportunity to examine how differences in student achievement, behavior, and teacher characteristics influence when teachers disagreed in their judgment of student disability. The results indicate that teachers were more likely to disagree when student behavior varied across classrooms, while differences in student achievement and teacher characteristics were not related to disagreement. The study also found that teachers disagreed more for students who were Black, male, and from lower socioeconomic backgrounds. The main findings highlight how disability is often a context-dependent social construct and have implications for how students are identified for special education.     

Analysis of assessment practice and subsequent performance of third year level students in natural sciences

Abstract

Summative assessment qualifies the achievement of a student in a particular field of specialization at a given time. Questions should include a range of cognitive levels from Bloom's taxonomy and be consistent with the learning outcomes of the module in question. Furthermore, a holistic approach to assessment, such as the application of the principles of the Herrmann Whole Brain Model, needs to be used to accommodate learning style diversity. The purpose of this study was to analyse, assess and compare the summative assessment of two third year level modules in the Bachelor of Science degree programme, namely Biochemistry and Zoology as part of action research with a view to enhancing the professional development of the lecturers involved. The questions posed in summative assessments were classified in terms of Bloom's differentiation of cognitive levels and the four different learning styles determined by Herrmann. Spearman's non-parametric analysis indicated that no correlation existed in this study between cognitive level and student performance based on achievement. In addition, there was not much difference between the cognitive levels and student performance between the two disciplines. Although the students seemed to do better at application level questions, the authors need to reflect on whether the assessments were valid with respect to the learning outcomes, methods of facilitating learning, and the assessments based on cognitive levels and learning style preferences. We conclude that continuous action research must be taken to improve the formulation of learning outcomes and students' achievement of these outcomes and quality of student learning – the main aim being the successful completion of the modules.     

Should teachers be accurate or (overly) positive? A competitive test of teacher judgment effects on students’ reading progress

Previous findings on effects of teachers' judgments on student learning have been contradictory leading to the question of what kinds of judgments are most beneficial: accurate or (overly) positive ones? In this study, we provide the first competitive test of prominent but contradictory hypotheses regarding the consequences of teachers' judgments in the context of reading proficiency using reading fluency and reading comprehension performance judgments from 145 teachers and measures of real performance and learning progress across eight points of measurement from 2880 students. Response Surface Analyses combined with an information-theoretic approach for model comparison revealed no evidence of positive effects of judgment accuracy or overestimation of student performance by teachers. Instead, progress in reading fluency and reading comprehension was best predicted by students' prior achievement. For reading comprehension, the positivity of teachers' judgments was additionally beneficial: The higher a teacher judged a student's performance, the more the student learned.     

Student Engagement and Student Learning: Testing the Linkages*

This study examines (1) the extent to which student engagement is associated with experimental and traditional measures of academic performance, (2) whether the relationships between engagement and academic performance are conditional, and (3) whether institutions differ in terms of their ability to convert student engagement into academic performance. The sample consisted of 1058 students at 14 four-year colleges and universities that completed several instruments during 2002. Many measures of student engagement were linked positively with such desirable learning outcomes as critical thinking and grades, although most of the relationships were weak in strength. The results suggest that the lowest-ability students benefit more from engagement than classmates, first-year students and seniors convert different forms of engagement into academic achievement, and certain institutions more effectively convert student engagement into higher performance on critical thinking tests. *A version of this paper was presented at the annual meeting of the American Educational Research Association, San Diego, April 2004.     

Teachers' explicit expectations and implicit prejudiced attitudes to educational achievement: Relations with student achievement and the ethnic achievement gap

In this paper, we consider whether teachers' explicit and implicit prejudiced attitudes underlie the ethnic achievement gap. To date, most research on teacher expectation effects has relied on explicit expectation measures that are prone to social desirability biases. In contrast, we examine the effects of teachers' (a) explicit ethnicity-based expectations for academic achievement and (b) implicit prejudiced attitudes about academic achievement on students' actual academic success over time. A total of 38 teachers completed both a traditional teacher expectation measure and a modified Implicit Association Task designed to assess ethnic stereotypes associated with academic achievement and failure. A multi-level analytic framework showed that students in classrooms of teachers with high expectations performed better in reading at the end of the year and that these effects were found across all ethnic groups. In contrast, whereas students' mathematics achievement scores were largely unrelated to teachers' explicit expectations, teachers' implicit prejudiced attitudes predicted student performance. Specifically, students benefited most academically when their teachers' implicit biases favored the ethnic group to which the student belonged. Findings are discussed in relation to differences in the salience of teachers' expectations and implicit prejudiced attitude in the classroom, and the ethnic achievement gap.