Alternative Assessment in a HighmStakes Environment

How generalizable are the results of alternative assessments? How much psychometric evidence d o we have about these approaches? What effects do alternative assessments have on subgroup differences in performance? What are the probable consequences of using alternative assessments?     

Effects of Introducing Classroom Peflormance Assessments on Student Learning

How can performance assessments be used as part of regular instruction? Will this raise student performance on external achievement measures? What aspects of examinee performance improve on the assessment exercises?     

Investigating the Cognitive complerrity of Science Assessments

How can knowledge of the development of expertise in a subject area inform the development of assessments in that area? How can one examine the relationship between what is anticipated from a performance assessment and what is actually seen?     

Commentary on Values and Standards in Performance Assessment

Is construct validity relevant to performance assessment? Can these assessments allow meaningful comparisons? How can we minimize validity-reducing errors?     

Generalizability of Performance Assessments

How can the contributions of raters and tasks to error variance be estimated? Which source of error variance is usually greater? Are interrater coefficients adequate estimates of reliability? What other facets contribute to unreliability in performance assessments?     

Validating Measures of Performance

How can the validity of performance assessments be established? What type of logical argumentation is necessary to generalize from a particular assessment to the ability or abilities of interest?     

Setting Standards for Complex Performances: An Iterative, Judgmental Policy capturing Strategy

What is judgmental policy capturing? How can it be applied to standard setting with performance assessments? Do judges value some exercises more in setting standards?     

Performance-Based Assessment: Implications of Task Specificity

How generalizable are the scores from performance-based assessments? How can we best increase the gener-alizability of results? What level of generalizability is needed for different testing purposes?     

Portfolios, Accountability, and an Interpretive Approach to Validity

How can the results of classroom-based portfolio assessment be communicated outside the classroom? How might a portfolio-based assessment system be designed and implemented? How can we evaluate the merits of portfolio-based assessments?     

Is the Curriculum a Reasonable Basis for Assessment Reform?

How should curriculum-assessment linkages be forged? Do local districts have sufficient resources and expertise to sustain these ongoing assessments? Can assessments developed at state level accurately reflect local curricula? What are the essential characteristics of a curriculum-based assessment system?     

Contextualizing the Interpretation of Reliability Data

What is the relationship between the context and substantive meaning of a measure and the assessment of the reliability of the measure? Does the interpretation of a reliability coefficient depend on the nature of the measure? How does this approach to reliability relate to alternative assessments?     

Evaluating the quality of peer and self evaluations as measures of student contributions to group projects

ABSTRACT

Educators in higher education commonly use peer and self evaluations to help assess student performance on group projects. Although these evaluations provide multiple benefits, many educators are wary of using them due to concerns about their quality. This study addresses three questions debated in the literature regarding the quality of these assessments. How much do students differentiate among peer contributions through their ratings? How reliable are peer ratings? How much agreement exists between peer and self ratings? Although these questions have been addressed to varying degrees in past work, their answers have been far from settled. While many studies focus on just one of the questions, this study’s data make it possible to address all three questions for the same group of students as well as examine each question by student performance level. The evaluations assessed in this study were completed by a large number of students under conditions associated with obtaining more valid and reliable ratings. Overall, the results provide support for using peer and self evaluations to help assess student contributions to group projects. Peer ratings were largely reliable as group members generally agreed on the scores given to their peers. In addition, most students differentiated among group member contributions through their ratings. Students also tended to rate themselves higher than their peers rated them. This study has implications for how peer and self evaluations can be most effectively used by educators to measure student performance in group work.     

Determining the Validity of Performance-Based Assessment

What criteria should be applied to the evaluation of performance measures? How consistent are the results from performance measures and norm-referenced achievement tests? How can we ensure fairness and credibility in performance measurement?     

How to Prepare Students for PerEorrnance Assessments

Now that the use of performance assessments for certification, graduation, and classroom use is burgeoning, what are the responsibilities of educators for preparing students for such assessments? Do the same guidelines apply for both multiple-choice tests and performance assessments?     

Cognitive-Style Differences in Testing Situations

Why will multiple-choice tests continue to serve the needs of education, business, and government? How d o differences in cognitive style affect performance on multiple-choice tests of different types? How does clarity and care in writing multiple-choice items affect performance?     

Learning science content through socio-scientific issues-based instruction: a multi-level assessment study

Science educators have presented numerous conceptual and theoretical arguments in favor of teaching science through the exploration of socio-scientific issues (SSI). However, the empirical knowledge base regarding the extent to which SSI-based instruction supports student learning of science content is limited both in terms of the number of studies that have been conducted in this area and the quality of research. This research sought to answer two questions: (1) To what extent does SSI-based instruction support student learning of science content? and (2) How do assessments at variable distances from the curriculum reveal patterns of learning associated with SSI-based instruction? Sixty-nine secondary students taught by three teachers participated in the study. Three teachers implemented an SSI intervention focused on the use of biotechnology for identifying and treating sexually transmitted diseases. We found that students demonstrated statistically and practically significant gains in content knowledge as measured by both proximal and distal assessments. These findings support the claim that SSI-based teaching can foster content learning and improved performance on high-stakes tests.     

Schooling and the Norming of Intelligence Test Scores

How does schooling affect the development of intelligence in children? How should the amount of schooling be considered when developing norms for turning intelligence test performance into IQ scores?     

Educational and Employment Testing: Changing Concepts in Measurement and Policy

How will the expansion of the concept of construct validity affect validation practice in employment testing? How does the need for consequential validity differ in educational and employment testing? How do the research bases differ for performance assessment in these settings? Are there parallel trends in policies for test use in education and industry?     

When We Inquire Into Our Own Practice: An Early Childhood Teacher Research Group

Every day, early childhood teachers confront issues, problems, and concerns in their classrooms. Sometimes they do nothing. Sometimes they use trial and error. Sometimes they go to a workshop or read an article. We have found a way to intentionally and systematically research and answer our own questions and to enrich our own professional development through teacher research. Teacher research provides early childhood educators with a strategy that supports reflective thinking and practice. This article describes the experience of a small group of early childhood teacher researchers. How is it different from other required assessments? How is it different from other professional development?     

Have Changes in the SAT Affected Women's Mathematics performance?

How has the SAT changed in content and format? Why was it changed? How have changes to the mathematics test affected men's and women's performance?