Development of Performance Assessments in Science: Conceptual, Practical, and Logistical Issues

What are practical and logistical constraints in developing science performance assessments (SPAs)? What are key components in a framework for conceptualizing the process? What are the major steps in SPA development?     

Language Variation and Score Variation in the Testing of English Language Learners,Native Spanish Speakers

We investigated language variation and score variation in the testing of English language learners, native Spanish speakers. We gave students the same set of National Assessment of Educational Progress mathematics items in both their first language and their second language. We examined the amount of score variation due to the main and interaction effect of student (s), item (i), rater (r), and language (l). We observed a considerable score variation due to the interaction, s × i × l, which indicates that each item has a unique set of linguistic challenges in each language and each student has a unique set of strengths and weaknesses in each language. Parallel results were obtained with samples of students tested across dialects of their native language.     

Probabilistic Approaches to Examining Linguistic Features of Test Items and Their Effect on the Performance of English Language Learners

This article addresses validity and fairness in the testing of English language learners (ELLs)—students in the United States who are developing English as a second language. It discusses limitations of current approaches to examining the linguistic features of items and their effect on the performance of ELL students. The article submits that these limitations stem from the fact that current ELL testing practices are not effective in addressing three basic notions on the nature of language and the linguistic features of test items: (a) language is a probabilistic phenomenon, (b) the linguistic features of test items are multidimensional and interconnected, and (c) each test item has a unique set of linguistic affordances and constraints. Along with the limitations of current testing practices, for each notion, the article discusses evidence of the effectiveness of several probabilistic approaches to examining the linguistic features of test items in ELL testing.     

Generalizability theory and the fair and valid assessment of linguistic minorities

We discuss generalizability (G) theory and the fair and valid assessment of linguistic minorities, especially emergent bilinguals. G theory allows examination of the relationship between score variation and language variation (e.g., variation of proficiency across languages, language modes, and social contexts). Studies examining score variation across items administered in emergent bilinguals' first and second languages show that the interaction of student and the facets (sources of measurement error) item and language is an important source of score variation. Each item poses a unique set of linguistic challenges in each language, and each emergent bilingual individual has a unique set of strengths and weaknesses in each language. Based on these findings, G theory can inform the process of test construction in large-scale testing programmes and the development of testing models that ensure more valid and fair interpretations of test scores for linguistic minorities.