On the cultural validity of science assessments

We propose the concept of cultural validity as a form of test validity in science assessment. The conceptual relevance of cultural validity is supported by evidence that culture and society shape an individual's mind and thinking. To attain cultural validity, the process of assessment development must consider how the sociocultural context in which students live influences the ways in which they make sense of science items and the ways in which they solve them. These sociocultural influences include the values, beliefs, experiences, communication patterns, teaching and learning styles, and epistemologies inherent in the students' cultural backgrounds, as well as the socioeconomic conditions prevailing in their cultural groups. We contend that current approaches to handling student diversity in assessment (e.g., adapting or translating tests, providing assessment accommodations, estimating test cultural bias) are limited and lack sociocultural perspective. We find that attaining cultural validity may conflict with current basic principles and assumptions in testing, such as item independence and standardization. We discuss the ways in which adopting cultural validity as a criterion for test validity makes it necessary to shift assessment paradigms and adopt new procedures for assessment development. © 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 553–573, 2001     

The validity of performance assessments for teacher licensure: Connecticut''s ongoing research

This article was based on a paper presented at the annual meeting of the American Educational Research Association, San Francisco, March 27 to March 31, 1989.     

Interpretive validity theory: mapping a methodology for validating educational assessments

Background:?Validity theory has evolved significantly over the past 30 years in response to the increased use of assessments across scientific, social and educational settings. The overarching trajectory of this evolution reflects a shift from a purely quantitative, positivistic approach to a conception of validity reliant on the interpretation of multiple evidence sources integrated into validity arguments. Moreover, within contemporary validity, interpretation has been emphasised as a central process; however, despite this emphasis, there have been few explicit articulations of specific interpretive methodologies applicable to the practice of validation.

Purpose:?To link contemporary theoretical foundations in validity to practical methods and structures to help guide the collection and analysis of interpretive validity evidence. By building upon existing validity theory, this paper aims to provide greater clarity on the practice of validation and contribute toward the larger developing framework for the validation of educational assessments.

Source of evidence:?An interdisciplinary, integrative review of over 60 research articles and sources related to the theory and practice of educational validation and interpretive inquiry approaches. Sources include literature from the fields of educational assessment and more broadly social scientific research.

Main argument:?As assessments in education increasingly aim to measure complex constructs that are value-laden and socially dependant, validity theory must keep pace and evolve in ways that address the inherent complexities associated with contemporary educational assessment. Through this paper, I assert that a greater understanding of interpretive methodologies represents one of the most promising areas for development of validation theory and practice. Specifically, I argue that dialectic, hermeneutic and transgressive forms of inquiry can be integrated within current argument-based structures for the collection, analysis and representation of validity evidence in several useful ways.

Conclusions:?Interpretive inquiry processes, namely dialectic, hermeneutic and transgressive forms of interpretation, serve to expand validation practice to include diverse evidences for the generation of multiple-perspective validity arguments. The paper concludes with specific implications for future research and practice within the field of interpretive validity theory.     

Threats to the validity of science performance assessments for English language learners

In recent years, at the same time that performance assessments in science have become more popular, the number of English language learners (ELLs) (i.e., students whose native language is other than English) served by the U.S. educational system has also increased rapidly. While the research base is growing in each of these areas independently, little attention has been paid to their intersection. This case study of the use of a science performance assessment with 96 ELLs in five high school science classes investigated the face, construct, and consequential validity of this intersection. Qualitative and quantitative data analyses showed that both teachers and students had an overall favorable response to the assessment, although students' English comprehension and expression skills were determining factors for certain items. While most responses were reliably scored, ELL spelling and syntax on certain responses were significant sources of error. The degree of specificity of teachers' guidance also significantly affected students' scores. Recommendations from this study include increasing the clarity of an assessment's design, allowing ELLs more time to complete assessments, and scoring by raters who are knowledgeable about typical patterns in written English for this student population. Furthermore, it is recommended that the use of performance assessments with ELLs be exploratory until such time as their validity and reliability with this population can be more adequately established. J Res Sci Teach 34: 721–743, 1997.     

Intake assessments for alternative teacher education: moving from legitimation towards predictive validity

There is little empirical evidence concerning the predictive validity of indicators used in intake assessments for alternative teacher education programs. However, pinpointing potentially successful candidates requires predictors of teacher competence, which may be used to assess candidates’ dispositions, knowledge and skills. Available evidence regarding six possible predictors of teacher competence is examined and their limitations and possibilities are pointed out. Recommendations and methodological guidelines are given for developing and using these indicators in valid and reliable ways. Directions are explored for instrument development and assessment practice.     

Validity inquiry process: Practical guidance for examining performance assessments and building a validity argument

Given the increased use of performance assessments (PAs) in higher education to evaluate achievement of learning outcomes, it is important to address the barriers related to ensuring quality for this type of assessment. This article presents a design-based research (DBR) study that resulted in the development of a Validity Inquiry Process (VIP). The study’s aim was to support faculty in examining the validity and reliability of the interpretation and use of results from locally developed PAs. DBR was determined to be an appropriate method because it is used to study interventions such as an instructional innovation, type of assessment, technology integration, or administrative activity (Anderson & Shattuck, 2012). The VIP provides a collection of instruments and utilizes a reflective practice approach integrating concepts of quality criteria and development of a validity argument as outlined in the literature (M.T. Kane, 2013; Linn, Baker, & Dunbar, 1991; Messick, 1994).     

课程知识的合法性问题--对《基础教育课程改革纲要(试行)》的社会学分析

本文通过对《基础教育课程改革纲要》中课程知识合法性的追问发现,国家、专家等角色在本次基础教育课程改革中发生了变化。“国家”角色开始隐退,“专家”由“象征性参与者”向“事实性参与者”过渡,使课程知识合法性的主体发生了变化:课程将不纯粹是“传递统治阶级意识形态,进行阶级关系再生产的工具”;“专家”与“国家”在角色、地位上既有磨合又有争夺,从而也改变了课程知识合法性的过程:课程由社会单方“控制”的中介变为多方“认同”、“妥协”的产物。     

The validity of current criticism

BLACKS IN AMERICAN FILM: TODAY AND YESTERDAY. By Edward Mapp. Metuchen, N.J.: The Scarecrow Press, 1972; pp. 278. $7.50.

BLACKS IN AMERICAN MOVIES: A SELECTED BIBLIOGRAPHY. Compiled and edited by Anne Powers. Metuchen, N.J.: The Scarecrow Press, 1974; pp. 157. $6.00.

TOMS, COONS, MULATTOES, MAMMIES, AND BUCKS. By Donald Bogle. New York: The Viking Press, 1973; pp. 260. $12.50.

FROM SAMBO TO SUPERSPADE. By Daniel J. Leab. Boston: Houghton Mifflin, 1975; pp. 301. $15.00.

SELF‐EVIDENT TRUTHS: BEING A DISCOURSE ON THE ORIGINS &; DEVELOPMENT OF THE FIRST PRINCIPLES OF AMERICAN GOVERNMENT — POPULAR SOVEREIGNTY, NATURAL RIGHTS, AND BALANCE &; SEPARATION OF POWERS. By Paul K. Conkin. Bloomington, Ind.: Indiana University Press, 1974, p. xii + 211. $7.95.

BODILY COMMUNICATION. By Michael Argyle. New York: International Universities Press, 1975; pp. 403. $17.50.     

The power of alternative assessments (AAs)

This article starts by discussing the potential disadvantages of traditional assessment towards young English as a Second Language(ESL)learners within the American public school education system.In response to such disadvantages,researchers’call for the implementation of alternative assessments(AAs)is therefore introduced along with the various benefits of AAs.However,the current mainstream education policy in the US,namely No Child Left Behind(NCLB)Policy,is still largely based on the tra ditional ways of testing,making policy-oriented implementation of AAs on large scales remarkably difficult.After careful analysis,the author points out several implications concerning how,under such an existing policy of NCLB,can practitioners effectively accommodate young ESL learners by applying the power of AAs.     

The integrity of educational outcome measures in international assessments

Educational Assessment, Evaluation and Accountability -     

The Role of Consequences in validity Theory

How do individuals make sense of and use the products and practices of testing in their everyday lives? What is the responsibility of the educational measurement community to take these issues into consideration in assessing what it is that we do?     

思想政治教育有效性的实现

思想政治教育的有效性是思想政治教育的生命之所在。思想政治教育有效性的实现,关键在于:第一,思想政治教育诸条件性因素的结构性组合必须具有优化性、合理性与科学性。第二,思想政治教育有效性的实现过程,是一个教育影响力生成、接受、内化、体验与评价选择的复合过程。在这一过程中,每一个子过程的衔接与协调发展是形成整体过程有效性的基础。第三,思想政治教育有效性的实现,还必须得到合理而有力的保障机制的支持。只有这样,思想政治教育的目的才能最终转化为有效性的结果。     

The construct validity and predictive validity of a self-efficacy measure for student teachers in competence-based education

This study intends to investigate the validity of a self-efficacy measure which is developed for predictive and diagnostic purposes concerning student teachers in competence-based education. CFA results delivered converging evidence for the multidimensionality of the student teacher self-efficacy construct and the bi-factor model as underlying structure, reflecting a teacher competence framework. Factor loadings of the bifactor model evidenced the theoretical assumption that incipient student teachers enter the programme with a global undifferentiated sense of teacher self-efficacy, having teaching experiences a further differentiation takes place to a partly differentiated sense of teacher self-efficacy. Logistic regression analysis revealed that the measure succeeds in predicting students’ first-year outcomes and delivered evidence for the diagnostic value of the scale.