The Conventional Wisdom About Group Mean Scores

Not infrequently, investigators assume that reliability for groups is greater than reliability for persons, and/or that error variance for groups is less than error variance for persons. Using generalizability theory, it is shown that this conventional wisdom is not necessarily true. Examples are provided from the course evaluation literature and the performance testing literature     

Generalizability of Performance Assessments

How can the contributions of raters and tasks to error variance be estimated? Which source of error variance is usually greater? Are interrater coefficients adequate estimates of reliability? What other facets contribute to unreliability in performance assessments?     

Political activity in escalation of commitment: Sport facility funding and government decision making in the United States

In the United States, the decision to commit extensive public resources to sport facilities is a contentious topic of debate. Elected officials often commit substantial public resources to sport facility projects amidst contrary empirical evidence and mixed residential approval. This behavior not only implicates the presence of political activity to advance a course of action, but also suggests an escalation of commitment (EoC), the subject of this study. The authors implemented a collective case study approach to examine three municipalities with long-standing histories of subsidizing professional sport facilities. Data collected from influential elected officials and public leaders (N = 13) as well as documents and records revealed (a) political action to be essential in EoC with decisions involving multiple stakeholder groups; (b) the influence and interaction of political action within each EoC determinant; (c) the substantial role of support, resources, power, and influence in efforts to increase and prevent commitment to a failing course of action; and (d) noticeable spillover effects from use of political activity in EoC.     

Higher education and social change

An agenda for future higher education research is proposed which incorporates four interconnected elements: changing social contexts; their implications for higher education; mechanisms of interaction between higher education and society; higher education’s impact on society. The role of comparative research in investigating these topics is discussed and a set of priorities for future research questions is proposed. This final article draws heavily on the HELF final report published by the European Science Foundation entitled Higher Education Looking Forward: an agenda for future research and written by the author together with Jurgen Enders, Christine Musselin, Ulrich Teichler and Jussi Valimaa. The author acknowledges the contributions of his co-authors of the HELF report to the ideas and words contained in the present article.     

Estimators of Conditional Scale-Score Standard Errors of Measurement: A Simulation Study

This paper describes four procedures previously developed for estimating conditional standard errors of measurement for scale scores: the IRT procedure (Kolen, Zeng, & Hanson. 1996), the binomial procedure (Brennan & Lee, 1999), the compound binomial procedure (Brennan & Lee, 1999), and the Feldt-Qualls procedure (1998). These four procedures are based on different underlying assumptions. The IRT procedure is based on the unidimensional IRT model assumptions. The binomial and compound binomial procedures employ, as the distribution of errors, the binomial model and compound binomial model, respectively. By contrast, the Feldt-Qualls procedure does not depend on a particular psychometric model, and it simply translates any estimated conditional raw-score SEM to a conditional scale-score SEM. These procedures are compared in a simulation study, which involves two-dimensional data sets. The presence of two category dimensions reflects a violation of the IRT unidimensionality assumption. The relative accuracy of these procedures for estimating conditional scale-score standard errors of measurement is evaluated under various circumstances. The effects of three different types of transformations of raw scores are investigated including developmental standard scores, grade equivalents, and percentile ranks. All the procedures discussed appear viable. A general recommendation is made that test users select a procedure based on various factors such as the type of scale score of concern, characteristics of the test, assumptions involved in the estimation procedure, and feasibility and practicability of the estimation procedure.     

Beyond the comfort zone: A curriculum and assessment “immersion” experience in graduate early childhood teacher education

Abstract

A graduate course in Early Childhood Education assessment which combines coursework and a practicum with young children is designed and implemented using constructivist principles and practice. Teacher education is more effective when field work and coursework are combined, and when students are able to apply what they learn in class to real life experiences with young children [Journal of Early Childhood Teacher Education 21 (1) (2000) 65], learning is enhanced. This course was constructed in a way that incorporated theory, application, reflection, and reconstruction, allowing students an opportunity to rethink their teaching practice through the experience of an early childhood practicum. The key components of the model — the components that are thought to most effectively help students to apply theory to practice, change some of their ways of thinking about teaching and learning are the following: the relationship of the coursework to the setting, continuity from previous courses, the laboratory school environment, the intensive faculty involvement, the co‐constructivist, collaborative nature of the faculty team, the collaborative nature of student work, the use of the three curriculum models, the use of embedded assessment, formative nature of the assessment, informed teaching (on‐going use of student feedback), the tools provided to student that assist in the process, and the practical hands‐on assessment techniques.

Changing students’ ways of thinking about the teaching, learning and assessment process is considered a vital part of the course. [Journal of Early Childhood Teacher Education 17(1) (Marshall, 1996) 43] has identified constructivist principles to guide teacher education. She believes that students must experience first‐hand those behaviors that she hopes they would exhibit in their own teaching (Rand, 1999, p. 126). It is through the processes of the course that graduate students are challenged to experience the implementation of emergent curriculum, examine their own beliefs in relation to their practice, articulate why they do what they do and see how classroom assessment strategies help to support meaningful teaching and learning. Furthermore, collaborative practice is considered an important factor that encourages and facilitates reflection, supports a more multi‐dimensional view of teaching and assessing young children, and enables students to listen and observe more effectively and look for the impact of children making choices. Through participation in this course, students became more aware of the impact their view of children and respect for children has on their ability to implement effective and appropriate assessment practices in the classroom. Students gained more of a conceptual understanding of the role of assessment in informed teaching, and were better able to meet individual children's instructional needs.

Finlly, students learned how to use tools inquiry, collaboration and reflection that are likely to assist them in future personal professional growth.