Comparing effects and side effects of different school inspection systems across Europe

In this article, different inspection models are compared in terms of their impact on school improvement and the mechanisms each of these models generates to have such an impact. Our theoretical framework was drawn from the programme theories of six countries’ school inspection systems (i.e. the Netherlands, England, Sweden, Ireland, the province of Styria in Austria and the Czech Republic). We describe how inspection models differ in the scheduling and frequency of visits (using a differentiated or cyclical approach), the evaluation of process and/or output standards, and the consequences of visits, and how these models lead to school improvement through the setting of expectations, the use of performance feedback and actions of the school's stakeholders. These assumptions were tested by means of a survey of principals in primary and secondary schools in these countries (n?=?2239). The data analysis followed a three-step approach: (1) confirmatory factor analyses, (2) path modelling and (3) fitting of multiple-indicator multiple-cause models. The results indicate that Inspectorates of Education that use a differentiated model (in addition to regular visits), in which they evaluate both educational practices and outcomes of schools and publicly report inspection findings of individual schools, are the most effective. These changes seem to be mediated by improvements in the schools’ self-evaluations and the schools' stakeholders’ awareness of the findings in the public inspection reports. However, differentiated inspections also lead to unintended consequences as principals report on narrowing the curriculum and on discouraging teachers from experimenting with new teaching methods.     

Sensitivity of School-Performance Ratings to Scaling Decisions

Policymakers usually leave decisions about scaling the scores used for accountability to their appointed technical advisory committees and the testing contractors. However, scaling decisions can have an appreciable impact on school ratings. Using middle-school data from New York State, we examined the consistency of school ratings based on two scaling approaches that differed in scaling decisions that are important in high-stakes testing contexts. We found that, depending on subject, grade, and year, a switch in scaling approach led to (1) average absolute shifts in ranks of between 50 and 132 positions (median = 69), which are appreciable shifts for a listing of 1,243 schools; and (2) between 7% and 45% (average = 20%) of schools experiencing shifts in assigned performance bands, depending on the classification scheme. Further, the effect of scaling approach was larger when the raw-score distribution has more severe ceiling effect, and in these cases, it was driven primarily by the difference in the location of the highest obtainable scale score from the two scaling approaches.     

An Empirical Comparison of DDF Detection Methods for Understanding the Causes of DIF in Multiple-Choice Items

This study compared and illustrated four differential distractor functioning (DDF) detection methods for analyzing multiple-choice items. The log-linear approach, two item response theory-model-based approaches with likelihood ratio tests, and the odds ratio approach were compared to examine the congruence among the four DDF detection methods. Data from a college-level mathematics placement test were analyzed to understand the causes of differential functioning. Results indicated some agreement among the four detection methods. To facilitate practical interpretation of the DDF results, several possible effect size measures were also obtained and compared.     

A Comparison of Teacher Effectiveness Measures Calculated Using Three Multilevel Models for Raters Effects

This study examines the use of cross-classified random effects models (CCrem) and cross-classified multiple membership random effects models (CCMMrem) to model rater bias and estimate teacher effectiveness. Effect estimates are compared using CTT versus item response theory (IRT) scaling methods and three models (i.e., conventional multilevel model, CCrem, CCMMrem). Results indicate that ignoring rater bias can lead to teachers being misclassified within an evaluation system. The best estimates of teacher effectiveness are produced using CCrems regardless of scaling method. Use of CCMMrems to model rater bias cannot be recommended based on the results of this study; combining the use of CCMMrems with an IRT scaling method produced especially unstable results.     

Implementing a Multi-Tiered System of Support (MTSS): Collaboration Between School Psychologists and Administrators to Promote Systems-Level Change

Current educational reform mandates the implementation of school-based models for early identification and intervention, progress monitoring, and data-based assessment of student progress. This article provides an overview of interdisciplinary collaboration for systems-level consultation within a Multi-Tiered System of Support (MTSS) framework. The roles of school psychologists and school-based administrators are presented in relation to the implementation of MTSS practices within an implementation science model. The training and expertise of each discipline are highlighted related to respective aspects of implementation drivers (i.e., competency, organization, leadership). Functions of principals and school psychologists during team-based, problem-solving MTSS practices are described based on a problem-solving framework consistent with school-based consultation. Future directions for graduate training of school psychologists and principals and directions for consultation research are provided.     

Outside of School Matters: What We Need to Know in Urban Environments

This article focuses on factors outside of school that we have identified from the established research literature as essential for educators to consider in supporting students in urban environments. We start by conceptualizing what it means for a school to be “urban.” Following this discussion, we describe four outside-of-school factors that emerge from the literature: (1) Understanding Student and Family Homelessness; (2) Understanding Geography and Social Contexts; (3) Understanding Policy and School Funding; and (4) Understanding Parental and Family Involvement. These factors are important to helping educators, especially pre- and in-service teachers, in their understanding of and, consequently, the practices they use to meet the needs of students in these social contexts. We conclude with recommendations for educational practice.     

From Awareness to Action: College Students’ Skill Development in Intergroup Dialogue

A central goal of intergroup dialogue (IGD) is to strengthen individual and collective capacities to foster social justice commitments by supporting new ways of thinking about oneself, others, and the social structures in which we live. Relatedly, IGD assists individuals with building multicultural competencies and skill sets that support peoples’ capacities to take action and work towards social change. The following qualitative study contributes to current research by describing the types of skills undergraduate students report learning as a result of their IGD experience and the challenges they encounter as they develop communication, cognitive, and affective skills.