Countering the Critics: Responses to Recent Criticisms of School Effectiveness Research

Criticisms of school effectiveness research (SER) from recent books are addressed. Fourteen criticisms are presented, followed by counterpoints, including: (1) many criticisms are based on simplistic/skewed readings of SER; (2) there is a wide diversity in SER; no single viewpoint prevails, as implied by critics; (3) many SER researchers have reported the direct/indirect impact of social class on student achievement, instead of ignoring it as implied by critics; (4) SER researchers believe schools have an impact beyond that of social class, and educators should influence what they can in their schools/classrooms; critics pessimistically focus on the social class/student achievement relationship; (5) pragmatism is a valid paradigm for studying schools; indeed, it has advantages over “critical” theory employed by other commentators. Resolution of these issues is unlikely given differences in commentators' world views.     

Is the School Composition Effect Real?: A Discussion With Evidence From the UK PISA Data

The school composition effect has attracted attention from researchers who believe that progress towards equality of educational opportunity is compromised by the existence of schools with “unbalanced” social rolls. Evidence for 2 central hypotheses in the “mix” theory is tested by an analysis of UK PISA data. It is suggested that the school composition effect is sometimes an artefact of statistical procedures and caused by “unmeasured” within-SES school selection for noncognitive dispositions and variable family resources within social classes. Some implications for the debate into the origins of social differences in education and the limits of “effective schooling” are discussed.     

AN INTRODUCTION TO THE MAJOR RESEARCH PARADIGMS AND THEIR METHODOLOGICAL IMPLICATIONS FOR SPECIAL NEEDS RESEARCH

Recently in special needs research qualitative methods have been advocated and utilised. Unfortunately, confusion exists concerning the methods of investigation and the researchers’ paradigmatic assumptions. We argue that it is the paradigmatic lens of the researcher which guide the whole research process rather than the methods employed. In this paper the three major paradigms currently operating in educational research will be described and the distinction between “method” and “paradigm” will be clarified. Although the issues raised here are applicable to all strands of educational research, the purpose of this paper is to guide special needs researchers in identifying their own assumptions and examining for themselves the implications of choices about research methodology based on those assumptions.     

Stichwort: Familie, Peers und (informelle) Bildung im Kindes- und Jugendalter

The importance of non-school educational stimuli which arise in “normal” everyday life (mainly through experience and “learning from life”) and the contribution of such stimuli to the educational development of young people are often underestimated. The educational value of the widely differing casual educational structures in family and peer relationships is often not fully appreciated in educational research until such structures are combined with formal school education. So far, researchers have only just begun to explore the value of informal education and to venture beyond the “traditional” limitation of empirical educational research in Germany to formal, institutionally organised education and competence acquisition processes for pupils. In recent years, attention has not only been directed towards the family as an independent place of education—studies have also focused on real life informal interaction situations in peer relationships and their importance as significant educational factors in the context of unsupervised child daytime activities. The researchers tend to regard the family as a “different type of educational sphere” and peer relationships as social interactions between equals with considerable educational potential, involving social and cultural exchange processes which constitute “everyday education” and which differ in their effects depending on the milieu. In view of the enormous differences in the way in which children and young people are integrated into family and peer-based everyday situations, it is important to point out the high educational significance of family and peer relationships for informal education, and this must always be analysed in relation to the social milieu of the family of origin and the type of school attended by the young people. The existing research initiatives and results are discussed here in relation to various research questions. The varied considerations in relation to the significance and impact of peer relationships and the existing approaches adopted in research serve as a starting point for further suggestions for future research, and these desirable new directions in research are then discussed from a theoretical and methodological perspective.     

School Effectiveness Research and its Critics: alternative visions of schooling

The paper examines (a) the values and assumptions which underpin school effectiveness research, and (b) the problem of applying its findings in schools. It does so in the light of a number of critiques of this research emanating from both the USA and the UK. The author argues that the issues at stake between school effectiveness researchers and their critics are best understood as disagreements about the nature and purposes of schooling rather than disagreements about the facts. What is required is more dialogue about the values and assumptions which shape both SER and the views of its critics. This paper attempts to provide an initial framework for starting such a dialogue.     

The (mis)identification of ineffective classroom teaching practice: critical interrogations of classroom teacher effectiveness research

This paper critiques specific forms of classroom teacher effectiveness research. In doing so, the paper suggests that education policy-making deems and employs teacher effectiveness research as a promising and capable contrivance for the identification of ineffective classroom teaching practice. The paper engages with this policy debate by using a specific policy example from the Australian state of Victoria, the Victorian Department of Education and Early Childhood Development (DEECD) Blueprint for Government Schools (2003/2008). Moreover, the attention given to “teacher effectiveness” as the means by which school systems aim to reverse student under-achievement positions classroom teachers as the controlling authority over educational outcomes. Indeed, teacher effectiveness is the defining quality of a policy-making debate that at its core dispenses with broader considerations of possible influence thought to substantially affect the learning outcomes of public school students.     

School composition and peer effects

Despite over 30 years of research into the effects of school composition or “mix,” there is remarkably little consensus over the nature and size of school compositional and peer effects. Developing an analytical review of international research in this area, this chapter begins by discussing conceptual and methodological problems related to theorising school compositional effects. A model is developed through which the adequacy of existing studies for capturing compositional effects can be judged. This model helps to explain why existing studies have failed to reach consensus since the modelling techniques vary and none approach the adequacy criteria articulated in the model. Qualitative research can go a long way towards illuminating the subtle processes that are likely to underlie school compositional effects. The inherent difficulties in “measuring” elements of compositional effects have left debate around the importance of school compositional effects open to political and ideological considerations, since the outcomes of key policy decisions such as school choice, effectiveness, and accountability depend on the findings.     

Alternative models of school effectiveness put to the test

The term “educational effectiveness” designates causal models of educational outcomes that may or may not contain school-level variables. The term “school effectiveness models” is used in the more restricted sense of outcome-oriented models that explicitly contain school-level variables. These models are categorized to a context-input-process-output structure, are multi-level, recognize causal chains, and sometimes include feedback loops. In this chapter the advantages of comprehensive vs. partial school effectiveness models are discussed. Furthermore alternative interpretations of the idea that higher-level variables some way facilitate lower-level variables in the production of educational outcomes are given. The debate concerns: (i) additive vs. interactive models, (ii) direct vs. indirect models, (iii) contextual vs. genuine effects, (iv) additive vs. synergetic interpretations, and (v) recursive vs. non-recursive models. In this chapter the results of empirical research are presented to test the first four completing interpretations. The results indicate that the indirect model is the least productive. The synergetic model looks promising, but more simple interactive models account for almost the same percentage of variance in students outcomes. The actual problem might be that the most likely school effectiveness models are indirect, non-recursive, contextual, synergetic, and interactive at the same time. To avoid complexities in future research decompositions are advocated. And therefore it is contended that organizational variables of interest first be examined on their possible alternative interpretation of cross-level facilitation, before formulating the model.     

Contextualising Catholic school performance in England

Despite there being significant numbers of state maintained Catholic schools in England, they have, until recently, proved to be of interest only to a minority of researchers. Government initiatives to promote greater scrutiny and accountability through the publication of school test and examination results have generated interest in their academic performance from proponents and critics. Early attempts by government to compare school performance using ‘raw’ examination and test scores were severely criticised. Newly introduced ‘Contextualised Value Added’ measures, when correctly applied, have largely overcome the deficiencies of earlier datasets. This paper presents new CVA performance data provided by Ofsted that may help the debate about Catholic school effectiveness to progress. Possible reasons for the CVA findings are explored and further areas for research are suggested.     

How “Pathetic” is Your Hiring Process? An Application of the Lessig “Pathetic Dot” Model to Educational Hiring Practices

Now, more than ever, the baby-boomer administrators are retiring in massive waves. Some researchers have predicted as high as 75% turnover by 2011 for education alone. For many years, not much attention has been placed upon succession planning or developing new leaders in some colleges; others have tended to create stockpiles of leadership candidates with not enough internal leadership positions to go around. In turn, in many schools this has left the talent pool of qualified individuals to be very small, and it creates a problem, not only for educational administrators but also, especially, for those in the entrepreneurial departments of community colleges. There has been much research about “getting the right person on the bus” and “moving from good to great,” yet no one, to the knowledge of the researchers, has extended industrial-organizational psychology research on “person-fit” into community college administration research. The purpose of this study was to examine the theoretical constructs of the educational hiring process. In turn, the application of the Lessig “Pathetic Dot” model is used to create a framework to examine hiring practices and possibly show some deficiencies. This is important, now more than ever, with community college administrations having to turn to external independent sources for validation of their practices and procedures. Finally, a brief discussion of how to strengthen the hiring process, in a time of heavy turnover and attrition, concludes the discussion. The authors also present practical applications, suggestions for future research, and implications for practitioners.     

Sociological and Political Concerns about School Effectiveness Research: Time for a New Research Agenda

In recent years school effectiveness research (SER) has become increasingly criticised for being a socially and politically decontextualised body of literature which provides support for inequitable educational reforms. This article demonstrates that SER proponents have not responded much to these criticisms and suggests that this is primarily because they do not share the same epistemological commitments as their critics. Nevertheless it is argued that the concerns of critics should be taken seriously by SER proponents because they speak powerfully to a number of key problem areas within the SER field. Three such areas are discussed: the overclaiming of SER; the continued undertheorising of SER, and the inability of SER to control the political use of its findings. The article concludes by noting that some SER researchers are attempting to connect more with the sociological and political concerns of their critics and argues that this has to be the key SER agenda for the future.     

School Organizational Effectiveness and Chronic Absenteeism: Implications for Accountability

ABSTRACT

Chronic absenteeism in K-12 schools is strongly associated with critical educational outcomes such as student achievement and graduation. Yet, the causes of chronic absenteeism are complex, with environmental, family/individual, and school factors all affecting the likelihood of a student attending school regularly. This exploratory study examines whether school organizational effectiveness has the potential to moderate external influences on chronic absenteeism. Using school-level scores from the 5Essentials surveys, we find that, in traditional public schools, schools that are organized for effectiveness have lower rates of chronic absenteeism, while controlling for student demographics and grade level. In particular, schools with higher scores for “involved families” have lower chronic absenteeism. While charter schools in Detroit have significantly lower rates of chronic absenteeism than traditional public schools, we did not find an association between organizational effectiveness and chronic absenteeism in charter schools. This suggests that student sorting by school type may produce variation in chronic absenteeism rates that is not moderated by school actions. These findings have important implications for practice and policy, as educators seek to reduce chronic absenteeism in response to pressures from high-stakes accountability systems.     

How new technology is addressed by researchers in Educational Studies: Approaches from high-performing universities in China and the UK

There is a crisis of expectation in relation to educational technology. This is sometimes interpreted as a failure of academic researchers to disseminate their work to educational practitioners. However, another interpretation dwells on the lack of vision characterising such research. Because teachers often encounter research most intensely during their own pre-service and in-service education, we review academic research here through a snapshot of output from 10 leading university Education departments sampled in the UK and China. Empirical papers with a central interest in new technology were scarce, representing only around 10% of the sample. Research was strongly situated in “classroom” contexts, although, as critics have suggested, with limited attention to the wider ecology of those places, and with teachers being the focal interest as much as students. A “learning outcomes” research orientation was less common than an interest in process and practice. Although this was approached with different methodologies in China and the UK. Discussion addresses the challenge of effective and authoritative dissemination and those constraints from the political economy of research practice.     

Emotional Intelligence and Education: A critical review

In recent years there has been an increased interest in the role of emotional intelligence in both the academic success of students and their emotional adjustment in school. However, promotion of emotional intelligence in schools has proven a controversial pursuit, challenging as it does traditional “rationalist” views of education. Furthermore, research findings in this area have been inconsistent at best. In this article we discuss the following key questions relating to this important debate. What do we mean by emotional “intelligence”? What impact would improved emotional intelligence have on learners’ emotional health and well‐being, academic achievement, and other adaptive outcomes? Can emotional intelligence be taught? It is felt that these are the key issues for consideration in developing policy, practice, and further research in this area.     

Primary School Exclusions

Most of the focus of research and debate about school exclusions in the U.K. has been about secondary age pupils, which is not surprising in that secondary school exclusions are much more numerous than at the primary school stage, accounting for approximately 87 per cent of all permanent exclusions (DfE, 1992). However, the research reported here takes the longer view and thus hopes to inform preventative work with primary age children, who are displaying great difficulties in mainstream schools, and special schools. It is based on the hope that there might be more potential for “success” when children are helped earlier. As primary age exclusions are relatively unusual they may present a different order of concern, in relation to the excluded children themselves. This paper will examine evidence about the incidence and nature of primary school exclusions and debate some of the issues which make these exclusions a particularly serious focus for concern. The data from a national questionnaire, three regional case studies and 265 excluded children within them, is reported here. The research project was supported by a grant from the Economic and Social Research Council.     

Kämpfen gegen Windmühlen und reale Mächte: William Lottigs Tagebuch (1919–1921) als Ausdruck der politischen,pädagogischen und schulinternen Machverhältnisse in einer Hamburger Gemeinschaftsschule zu Beginn der 1920er Jahre

In his daily journal on the founding of the public experimental school, a “community school” at the Berliner Tor in Hamburg between spring 1919 and September 1921, Lottig describes the everyday issues confronting the principal of the “new school” at that time. These concern classroom instruction, teachers, parents, external pressures on the Berliner Tor-School, the relationship with the school administration, political issues prevalent in Hamburg at that time, ideological and philosophical debates as well as personal and family relationship problems, all of which Lottig describes in his journal. Lottig also noted the reasoning underpinning the development of the school experiments: the “old” schools in Hamburg had been closed, and the state had in their place established experimental schools. The journal clearly records the difficulties, issues and successes of a principal of one of the newly established community schools (Lebensgemeinschaftsschulen), which had been established as experimental schools. A perusal of the diary indicates that Jakob Robert Schmid’s sole and up to now only one known analysis of the journal comes off as biased and misleading. Schmid, professor of education at the University of Berne, had, at the beginning of the 1930s, only perused and analysed those portions of Lottig’s journal in which Lottig describes the rather turbulent if inspiring – and yet chaotic – operation of the community school in its first two years. While Schmid analysed these portions, he did not consider Lottig’s other, more favourable and constructive comments. Schmid also did not explain his one-sided selection of journal passages. Schmid brands Lottig and his team of teachers as educational novices and classifies the Berliner Tor-School as an “anti-authoritarian” institution, an experimental school like any other school experiment which overshoots the mark, not being educationally and institutionally meaningful. A more objective and principled approach in examining Lottig’s journal would have revealed that Lottig and his teachers were well aware of the main issue confronting the school, an issue that Schmid would also have found relevant: the relation of freedom and compulsion, within a setting that Lottig wanted to revitalise, to productively equilibrate without employing the customary disciplinary instruments. Lottig furthermore again and again points emphatically to the “growing pains” of all alternative schools (even when regulated by the state as an experimental school), whose goal it had been, to establish, even under difficult circumstances, a “new school-type” not utilising the traditional instruments of discipline, instruction and school management. This proves that Lottig was neither an educational ignoramus nor unaware of the basic issue of classroom instruction: how one can instruct with or without compulsion. Lottig’s goal had always been – and this Schmid also disregarded – to replace traditional, imposed, mandated or even self-imposed rules and regulations by new, commonly worked out rules. Lottig’s journal is a good example of the steadfast, unrelenting and energy-sapping aspiration of a school principal to balance the relation of school management versus a school’s self-development under the given circumstances. In addition, Schmid’s misinterpretation is a good example of how an observer, who hardly knew the Berliner Tor-School, would misuse this historical source by means of a biased interpretation to further his own views on scholastic education, views that Lottig himself would have preferred to provocatively examine – Schmid’s “authoritative pedagogy”, which goes beyond all authoritarian and non-authoritarian educational policies. What, then, would Lottig have recorded in his journal about a meeting with Schmid?     

A Partisan Evaluation‐‐John Elliott on school effectiveness

This paper is a response to ‘School effectiveness research and its critics; alternative visions of schooling’ by John Elliott (Cambridge Journal of Education, 1996, 26, pp. 199‐223). It is argued that Elliott knows little of the knowledge base, misinterprets what he knows and fails to support his criticisms with evidence. It is concluded that educational advance and improvement can be furthered by school effectiveness research.     

“语言—文化”模式:教育不公平的结构与行动——S.B.希斯的语言民族志研究

伯恩斯坦和布迪厄的“语言—阶层”模式以语言为抓手打开了教育不公平的微观运作机制,成为“语言与教育公平”议题的主导分析路径。然而,由于他们将社会阶层视为整体性概念,阶层内部的文化多样性被遮蔽。新读写研究开创者S.B.希斯将民族志引入语言学习研究,以民族志为中心建立了教育不公平的“语言—文化”分析模式和行动路径。这一分析模式强调“文化模式”对语言的形塑,超越“语言—阶层”模式的抽象性,展现了教育不公平微观运作机制的文化多样性。以此为基础,她打破研究者对民族志的“专权”,创造性地将民族志作为教师和学生实现语言文化自觉、学校教育走出再生产的行动工具。希斯的“语言—文化”模式从理论视角、研究方法和行动路径层面为“语言与教育公平”构建了另一范式。     

What works may hurt: Side effects in education

Medical research is held as a field for education to emulate. Education researchers have been urged to adopt randomized controlled trials, a more “scientific” research method believed to have resulted in the advances in medicine. But a much more important lesson education needs to borrow from medicine has been ignored. That is the study of side effects. Medical research is required to investigate both the intended effects of any medical interventions and their unintended adverse effects, or side effects. In contrast, educational research tends to focus only on proving the effectiveness of practices and policies in pursuit of “what works.” It has generally ignored the potential harms that can result from what works. This article presents evidence that shows side effects are inseparable from effects. Both are the outcomes of the same intervention. This article further argues that studying and reporting side effects as part of studying effects will help advance education by settling long fought battles over practices and policies and move beyond the vicious cycle of pendulum swings in education.