Princesses are bigger than elephants: Effect size as a category error in evidence‐based education

Much of the evidential basis for recent policy decisions is grounded in effect size: the standardised mean difference in outcome scores between a study's intervention and comparison groups. This is interpreted as measuring educational influence, importance or effectiveness of the intervention. This article shows this is a category error at two levels. At the individual study level, the intervention plays only a partial role in effect size, so treating effect size as a measure of the intervention is a mistake. At the meta‐analytic level, the assumptions needed for a valid comparison of the relative effectiveness of interventions on the basis of relative effect size are absurd. While effect size continues to have a role in research design, as a measure of the clarity of a study, policy makers should recognise the lack of a valid role for it in practical decision‐making.     

Separating arguments from conclusions: the mistaken role of effect size in educational policy research

ABSTRACT

Effect size is the basis of much evidence-based education policymaking. In particular, it is assumed to measure the educational effectiveness of interventions. Policy is being driven by the influential work of John Hattie, the Education Endowment Foundation, and others, which is grounded in this assumption. This article demonstrates the assumption is false and notes that, when criticized, proponents either attempt to inoculate themselves by listing (without checking) assumptions or use the specious reasoning that, however flawed their argument, no-one has disproved their conclusions.     

Closing the Gap: Modeling within-school variance heterogeneity in school effect studies

Effective schools should be superior in both enhancing students’ achievement levels and reducing the gap between high- and low-achieving students in the school. However, the focus has been placed mainly on schools’ achievement levels in most school effect studies. In this article, we focused our attention upon the school-specific achievement dispersion as well as achievement level in determining effective schools. The achievement dispersion in a particular school can be captured by within-school variance in achievement (σ²). Assuming heterogeneous within-school variance across schools in hierarchical modeling, it is possible to identify school factors related to high achievement levels and a small gap between high- and low-achieving students. By analyzing data from the TIMMS-R, we illustrated how to detect variance heterogeneity and how to find a systematic relationship between within-school variance and school practice. In terms of our results, we found that schools with a high achievement level tended to be more homogeneous in achievement dispersion, but even among schools with the same achievement level, schools varied in their achievement dispersion, depending on classroom practices.     

Stereotypes and teacher characteristics as an explanation for the class-specific disciplinary practices of pre-service teachers

This paper explores the importance of class-related stereotypes for the discipline practice of pre-service teachers and whether stereotypes and discipline practice are related to their students' outward appearance. Pre-service teachers were asked to assign adjectives to photographs of children from the lower and middle social classes and to choose disciplinary actions for photographs of disruptive situations involving children from these classes. Results show that 40% of pre-service teachers treated children unfairly based on class affiliation, of these 50% punished lower-class children more harshly. The unfair treatment of lower-class children was linked to class-related stereotypes and to indicators of social identity.     

Detection of explanation obstacles in scientific texts: the effect of an understanding task vs. an experiment task

The aim of this study is to analyse the effect of tasks on the detection of explanation obstacles when secondary school students read scientific texts. Students were instructed to read short passages under different task conditions, and to ask questions if necessary. Obstacle detection was operationalised in terms of the type of questions asked by the students. The experiment examined the influence of goals associated with the task of reading to understand a text vs. reading to perform a procedure described by the text (a science experiment). Significantly, more explanation obstacles were found in the understanding condition than in the experiment condition. Scientific text also had an effect on the explanation obstacles detected.     

Absolute effects of schooling as a reference for the interpretation of educational intervention effects

Knowledge of the absolute effects of schooling provides a useful reference for the interpretation of the effectiveness of educational interventions. We use discontinuities in test scores between the oldest pupils in one birth cohort and the youngest in the next to assess the absolute effects of schooling. Our study includes 90 % of all pupils in year-groups 4–6 of primary education (ages 7–10) in Northern Ireland. Assignment to year-groups is strictly determined by date of birth in Northern Ireland. This creates a situation which parallels randomized controlled experimentation. The findings support the view that the guidelines suggested by Cohen (in 1969) may be overly ambitious when evaluating the effectiveness of educational interventions.     

Investigating the testing effect: Retrieval as a characteristic of effective study strategies

Presently, the most common approach to examining the testing effect is using a free recall form of retrieval practice. In this experiment, we compared free recall to other retrieval-based study strategies including practice quizzing, test-generation, and keyword. We also examined the possible benefit of coupling these retrieval-based strategies with free recall. A total of 338 undergraduates were randomly assigned to one of the nine conditions: a repeated retrieval (study-test) learning condition paired with one of the four retrieval-based strategies, a single retrieval (study-study) learning condition paired with a retrieval-based strategy, or a rehearsal (study-study-study) condition. Following a 7-day delay, students completed a test assessing retention of information learned. There was a significant interaction between learning condition (repeated vs. single retrieval practice) and type of retrieval-based strategy. Free recall and practice quizzing were the most effective types of retrieval practice, and coupling testgeneration and practice quizzing with free recall led to significant benefits in performance.     

University teachers' thinking about lecturing: student evaluation of lecturing as an improvement perspective for the lecturer

Student evaluation of instruction (SEI) is a common practice in higher education. Despite a great deal of progress that has been made during the last two decades of SEI-research, some questions remain unanswered. One of the unsolved problems concerns theutility of SEI, i.e., its efficacy for improving instruction. Apparently there is a need for understanding ‘how university teachershandle SEI-feedback.’ In that perspective cognitive and motivational psychology stress the importance of the so-calledsubjective theory of a teacher about his lecturing. This empirical study was focused on teachers' conceptualisation of lecturing, as related to student ratings. For representing a teacher's subjective theory and his interpretation of SEI-feedback, theHiclas-algorithm of De Boeck and Rosenberg (1988) was used, corresponding to theirHierarchical Classes Model. Results indicate that the lecturer's interpretation of student ratings and his/her ‘thinking about lecturing’ are crucial elements for an effective use of SEI as an improvement strategy. It gives one explanation why student evaluation of university teaching does (or does not) change teaching behaviour. The author is indebted to Prof. Piet J. Janssen and Prof. Joost Lowyck for their assistance in this research.     

Sweating the small stuff in educational psychology: how effect size and power reporting failed to change from 1969 to 1999, and what that means for the future of changing practices 1

Methodologists have written for years about the importance of attending to important details in quantitative research, yet there has been little research investigating methodological practice in the social sciences. This study assessed the extent to which innovations and practices are adopted by researchers voluntarily. In particular, I use the case of power analysis and effect size reporting as the primary example, but I also examine other reporting behaviours. Results show that while observed power and effect sizes in the educational psychology literature tend to be strong, researchers do not seem eager to adopt practices such as reporting effect sizes and power, and neither do they tend to report their testing assumptions or the quality of their measurement. There is room for much improvement in how we attend to the basics of quantitative research, and it does not appear that persuasion and professional communication are effective in changing practice.     

试答钱学森之问：加强高校师生互动——基于控制生师比和班级规模的探讨

回答钱学森之问必须考虑师生互动这一重要因素,师生互动对于杰出人才培养的重要性不仅有理论支撑,也得到了实践证明。生师比和班级规模作为影响师生互动的两个重要外部指标,分别从宏观和微观两个层面共同反映师生互动的效果。＂985高校＂作为培养杰出人才的领头羊,其与美国顶尖大学仍有较大差距。通过借鉴美国顶尖大学办学经验,结合学校现状,985高校应缩小生师比、控制班级规模。     

Progress in the past decade: an examination of the precision of cluster randomized trials funded by the U.S. Institute of Education Sciences

ABSTRACT

This article examines the statistical precision of cluster randomized trials (CRTs) funded by the Institute of Education Sciences (IES). Specifically, it compares the total number of clusters randomized and the minimum detectable effect size (MDES) of two sets of studies, those funded in the early years of IES (2002–2004) and those funded in the recent years (2011–2013). Overall, the average precision in terms of MDES of studies in the recent cohort was more than double that of the early cohort (i.e. 0.48 compared to 0.23). The findings suggest a consistent and substantial increase in the precision of CRTs funded by IES in the past decade which is a critical step towards designing studies that have the potential to yield high-quality evidence about the effectiveness of educational interventions.     

救生艇中的生存：哈丁的南北政治观述评

救生艇理论是加勒特·哈丁（Garrett Hardin）所提出的有关全球环境恶化背景下的南北政治观。哈丁把南北国家（穷国和富国）隐喻为漂流在大海上的救生艇。当穷国的救生艇上的人处于危机状态向富国的救生艇求救时,哈丁认为,最理性的做法就是独自离去,谁也不救。以这个隐喻模型为依据,哈丁围绕南北关系论提出了反援助观,质疑和批评了世界银行的成立和所谓的绿色革命,阐述了执行关门主义移民政策的必要性与合理性。     

Examining the effect of class size on classroom engagement and teacher–pupil interaction: Differences in relation to pupil prior attainment and primary vs. secondary schools

It is widely recognized that we need to know more about effects of class size on classroom interactions and pupil behavior. This paper extends research by comparing effects on pupil classroom engagement and teacher–pupil interaction, and examining if effects vary by pupil attainment level and between primary and secondary schools. Systematic observations were carried out on 686 pupils in 49 schools. Multilevel regression methods were used to examine relationships between class size and observation measures, controlling for potentially confounding factors like pupil attainment. At primary and secondary levels smaller classes led to pupils receiving more individual attention from teachers, and having more active interactions with them. Classroom engagement decreased in larger classes, but, contrary to expectation, this was particularly marked for lower attaining pupils at secondary level. Low attaining pupils can therefore benefit from smaller classes at secondary level in terms of more individual attention and facilitating engagement in learning.     

我国中等收入者比重相对过低的现状、成因及其负面影响

自1978年以来,随着经济与社会的转型,使我国中等收入者的比重不断扩大,其发展势头极其迅猛。但从总体上来讲,由于城乡二元分割、不同经济所有制关系分割等原因,其比重仍然相对过低,以致产生消费倾向降低,有效需求不足;消费结构断层,持续发展受阻;社会分层畸形,不稳定因素增加以及人力素质下降,科技创新乏力等负面影响。     

解读方立天先生的佛教文化内涵——以因果报应思想为例

方立天先生在多种场合提到了佛教文化内涵。先生认为,佛教文化内涵包括信仰、哲学理念、伦理道德、文学艺术、民风习俗等五个方面。以因果报应思想为例,对先生佛教文化内涵的解释进行解读。     

Thinking about the school most of the time: studio as generative metaphor for critical reflection

ABSTRACT

Busy leaders need time to reflect and renew. They need to consider the particularities of their school and the ways in which they can work with others in the school community to address pressing issues, as well as to make future plans. The metaphor of the studio offers some helpful avenues for thinking how this reflection might occur. Artists use their studios for respite, not knowing, generating new ideas, risk taking, integrating theory and practice, exercising criticality and developing socially just practices. Leaders might take a lead from artists; however, there are both space and time challenges to making the studio a practical possibility.     

高职旅游管理专业实施项目教学的实证研究及效果测量——以《旅游市场营销》课一个教学项目设计为例

项目教学是师生通过共同实施一个完整的项目而进行的教学活动．它使学生能够运用相应的知识来完成指定的项目任务,实现理论与实践的结合。本文以《旅游市场营销》课一个教学项目设计为例,对项目教学法做了一些实证研究,并对教学效果进行了测量。     

基于融资约束的环境不确定性对投资效率的影响分析——以电子信息制造业上市公司为例

基于我国2013-2020年206家电子信息制造业A股上市公司相关数据,从融资约束视角,分析环境不确定性对投资效率的影响及影响的路径,并进行了检验。研究表明,环境不确定性会对企业投资效率产生负向影响,且环境不确定性程度每提高1%,投资效率则下降0.51%;融资约束在二者之间发挥着中介作用,并且放大了不确定性对投资效率的消极影响。     

The paradoxical attenuation effect in tests based on classical test theory: mathematical background and practical implications for the measurement of high abilities

The aim of this paper is to demonstrate the dramatic consequences the application of cut‐off points can have in the practice of identifying gifted individuals. The paradoxical attenuation effect describes the frequent situation in which measurements of the gifts and talents individuals possess are lower than their true values. However, in assessing the results of a measurement, one should suspect that the talent being measured is in actuality even lower than the score representing it. The practical implications of the paradoxical attenuastion effect are taken under discussion, including how high α errors and β errors are for specific IQ cut‐off points. For example, among persons with IQs of 130 and above, and using reliabilities of .80, most of the persons measured to be gifted are actually not gifted at all.     

The attitudes and opinions of Cypriot primary teachers about the use of the Internet for their professional development and as an educational tool

This article explores the attitudes and opinions of Cypriot primary teachers about the use of the Internet for their personal development and as an educational tool. It also examines the teachers’ concerns about the implementation of the Internet in schools and their opinions about what has to be done in order to ensure its effective integration into the educational settings. The research revealed that Cypriot primary teachers had positive attitudes towards using the Internet both for their professional development and as an educational tool. The only factor that appeared to have a significant positive relationship with the attitudes of the teachers was the degree of confidence in using the Internet. Additionally, teachers considered training in the use of the Internet as the most important need for its effective implementation in primary schools.