Multilevel design of school effectiveness studies in sub-Saharan Africa

School-based improvement programs represent a core strategy in improving education because they can leverage pre-existing social and organizational structures to promote coordinated and comprehensive change across multiple facets of schooling. School-based programs are generally designed to be implemented by intact schools/districts, frequently making it infeasible or atheoretical to assign students within the same school to different conditions while ensuring study validity. Rather, studies frequently assign intact schools/districts to treatment conditions to accommodate the multilevel structure of schooling and the theory of action underlying many school-based programs. In planning such studies, effective and efficient design requires plausible values of the variance partition coefficients and the variance explained by covariates during the design stage. Using representative samples of each country, we develop empirical estimates of design parameters within and across 15 countries that are intended to inform and facilitate the efficient design of multisite cluster-randomized studies of school improvement in sub-Saharan Africa.     

Factors affecting 12th-grade students' physics achievement

The current study aimed to determine and predict 12th-grade students' physics achievement in terms of some selected cognitive, physics-related effective, personal, and socioeconomic characteristics in Turkey. Four hundred and ninety-six students (273 female and 223 male) from 18 high schools formed the study group. Findings indicated that cognitive characteristics including science achievement and physics course grades explained the largest amount of variance (60.6%) in the achievement by itself. Physics interest and physics motivation as well as gender and perception of teacher-directed activities also explained a small amount of variance in student achievement. Although none of the socioeconomic characteristic variables were found to be significant, the entire model accounted for 64% of the variance in physics achievement.     

Mathematics achievement in high- and low-achieving secondary schools

This paper identifies the amount of variance in mathematics achievement in high- and low-achieving schools that can be explained by school-level factors, while controlling for student-level factors. The data were obtained from 2679 Iranian eighth graders who participated in the 2007 Trends in International Mathematics and Science Study. Of the total sample, 1422 and 1257 students were from high- and low-achieving schools, respectively. Two-level hierarchical linear modelling was applied. The results indicated that of the total variance in mathematics achievement, 27.95 and 6.70% were due to between-school differences in the high- and low-achieving schools, respectively. Controlling for the school-level factors, the better-performing students were those with a higher level of confidence in learning mathematics in both samples. After controlling for the student-level factors, inadequacies in school resources and school type yielded the strongest link to achievement in the high- and low-achieving schools, respectively.     

Interpersonal Behavior,Laboratory Learning Environments,and Student Outcomes in Senior Biology Classes

The purpose of this study was to investigate associations between students' perceptions of their biology teachers' interpersonal behavior and their laboratory learning environments and their attitudinal, achievement, and performance outcomes. A sample of 489 students from 28 senior biology classes in eight schools in Tasmania, Australia completed the Questionnaire on Teacher Interaction (QTI) and the Science Laboratory Environment Inventory (SLEI). Responses to two attitude questionnaires, achievement on an external written examination, and performance in laboratory practical tests were used as student outcome measures. Statistical analyses supported the reliability and validity of the QTI and the SLEI when used with senior secondary biology students. We investigated associations between students' perceptions of teacher behavior and their laboratory learning environment with student outcomes, including the unique and common contributions of the QTI and SLEI to variance in student outcomes. Associations with students' perceptions of the learning environment were stronger for the attitudinal outcomes than for the cognitive or practical skills outcomes. Some commonality between the QTI and SLEI scales was found in their contributions to the variance in attitudinal outcomes, but not in their contributions to variance in cognitive and practical skill outcomes. © 1999 John Wiley & Sons, Inc. J Res Sci Teach 37: 26–43, 2000     

Role of linguistic and sociocultural diversity in reading literacy achievement: a multilevel approach

This study examined how linguistic and sociocultural diversity have an impact on the reading literacy outcomes of a representative sample of 3,549 first‐language (L1) and 208 second‐language (L2) fourth‐grade students in the Netherlands. A multilevel modelling analysis was conducted using Progress in International Reading Literacy Study 2006 data to explore to what extent linguistic background, socioeconomic status (SES), home and school literacy environment and reading attitudes explain differences in reading literacy achievement. Significant differences between L1 and L2 students were found with regard to reading literacy achievement, SES and the home and school literacy environment. Multilevel modelling analysis showed 34.7% of explained variance in reading literacy achievement, whereby the student level accounts for most of the explained variance. In the final model, linguistic background, SES, home and school literacy environment and reading attitudes were found to have a significant effect on reading literacy achievement.     

Closing the Gap: Modeling within-school variance heterogeneity in school effect studies

Effective schools should be superior in both enhancing students’ achievement levels and reducing the gap between high- and low-achieving students in the school. However, the focus has been placed mainly on schools’ achievement levels in most school effect studies. In this article, we focused our attention upon the school-specific achievement dispersion as well as achievement level in determining effective schools. The achievement dispersion in a particular school can be captured by within-school variance in achievement (σ²). Assuming heterogeneous within-school variance across schools in hierarchical modeling, it is possible to identify school factors related to high achievement levels and a small gap between high- and low-achieving students. By analyzing data from the TIMMS-R, we illustrated how to detect variance heterogeneity and how to find a systematic relationship between within-school variance and school practice. In terms of our results, we found that schools with a high achievement level tended to be more homogeneous in achievement dispersion, but even among schools with the same achievement level, schools varied in their achievement dispersion, depending on classroom practices.     

Classroom,Home and Peer Environment Influences on Student Outcomes in Science and Mathematics: An analysis of systemic reform data

Using secondary analysis of a large database from a Statewide Systemic Initiative, we examined the effects of several types of environments on student outcomes. Over 3 years, nearly 7,000 students in 392 classes in 200 different schools responded to a questionnaire that assesses class, home, and peer environments as well as student attitudes. Students also completed an achievement measure that, developed by scientists, teachers, and science educators, was not aligned with any particular curriculum. Students were enrolled in middle‐school science and mathematics classes in schools that had participated in the Statewide Systemic Initiative. Rasch analyses allowed us to compare across student cohorts and across schools. Findings confirmed the importance of extending research on classroom learning environments to include the learning environments of the home and the peer group. Although all three environments accounted for statistically significant amounts of unique variance in student attitudes, only the class environment (defined in terms of the frequency of use of standards‐based teaching practices) accounted for statistically significant amounts of unique variance in student achievement scores. The findings are supported by other studies of systemic reform in the United States.     

Homework and Student Math Achievement in Junior High Schools

In the school year 1994/1995 homework characteristics and effectiveness in math education among secondary schools were compared. The study examined the 1st year of secondary education (average age level: 12.3 years). The sample consisted of 28 schools, 56 classes and 1,394 students. The results show a lack of policy directed at the co-ordination of and increase in the amount of homework in Dutch schools. Nevertheless most math teachers give homework every lesson. Multi-level analysis shows that the amount of homework was the only homework variable related to achievement. It explained 2.4% of the variance. Frequency of homework and homework time were not related to achievement. There is a sharp contrast between Dutch and US teachers in ‘checking and grading’ of homework. However, in this study this variable was not related to math achievement.     

School effectiveness in 14 sub-Saharan African countries: Links with 6th Graders' reading achievement

In this article school characteristics linked with educational effectiveness in 14 sub-Saharan African countries are identified. Effectiveness has been defined in terms of students' literacy achievement at the end of Grade 6, after taking their social and academic backgrounds into account. The data used are from the second major educational policy research study conducted by the 14 countries that constitute the Southern African Consortium for Monitoring Educational Quality study (SACMEQ). This study, known as the SACMEQ II Project, collected questionnaire and test survey data from around 42,000 students in 2,300 schools in the 14 countries. Information about schools was obtained from questionnaires administered to teachers and principals. The multilevel methods (HLM) we used are appropriate for such analyses and the SACMEQ II nested data design. Schools are differentiated in terms of their social compositions, contexts, and resources (physical and human). Though student achievement is strongly associated with students' social and academic background in all SACMEQ countries, school effects vary across countries. Evident is a pattern of higher achievement in urban schools, with more resources and higher-quality teachers; achievement is typically lower in large schools and those that offer education in “shifts.” Policy implications are emphasized.     

School effects on Chilean children’s achievement growth in language and mathematics: An accelerated growth curve model

This study investigates school effects on primary school students’ language and mathematics achievement trajectories in Chile, a context of particular interest given its large between-school variability in educational outcomes. The sample features an accelerated longitudinal design (3 time points, 4 cohorts) together spanning Grades 3 to 8 (n = 19,704 students in 156 schools). The magnitudes of school effects on students’ growth trajectories were found to be sizeable (generally larger than school effects in Western industrialised countries) and moderately consistent across school subjects. School composition effects on student achievement status were found for both school subjects. However, there was no evidence of composition effects on student achievement growth. The study provides new evidence on the size and nature of school effects in a developing country context based on state-of-the-art methods (i.e., accelerated longitudinal and growth curve models).     

Universals and specifics in learning strategies: Explaining adolescent mathematics, science, and reading achievement across 34 countries

We examined whether strategies of memorization, transfer through elaboration, and metacognition accounted for reading, science, and mathematics achievement across 34 countries. 158,848 fifteen-year-olds completed a reading literacy test and a questionnaire. Of these students, 88,401 completed a science test, and 88,590 completed a mathematics test. We analyzed the data using multi-level regressions of Rasch-estimated test scores and modeled differences across countries and across schools. Students who reported using memorization strategies often scored lower in all subjects. Transfer through elaboration was not significantly linked to any achievement scores. Lastly, students reporting greater use of metacognitive strategies often scored higher. Compared to students in individualistic societies, to achievement scores of students in collective cultures were linked more strongly to schoolmates' use of metacognitive strategies and less strongly to their own use of metacognitive strategies. These results highlight how cultural contexts can moderate the links between adolescents' learning strategies and their academic achievement.     

Civic knowledge and open classroom discussion: explaining tolerance of corruption among 8th-grade students in Latin America

Abstract

The endorsement of anti-corruption norms is a normative assumption in legal systems with freedom of information acts, where citizens are expected to act as monitors of the public service. Tolerance of corruption counteracts this assumption. We studied tolerance of corruption among 8th graders from Latin-American samples of the International Civic and Citizenship Study 2009. We proposed a model where associations between students’ socioeconomic status (SES) and tolerance of corruption are explained by civic knowledge, authoritarianism and open classroom discussion. This model accounted for 36–43% of the variance within schools, and 87–96% of the variance between schools, across six countries. The socioeconomic gap in tolerance of corruption was mainly present between schools. In addition, students with higher civic knowledge were less tolerant of corruption, partially explained by authoritarianism, while open classroom discussion also had indirect associations with tolerance of corruption.     

The Relationship of Effective Schools Indicators and Changes in the Social Distribution of Achievement

ABSTRACT

This study focused on school effectiveness in terms of changes in the distribution of achievement across socio‐economic status (SES) for a cohort of 9700 students as they progressed from grade 1 to grade 3. The achievement/SES link was operationalized as the standardized mean difference in achievement (EFFSIZE) between samples of high and low SES students for 165 schools. EFFSIZE measures in reading and mathematics were analyzed in an hierarchical linear model which allowed an assessment of the impact of 13 school characteristics on initial (grade 1) status and, more importantly, on trends over time. The results indicate that, on the average, low SES students are at an initial disadvantage relative to their high SES peers in both subjects and that the difference widens across the first three grades. True school variance in slopes was found in mathematics but not reading, a result consistent with previous research. The school characteristics, which included six indicators based on the effective schools literature, were found to be ineffective predictors of these growth parameters.     

Factors influencing the academic achievement of the Turkish urban poor

This study estimates the individual and combined effects of selected family, student and school characteristics on the academic achievement of poor, urban primary-school students in the Turkish context. Participants of the study consisted of 719 sixth, seventh, and eighth grade primary-school students from 23 schools in inner and outer city squatter settlements. The findings indicated that the set of variables comprising student characteristics, including well-being at school, scholastic activities and support, explained the largest amount of variance in academic achievement among the urban poor. Although the effect sizes are small, family background characteristics and school quality indicators were also found to be significantly related to academic achievement. The implications of this study for improving primary schools in urban poor neighborhoods are discussed.     

Low expectations equal no expectations: Aspirations,motivation, and achievement in secondary school

Underachievement and failure to complete school have long-term negative consequences for students. Aspirations regarding completion of secondary school that predict achievement outcomes are related to factors amenable to intervention. This study investigates relationships between academic achievement and self-reported educational aspirations, motivation, affiliation with peers and teachers, and attributions. Survey participants were 5369 Year 10 and Year 11 students at 19 nationally representative secondary schools in New Zealand, and available achievement records were sourced for 2439 Year 11 students. Survey data were factor analyzed followed by further examination of relationships across demographic factors, self-reported aspirations, motivational factors (Doing My Best and Doing Just Enough), attributions, and interpersonal affiliations (Teacher and Peer). For Year 11 students, relationships between different factors and subsequent achievement were also analyzed. Students who indicated no aspiration to complete a school qualification were indistinguishable from those with low or moderate aspirations, and the analyses supported only two divergent groups comprising students with either low or high aspirations to complete qualifications. Aspirations were significantly related to different patterns of motivation, affiliation, and attributions predictive of academic achievement. Students of different ethnicity and gender also fell unequally across the two groups. These results suggest that promoting low or even moderate expectations and aspirations for student achievement may actually reinforce lower academic achievement. Instead, teachers and schools should communicate high expectations to prevent school failure and effective interventions to enhance student outcomes.     

Is exam hell the cause of high academic achievement in East Asia? The case of Japan and the case for transcending stereotypes

High student achievement across East Asia is often explained as an outcome of highly competitive, stress‐inducing college entrance exams across the region. This ‘exam hell’ drives students to study longer and harder than their peers worldwide, a race that leads—unsurprisingly—to higher marks in international comparisons such as the Programme for International Student Assessment (PISA). Yet, the race comes at a considerable cost: psychological pressure, forfeited childhoods, regimes of rote memorisation, lack of creativity and private outlays for cram schools. In this article, we seek to refute this outdated narrative by focusing on changes in Japanese education over the past three decades. We first analyse quantitative data for learning time based on three longitudinal surveys conducted by researchers in Japan to show that learning time for Japanese upper secondary students declined during 1980–2000 (Study 1). We then turn to analyse PISA 2015 data to show that now Japanese students study less than their peers in major Anglo‐American countries: the USA, the UK and Australia (Study 2). These findings run counter to well‐worn images frequently rehearsed in both the academic and popular literature. In helping to remove one of the most persistent stereotypes about East Asian education, we seek to help make visible other explanations for East Asian student achievement.     

Sense of coherence and academic achievement of domestic and international students: a comparative analysis

Traditional models of educational outcomes relate academic achievement to university experiences controlling for background characteristics, like former levels of achievement. In these models, most of the variance in the outcome under consideration is explained not by experiences inside the university but by background characteristics, such as prior levels of academic achievement. In many instances the contribution of institutional experiences to outcomes under consideration is small. To date, researchers have not included sense of coherence (SOC) among background characteristics. In the current study traditional models are modified to include SOC as a possible contributor to first year academic achievement among domestic and international students with English and other first languages at four Canadian universities. It is found that a model including SOC better fits data for commuter and residence students than a model in which SOC is omitted. Although the effect of SOC on first year academic achievement is small, it is larger than the effects of some institutional experiences. As a result, SOC should be included in attempts to explain first year academic achievement.     

Perspective matters: The internal/external frame of reference model for self- and peer ratings of achievement

The internal/external frame of reference (I/E) model posits that students' academic self-concept in one domain (e.g., math) is positively associated with achievement in the same domain but negatively associated with achievement in a comparison domain (e.g., English). Whereas academic self-concept encompasses self-ratings of achievement, peer academic reputation (PAR) encompasses peer ratings of the same achievement and PAR was shown to have an incremental predictive effect on a range of positive academic outcomes. Considering the importance of PAR, we hypothesized that the I/E model would also apply to PAR both as predictor and as outcome. We tested this hypothesis in a sample of 850 seventh-grade students in 47 classes in math and English. Therein, PAR was measured in a round-robin design where students rated each classmate's level of achievement. We tested I/E models with academic self-concept and PAR as outcomes. Results supported the I/E model for academic self-concept as outcome with academic achievement and PAR as predictor but not for PAR as outcome with achievement as predictor. For PAR as outcome, the effects across domains were positive rather than negative. These results were consistent across replications over three years using the same sample. Thus, whereas academic self-concept was characterized by contrasts between domains, PAR was characterized by assimilation between domains.     

Academic outcomes from between-class achievement grouping: the Australian primary context

Grouping students by academic achievement level has been practised in a wide variety of forms and contexts for over a century. Despite a general consensus in the research that between-class achievement grouping provides no overall benefit for students, the practice has persisted in various guises. Between-class achievement grouping is common in high schools, and is also practised in a number of primary schools in various countries. While the affective outcomes of such practices have been investigated recently, academic outcomes at primary level have not been studied in recent decades. This paper examines the academic outcomes of between-class achievement grouping in literacy and numeracy classes in Australian primary schools. Results from standardised tests are compared between two groups of schools—one regroups students for these areas, and one maintains mixed-achievement classes. It is argued that the current regrouping practice closely resembles streaming and provides no apparent academic advantage for students.     

Low-stakes performance testing in Germany by the VERA assessment: analysis of the mode effects between computer-based testing and paper-pencil testing

The German school system employs centrally organized performance assessments (some of which are called “VERA”) as a way of promoting lesson development. In recent years, several German federal states introduced a computer-based performance testing system which will replace the paper-pencil testing system in the future. Scores from computer-based testing are required to be equivalent to paper-pencil testing scores so that the new testing medium does not lead to disadvantages for students. Therefore, the current study aimed at investigating the size of the mode effect and the moderating impact of students’ gender, academic achievement and mainly spoken language in everyday life. In addition, the variance of the mode effect across tasks was investigated. The study was conducted in four German federal states in 2019 using a field experimental design. The test scores of 5140 eighth-graders from 165 schools in the subject German were analysed. The results of multi-level modelling revealed that students’ test scores in the computerized version of the VERA test were significantly lower than in the paper-pencil version. Students with a lower academic achievement were more disadvantaged by the VERA computerized test. The results were inconsistent regarding the interactions between testing mode and students’ gender and mainly spoken language in everyday life. The variance of the mode effect across tasks was high. Research into different subjects and in other federal states and countries under different testing conditions might yield further evidence about the generalizability of these results.