Predicting Reading Proficiency in Multilevel Models: An ANOVA-like approach of interpreting effects

This study used an analysis of variance (ANOVA)-like approach to predict reading proficiency with student, teacher, and school-level predictors based on a 3-level hierarchical generalized linear model (HGLM) analysis. National Assessment of Educational Progress (NAEP) 2000 reading data for 4th graders sampled from 46 states of the United States of America were used. The study found that both the rich and poor minority students in a rich school benefited the most in reading performance, whereas rich and average socioeconomic status non-minority students in a rich school being taught in the non-crowded classroom achieved considerably high in reading proficiency. Based on the 3-level HGLM analysis, the ANOVA-like approach enabled the researchers to predict reading proficiency and interpret predictors' effects in a simple fashion.     

Student trust in teachers and student perceptions of safety: positive predictors of student identification with school

This study examined the effects of student trust in teacher and student perceptions of safety on identification with school. Data were collected from one large urban district in an eastern state. Participants included 5441 students in 3rd through 12th grades from 49 schools. Students responded to surveys that assessed student trust in teachers, safety and identification with school. Free and Reduced Lunch (a proxy for SES), school level and ethnicity were included as control variables. Two hypotheses guided the study. Hypothesis 1 stated that student trust, safety and identification with school would positively covary. Hypothesis 2 predicted that student trust in teachers and perceptions of safety were school properties that would individually and collectively explain differences among schools in student identification with school. Both hypotheses were confirmed. The results of the hierarchical linear model indicated that student trust in teachers and school safety accounted for 98% of the 22% of variance that exists among schools in student identification. SES, school level and ethnicity were not significant predictors of identification with school. Findings suggest that student trust in teachers and perceptions of safety may significantly contribute to student identification above the effects of SES, ethnicity and school level.     

Educational Data Mining: Identification of factors associated with school effectiveness in PISA assessment

With the main goal of identifying the process factors associated with school effectiveness in secondary education, this work presents an innovative methodological proposal. Based on secondary data from the Spanish sample of PISA 2015, high- and low-effectiveness schools were selected by analysing the residuals of the school level (level 2) in multilevel models. Subsequently, decision trees were used to identify the process variables with a greater predictive power for the identification of high- and low-effectiveness schools. While the hierarchical linear models obtained show inter-school variance scores greater than 10%, decision trees achieve a precision greater than 90%. We conclude by analysing the suitability of using decision trees in data originating from large-scale assessments, and examining the obtained factors associated with school effectiveness.     

Teacher commitment and its effects on student achievement in American high schools

This study explored the effects of teacher commitment on student achievement. Three teacher commitment dimensions of organizational, professional, and student commitment were derived. The three-dimensional teacher commitment measurement model was tested by a confirmatory factor analysis. Then, the relationships among individual and organizational variables, teacher commitment, and student achievement were analyzed by a 2-level hierarchical linear modeling method. As the results, the greater portions of teacher commitment and student achievement variances were within schools. The individual and organizational variables had differential impacts on each teacher commitment dimension. Finally, while teacher commitment effects on student achievement were differentially found depending on teacher commitment dimensions at the individual level, there was no evidence to support significant impacts of teacher commitment on student achievement at the organizational level.     

Dimensions of national test performance: A two-level approach

The research project presented in this article was set in the Swedish school context and carried out on a set of compulsory national tests for English, Swedish, and mathematics used at the end of compulsory school. The aims were: (a) to gain a deeper knowledge of the internal structure of the tests and (b) to separate individual performance from school performance. Individual test results from a representative sample of approximately 30% of the cohort (n = 26,360) constituted the data. Factor analysis, of a basically confirmatory nature, was applied in the study. Modelling at the individual level resulted in a theoretically sound, 11-factor model demonstrating good fit. The corresponding analyses at the school level, using 2-level structural equation modelling techniques, generated a similar, although somewhat less complex, model. Approximately 12% of the total variation in performance on the tests proved to be associated with schools.     

Measurement Bias in Multilevel Data

Measurement bias can be detected using structural equation modeling (SEM), by testing measurement invariance with multigroup factor analysis (Jöreskog, 1971;Meredith, 1993;Sörbom, 1974) MIMIC modeling (Muthén, 1989) or restricted factor analysis (Oort, 1992,1998). In educational research, data often have a nested, multilevel structure, for example when data are collected from children in classrooms. Multilevel structures might complicate measurement bias research. In 2-level data, the potentially “biasing trait” or “violator” can be a Level 1 variable (e.g., pupil sex), or a Level 2 variable (e.g., teacher sex). One can also test measurement invariance with respect to the clustering variable (e.g., classroom). This article provides a stepwise approach for the detection of measurement bias with respect to these 3 types of violators. This approach works from Level 1 upward, so the final model accounts for all bias and substantive findings at both levels. The 5 proposed steps are illustrated with data of teacher–child relationships.     

Narrowing in on Educational Resources That Do Affect Student Achievement

In an era dominated by issues of school finance adequacy, it seems particularly important to provide evidence that, despite a number of claims to the contrary, educational resources are indeed positively related to improved student achievement. One of the hypotheses of this article is that expenditures per pupil must be disaggregated into more meaningful categories to discern the relationship between resources and student achievement. To explore this question, this article uses data from the Washoe County School District in Reno, Nevada, which reports its school expenditures using a program called InSite. This program disaggregates expenditures into 4 categories: instruction, instructional support, leadership, and operations and maintenance. This school-level variable is the primary explanatory variable in this covariate adjustment model using a 3-level hierarchical linear modeling analysis of students (approximately 7,000) nested in classrooms (approximately 420) nested in schools (approximately 55). This model also includes a number of contextual and school compositional factors that research tells us affect student achievement, including student demographic characteristics and pretest score; teacher experience, education, and a measure of his or her instructional practice; and school size, school-level poverty, and expenditures broken out into 4 categories: instruction, instructional support, leadership, and operations and maintenance. The results show that expenditures for instruction and instructional support were positively related and statistically significant for the reading achievement of 3rd, 4th, 5th, and 6th graders in the 2002-03 school year.     

Different teacher-level effectiveness estimates,different results: inter-model concordance across six generalized value-added models (VAMs)

In this study, researchers compared the concordance of teacher-level effectiveness ratings derived via six common generalized value-added model (VAM) approaches including a (1) student growth percentile (SGP) model, (2) value-added linear regression model (VALRM), (3) value-added hierarchical linear model (VAHLM), (4) simple difference (gain) score model, (5) rubric-based performance level (growth) model, and (6) simple criterion (percent passing) model. The study sample included fourth to sixth grade teachers employed in a large, suburban school district who taught the same sets of students, at the same time, and for whom a consistent set of achievement measures and background variables were available. Findings indicate that ratings significantly and substantively differed depending upon the methodological approach used. Findings, accordingly, bring into question the validity of the inferences based on such estimates, especially when high-stakes decisions are made about teachers as based on estimates measured via different, albeit popular methods across different school districts and states.     

Effects of school quality,school citizenship policy,and student body composition on the acquisition of citizenship competences in the final year of primary education

This study examines the effects of general educational quality of schools, school citizenship policy, and ethnic homogeneity of the student body on the acquisition of citizenship competences in the final year of primary education. The theoretical framework is based on developmental, psychological, and sociological studies into effects of social context on educational outcomes and research into effective schools. The effects of school quality, school policy, and student population were analysed using 2-level (students, school) multilevel models. The results show that differences in citizenship competences between students and schools are mainly explained by factors at student level. Although the school also appeared to play a role, the school variables used in the analysis did not offer sufficient explanation for these differences. In order to further investigate the relationship between school factors and students’ citizenship, more insight is needed into characteristics of citizenship practices of schools.     

Equity in educational resources at the school level in Korea

This paper analyzed the equity of resources at the elementary school level in Korea using hierarchical linear modeling (HLM). The data included 2,327 Korean public elementary schools in 101 Local Governments within five Local Educational Offices (LEOs). This study found that schools in low property tax per resident areas receive fewer grants, showing a possible, positive relationship between wealth and educational resources. HLM analyses found that schools with higher percentages of free lunch students received more educational resources, indicating vertical equity. However, schools with higher percentages of special students received fewer resources, indicating inequity. These findings suggest recalibrating the school funding formula so that schools with high percentages of special students and with fewer grants from Local Governments could receive more funds from LEOs.     

Examining Intermediate Omitted Levels in Hierarchical Designs via Latent Variable Modeling

This article outlines an interval estimation procedure that can be used in a 3-level setting to evaluate the proportion of outcome variance attributable to the second level of clustering. The method is useful for examining the necessity of including a possibly omitted intermediate level of nesting in analyses of data from a multilevel study, and represents an informative addendum to current statistical tests of second-level variance. The approach is developed within the framework of latent variable modeling and can be used as an aid in the process of choosing between 2-level and 3-level models in a hierarchical design. The discussed procedure is illustrated with an empirical example.     

Analysis of the Effects of School Variables Using Multilevel Models

Multilevel models allow data to be analysed which are hierarchical in nature; in particular, data which have been collected on pupils grouped into schools. Some of the associated variables may be measured at the pupil level, and others at the school level. The use of multilevel models produces estimates of variances between schools and pupils, as well as the effects of background variables in reducing or explaining these variances. One data set which has been analysed relates to the national surveys of mathematics carried out in England, Wales and Northern Ireland. In this case the basic unit of analysis was a pupil's performance in a group of items within one of 12 sub‐categories of maths. Each pupil tackled two such item groups (or sub‐tests) and thus a three‐level model was required, with the levels representing sub‐tests, pupils and schools. A number of background variables at both pupil and school levels were also measured, and interesting results were obtained when a multilevel model was fitted. The program used was a version of one developed by Professor H. Goldstein. A quite different data set related to pupils’ responses to a questionnaire survey about their reactions to their current course of study. The dependent variable was a measure of pupils’ satisfaction with the course derived from their responses, and other pupil level variables were also derived, relating to their school experiences and personal attributes. School level variables such as size and type of school were obtained from a schools data base. The program Hierarchical Linear Model (HLM) was used to model these data, using only two levels. The two multilevel program used have different strengths and capabilities, but are related in terms of the kinds of models that can be fitted. Such models can lead to greater insights into the relationships between school and pupil level variables, and their influence on pupil results or attitudes.     

Factors promoting and hindering data-based decision making in schools

Although data-based decision making can lead to improved student achievement, data are often not used effectively in schools. This paper therefore focuses on conditions for effective data use. We studied the extent to which school organizational characteristics, data characteristics, user characteristics, and collaboration influenced data use for (1) accountability, (2) school development, and (3) instruction. The results of our hierarchical linear modeling (HLM) analysis from this large-scale quantitative study (N = 1073) show that, on average, teachers appear to score relatively high on data use for accountability and school development. Regarding instruction, however, several data sources are used only on a yearly basis. Among the factors investigated, school organizational characteristics and collaboration have the greatest influence on teachers’ data use in schools.     

How is the approach to teaching at secondary school related to first-year university achievement?

The study presented here is an elaboration on recent educational effectiveness research focusing on long-term school effects. Central in this study is the approach to teaching at secondary schools and its relation to student perception of the fit between school and university and 1st-year academic achievement. Based on previous research, a coding scheme is developed to classify pre-university schools based on the approach to teaching in the upper 3 years. Schools are scored according to this coding scheme and grouped with hierarchical cluster analysis, resulting in 3 groups: 1 characterized by a strong teacher-regulation approach, 1 with a shared regulation approach to learning, and 1 intermediate group. Multiple analysis of covariance shows differences between school clusters regarding student 1st-year achievement in soft science programmes.     

The investigation of school effects on student achievement in science: A multilevel analysis of educational data

The Second International Science Study provides a large Australian data base for the purpose of secondary analyses. This data base consists of a large number of student and school level variables which were examined with reference to the students nested within the schools. Multilevel analysis involves the use of the hierarchical linear model to adequately compensate for variability between-schools, as well as within-schools. The role of the school organization and effects such as the average student ability and average social factors were found to substantially influence student achievement in science. These school effects were also found to influence boys and girls differently with respect to their science achievement as measured by the tests in this study. Specializations: large scale data analysis, gender differences, socioeconomic factors, epidemiology.     

The use of a school value-added model for educational improvement: a case study from the Portuguese primary education system

This article focuses on the use of a value-added approach for promoting school improvement. It presents yearly value-added estimates, analyses their stability over time, and discusses the contribution of this methodological approach for promoting school improvement programmes in the Portuguese system of evaluation. The value-added model is applied to longitudinal data gathered from a representative sample of students enrolled in primary school, using mathematics scores as outcome variable. The estimates are used to determine a systematic positive or negative school value-added effect. Results indicate that at least 41% of schools remain at the same level over 3 years, 15% present an instable pattern, and approximately 40% show a positive or negative trend. It is argued that these results, especially those that reveal systematically underperforming schools, could have relevant implications for educational policy geared towards school improvement.     

Improving mathematics results: does teachers’ academic optimism matter? A study of lower secondary schools

Educational effectiveness research has provided evidence about the importance of teacher beliefs and attitudes for teaching and learning. This study builds on the concept of academic optimism, which combines 3 aspects of a teacher’s professional creed: self-efficacy, trust, and academic emphasis. The study explores the functioning of the collective and individual measures of academic optimism in the Czech environment and studies its impact on students’ outcomes. The analyses are based on pilot data from 39 schools, 325 teachers, and 1,316 Grade 9 students and on the data from the Czech Longitudinal Study in Education (CLoSE), covering 163 schools, 1,469 teachers, and 4,798 students. The individual measure was selected for further studies based on 2-level confirmatory factor analysis. Two-level structural equation modelling showed a significant impact of a school’s academic optimism on students’ achievement even after controlling for prior achievement and socioeconomic status at both the student and the school level.     

The Relationship of Effective Schools Indicators and Changes in the Social Distribution of Achievement

ABSTRACT

This study focused on school effectiveness in terms of changes in the distribution of achievement across socio‐economic status (SES) for a cohort of 9700 students as they progressed from grade 1 to grade 3. The achievement/SES link was operationalized as the standardized mean difference in achievement (EFFSIZE) between samples of high and low SES students for 165 schools. EFFSIZE measures in reading and mathematics were analyzed in an hierarchical linear model which allowed an assessment of the impact of 13 school characteristics on initial (grade 1) status and, more importantly, on trends over time. The results indicate that, on the average, low SES students are at an initial disadvantage relative to their high SES peers in both subjects and that the difference widens across the first three grades. True school variance in slopes was found in mathematics but not reading, a result consistent with previous research. The school characteristics, which included six indicators based on the effective schools literature, were found to be ineffective predictors of these growth parameters.     

A Canonical Correlation Analysis ofBehavior Problems and School Achievementfor Different Grades,Sexes, and Races

Abstract

Using education survey data from 6,883 Grade 6 students in 148 schools and from 6,868 Grade 8 students in 92 schools in New Brunswick, Canada, the author applied student and school characteristics to explain differences among students and schools regarding students' sense of belonging to school. Results of hierarchical linear modeling showed that in Grades 6 and 8, discrepancies in students' sense of belonging were mainly within schools, rather than between schools. At the student level, sense of belonging in both grades was affected more by students' mental and physical conditions and less by their individual and family characteristics. Students' self-esteem was the single most important predictor of their sense of belonging, followed by their health status. At the school level, school climate (academic press or expectation in Grade 6 and disciplinary climate in Grade 8) was more important than school context in shaping students' sense of belonging.     

Teacher hiring practices and educational efficiency

This paper analyses the relationship between teacher hiring practices and educational efficiency in Norwegian school districts. The hiring decision is made at the school level by the principal or at the school district level. According to the data, efficiency is the highest in districts where hiring is decentralized. Hiring practices are decided by the school district, and linear estimates of the effect of decentralized hiring on efficiency may be biased because of non-random selection. First, I approach this problem by including a large set of controls in a school district level analysis, which does not alter the qualitative result. Second, I perform a school level analysis with district fixed effects. The results indicate, as expected, that the effect of decentralization is stronger for schools facing excess teacher supply than for schools without excess supply.