Analysis of the Effects of School Variables Using Multilevel Models

Multilevel models allow data to be analysed which are hierarchical in nature; in particular, data which have been collected on pupils grouped into schools. Some of the associated variables may be measured at the pupil level, and others at the school level. The use of multilevel models produces estimates of variances between schools and pupils, as well as the effects of background variables in reducing or explaining these variances. One data set which has been analysed relates to the national surveys of mathematics carried out in England, Wales and Northern Ireland. In this case the basic unit of analysis was a pupil's performance in a group of items within one of 12 sub‐categories of maths. Each pupil tackled two such item groups (or sub‐tests) and thus a three‐level model was required, with the levels representing sub‐tests, pupils and schools. A number of background variables at both pupil and school levels were also measured, and interesting results were obtained when a multilevel model was fitted. The program used was a version of one developed by Professor H. Goldstein. A quite different data set related to pupils’ responses to a questionnaire survey about their reactions to their current course of study. The dependent variable was a measure of pupils’ satisfaction with the course derived from their responses, and other pupil level variables were also derived, relating to their school experiences and personal attributes. School level variables such as size and type of school were obtained from a schools data base. The program Hierarchical Linear Model (HLM) was used to model these data, using only two levels. The two multilevel program used have different strengths and capabilities, but are related in terms of the kinds of models that can be fitted. Such models can lead to greater insights into the relationships between school and pupil level variables, and their influence on pupil results or attitudes.     

Three-Level Models for Indirect Effects in School- and Class-Randomized Experiments in Education

Due to the clustered nature of field data, multi-level modeling has become commonly used to analyze data arising from educational field experiments. While recent methodological literature has focused on multi-level mediation analysis, relatively little attention has been devoted to mediation analysis when three levels (e.g., student, class, school) are present in a study setting. This article presents analysis models that can be used to test indirect effects in experimental designs having three levels where random assignment is at the third (school) or second (class) level and where the indirect effect may be random. In the presentation, simulated datasets are used to illustrate model specification and results interpretation for hypothetical three-level educational experiments involving mediation and moderation of treatment effects.     

The questionable role of higher education as an occupational screening device

Employers optimally pursue activities which facilitate the coordinating of employee characteristics and the requirements of the job. One allegedly important employee characteristic is the level of education. Employees with higher levels of education are rewarded with higher wages than employees with lower levels. This may occur if higher levels of education make an employee truly more productive or if because of an employer's beliefs only those individuals with higher levels of education are allowed to enter the higher paying positions (occupational screening).The above propositions are testable, depending crucially upon the theoretical model employed for determining occupational choices. We shall compare the implications of two possible occupation choice models: (1) enter the job which offers the highest lifetime income, (2) enter the job which offers the highest level of overall satisfaction. We estimate these two models using the NBER-TH data sample. By distributing our estimated results and the actual distribution of occupations over the education levels of high school, some college and BA we can see if more or less people are expected to enter specific occupations at each education level. Support for screening exists if more people are expected in high status occupations at low education levels than are actually in those occupations.When comparing the estimated results for each model we see different outcomes emerge. The latter indicates that screening does not exist while the former does. We present arguments as to why we feel that the second model is the more correct and appropriate and, consequently, why we feel that education is not an effective screening device.     

Over-education: What influence does the workplace have?

The wage and job satisfaction impacts for over-educated workers have been well-documented; yet little attention has been paid to the consequences for firms. In this paper we examine over-education from the perspective of the workplace. Using linked employer–employee data for the United Kingdom, we derive the standard worker-level penalties on wages and job satisfaction. We then show how over-education rates across workplaces adversely influence workplace pay and workplace labor relations. For individual workers who may be at-risk of over-education, we also distinguish between workforce composition effects and workplace labor practices, such as hiring. The effect of over-education on job satisfaction is particularly strong and its effects are evident at the workplace level. Our results suggest that investigations of over-education at the level of the firm are a promising area of inquiry.     

Power Computations for Polynomial Change Models in Block-Randomized Designs

Education experiments frequently assign students to treatment or control conditions within schools. Longitudinal components added in these studies (e.g., students followed over time) allow researchers to assess treatment effects in average rates of change (e.g., linear or quadratic). We provide methods for a priori power analysis in three-level polynomial change models for block-randomized designs. We discuss unconditional models and models with covariates at the second and third level. We illustrate how power is influenced by the number of measurement occasions, the sample sizes at the second and third levels, and the covariates at the second and third levels.     

基于Stacking算法与多数据源的有效药物组合

对于癌症、心血管疾病等复杂疾病,采取组合用药克服耐药性和改善功效已成为标准治疗方案。鉴定药物组合标准的方法是进行体内或体外药物筛选实验,但这一过程很缓慢,代价高昂。各种高通量组学技术产生度量药物效应的各层次数据,使得从计算角度挖掘数据进而预测有效药物组合成为主流手段。针对有效药物组合的预测模型大多是利用单一机器学习模型建模。为获得更高的精度,提出一种新的有效药物组合预测方法。该方法充分利用5种不同层次的药物信息构建相似性特征,特别引入药物靶标的序列信息和功能信息,基于Stacking算法融合多个传统机器学习模型和最新的集成学习模型LightGBM。实验表明,该方法预测的AUC值为0.953,精度比单一机器学习模型有显著提升。     

The Importance of Identifying Levels in Multilevel Analysis: An Illustration of the Effects of Ignoring the Top or Intermediate Levels in School Effectiveness Research

In this article, the effect of ignoring one or more levels of variation in hierarchical linear regression analysis is explored. A model with four hierarchical levels is used as a reference model. A distinction is made between ignoring top and intermediate levels. The effects of ignoring levels on the fixed and on the random parameters of different random intercept models are explored by means of a real data set. The results show that ignoring an important level causes an effect on specific fixed coefficients, variance components and their corresponding standard error. Therefore, ignoring an important level can lead to different research conclusions.     

Comparison of Inverse Wishart and Separation-Strategy Priors for Bayesian Estimation of Covariance Parameter Matrix in Growth Curve Analysis

Growth curve modeling provides a general framework for analyzing longitudinal data from social, behavioral, and educational sciences. Bayesian methods have been used to estimate growth curve models, in which priors need to be specified for unknown parameters. For the covariance parameter matrix, the inverse Wishart prior is most commonly used due to its proper and conjugate properties. However, many researchers have pointed out that the inverse Wishart prior might not work as expected. The purpose of this study is to investigate the influence of the inverse Wishart prior and compare it with a class of separation-strategy priors on the parameter estimates of growth curve models. In this article, we illustrate the use of different types of priors with 2 real data analyses, and then conduct simulation studies to evaluate and compare these priors in estimating both linear and nonlinear growth curve models. For the linear model, the simulation study shows that both the inverse Wishart and the separation-strategy priors work well for the fixed effects parameters. For the Level 1 residual variance estimate, the separation-strategy prior performs better than the inverse Wishart prior. For the covariance matrix, the results are mixed. Overall, the inverse Wishart prior is suggested if the population correlation coefficient and at least 1 of the 2 marginal variances are large. Otherwise, the separation-strategy prior is preferred. For the nonlinear growth curve model, the separation-strategy priors work better than the inverse Wishart prior.     

Precollege Factors and Leading Indicators: Increasing Transfer and Degree Completion in a Community and Technical College System

Today, community colleges are challenged to maintain their historical identity of open access while increasing student success. This challenge is particularly salient in the context of performance-based funding models. These models create student achievements, which determine institutional levels of state funding. Therefore, these new student success metrics are important to the fiscal health of community colleges. In an effort to better identify the likelihood of meeting these metrics, some scholars have suggested leading indicators. The purpose of this study was to examine the effects of leading indicators on transfer to a four-year institution and associate degree completion for community and technical college students at Kentucky’s two-year public institutions for groups based on student characteristics. Logistic regression analyses showed that leading indicators do predict transfer to a four-year institution and associate degree completion, but with varying levels of affect. Earning 30 credit hours by the end of the first year, passing a summer class and completing a college-level English class had the greatest effect on transfer to a four-year institution and associate degree completion. For performance-based funding models to be most effective and fair, policies and practices should consider precollege factors in their models. Also, these findings have implications for institutional level policy-making and practices.     

Literacy as a Unidimensional Multilevel Construct: Validation,Sources of Influence,and Implications in a Longitudinal Study in Grades 1 to 4

This study examined the extent to which literacy is a unitary construct, the differences between literacy and general language competence, and the relative roles of teachers and students in predicting literacy outcomes. Much of past research failed to make a distinction between variability in outcomes for individual students and variability for outcomes in the classrooms students share (i.e., the classroom level). Utilizing data from 1,342 students in 127 classrooms in Grades 1 to 4 in 17 high-poverty schools, confirmatory factor models were fit with single- and two-factor structures at both student and classroom levels. Results support a unitary literacy factor for reading and spelling, with the role of phonological awareness as an indicator of literacy declining across the grades. Writing was the least related to the literacy factor but the most impacted by teacher effects. Language competence was distinct at the student level but perfectly correlated with literacy at the classroom level. Implications for instruction and assessment of reading comprehension are discussed.     

Exploring differential bundle functioning in mathematics by gender: the effect of hierarchical modelling

Researchers interested in exploring substantive group differences are increasingly attending to bundles of items (or testlets): the aim is to understand how gender differences, for instance, are explained by differential performances on different types or bundles of items, hence differential bundle functioning (DBF). Some previous work has modelled hierarchies in data in this context or considered item responses within persons, but here we model the bundles themselves as explanatory variables at the item level potentially explaining significant intra-class correlation due to gender differences in item difficulty, and thus explaining variation at the second item level. In this study, we analyse DBF using single- and two-level models (the latter modelling random item effects, which models responses at Level 1 and items at Level 2) in a high-stakes National Mathematics test. The models show comparable regression coefficients but the statistical significances of the two-level models are smaller due to the larger values of the estimated standard errors. We discuss the contrasting relevance of this effect for test developers and gender researchers.     

Do increases in national-level preschool enrollment increase student achievement? Evidence from international assessments

The main purpose of the study is to analyze whether globally observed trends towards preschool expansion have impacted student achievement in primary and secondary school. We use data from multiple study cycles of two international large-scale assessments that have a longitudinal component at the country level—PIRLS and PISA—and combine these data with a country-level measure of preschool enrollment rates as the main explanatory variable. Employing a multilevel regression with fixed effects for countries and years, we find that changes in preschool enrollment are unrelated to changes in average student achievement. Even after controlling for covariates on the individual and country levels, we do not find any support for the policy expectation that expanding preschool enrollment per se leads to better student achievement on country level.     

Counseling Students' Level of Ego Development and Perceptions of Clients

Several current models of counseling supervision provide an outline of the developmental stages of counseling students' growth in conceptual and behavioral skills. Research based on these models, however, has grouped students by experience level rather than by developmental level. In this study the authors investigated the relationship of 63 counseling students' level of ego development and level of experience with their perceptions of clients. Analysis of structural complexity and content of students' perceptions of eight actual clients and their level of ego development revealed no significant main effects or interaction of either ego level or experience level on the structural complexity of client perceptions. Students at high ego levels described their clients more frequently in interactional terms than did those at low ego levels.     

Investigating the relationship between students’ attentive–inattentive behaviors in the classroom and their literacy progress: Chapter 4 Model specifications,results and comments: fitting multi–level,non-recursive structural equation models

The models fitted in this chapter are consistent with the major proposition of the investigation. The overlap between students’ inattentive behaviors in the classroom and their literacy achievement is reciprocal and mediated by the dynamic, interdependent effects of prior and concurrent inattentive behaviors and literacy achievements, as well as being subject to background and contextual influences — both at the student level and at the class/teacher level. To this end, the results of fitting two bi-directional explanatory models to the cross-validated data are compared, the findings from which are examined as a basis for fitting a third multilevel, non-recursive, structural equation model. In sum, the findings indicate that whereas students’ inattentive behaviors have significant negative effects on their literacy progress, literacy achievement has notably stronger effects on decreasing their early and subsequent inattentive behaviors.     

Testing Measurement Invariance Across Unobserved Groups: The Role of Covariates in Factor Mixture Modeling

Factor mixture modeling (FMM) has been increasingly used to investigate unobserved population heterogeneity. This study examined the issue of covariate effects with FMM in the context of measurement invariance testing. Specifically, the impact of excluding and misspecifying covariate effects on measurement invariance testing and class enumeration was investigated via Monte Carlo simulations. Data were generated based on FMM models with (1) a zero covariate effect, (2) a covariate effect on the latent class variable, and (3) covariate effects on both the latent class variable and the factor. For each population model, different analysis models that excluded or misspecified covariate effects were fitted. Results highlighted the importance of including proper covariates in measurement invariance testing and evidenced the utility of a model comparison approach in searching for the correct specification of covariate effects and the level of measurement invariance. This approach was demonstrated using an empirical data set. Implications for methodological and applied research are discussed.     

The relative and absolute risks of disadvantaged family background and low levels of school resources on student literacy

There has been a long-lasting debate of whether the effects of family background are larger than those of school resources, and whether these effects are a function of national income level. In this study, we bring a new perspective to the debate by using the concepts of relative risk and population attributable risk in estimating family and school effects. The study uses data from the Programme of International Student Assessment (PISA), a large international comparative study of the skills of 15-year-old students in 43 countries. The study finds that: (1) there is a curvilinear association between family effects, measured by both relative and attributable risk, and national income level; (2) there is no association between school effects and national income level; (3) the risk associated with low levels of family background is larger than that of low levels of school resources, regardless of national income level.     

斯坦福(Stanford)成就阅读理解考试的原本和客户本的结构等价

本文是第一篇探索斯坦福成就阅读考试(第十版)的原本及其客户化版本的结构相似性的文章。研究分析是跨年级在多个观测变量(个别题目,题组,题包)上进行的。分析方法主要包括线性和非线性的探索性和实证性因素分析。分析结果表明在所有文章内的试题,都有不同程度的题组效应。在所有的模型当中,个别题目作为观测变量的模型的拟合度最低,题组作为观测变量的模型的拟合;其次,题包作为观测变量的模型的拟合度最高。在三种结构等性等级:同性等性(congenric),陶性等性(tau-equivalent)和并行等性(parallel)中,斯坦福成就阅读考试原本与其客户化版本的结构具有同性相似。     

Synchronized analysis model: Linking Gilbert's behavior engineering model with environmental analysis models

Performance technology has many analysis models and selecting which to use can be challenging. Arguably, the most prestigious and most used HPT model—a cause analysis model—is Gilbert's behavior engineering model (BEM). However, even this powerful cause analysis model has its limits; although it does examine environmental symptoms in general, it doesn't account for the organizational or environmental levels at which performance problems occur. For data on such levels the practitioner may turn to environmental analysis models such as those developed by Kaufman, Langdon, Rummler & Brache, or Rothwell. But the practitioner who uses both a cause analysis model and an environmental analysis model will be left with two sets of data that do not easily integrate into a useful guide to action. The model presented here—the synchronized analysis model (SAM)— is an effort to remedy this situation. By integrating the cause analysis model of Gilbert's BEM with levels derived from the environmental analysis models, the SAM offers the practitioner an enhanced tool for resolving performance problems.     

Modelling within-person idiographic variance could help explain and individualize learning

Learning analytics is a fast-growing discipline. Institutions and countries alike are racing to harness the power of using data to support students, teachers and stakeholders. Research in the field has proven that predicting and supporting underachieving students is worthwhile. Nonetheless, challenges remain unresolved, for example, lack of generalizability, portability and failure to advance our understanding of students' behaviour. Recently, interest has grown in modelling individual or within-person behaviour, that is, understanding the person-specific changes. This study applies a novel method that combines within-person with between-person variance to better understand how changes unfolding at the individual level can explain students' final grades. By modelling the within-person variance, we directly model where the process takes place, that is the student. Our study finds that combining within- and between-person variance offers a better explanatory power and a better guidance of the variables that could be targeted for intervention at the personal and group levels. Furthermore, using within-person variance opens the door for person-specific idiographic models that work on individual student data and offer students support based on their own insights.

Practitioner notes

What is already known about this topic

• Predicting students' performance has commonly been implemented using cross-sectional data at the group level.
• Predictive models help predict and explain student performance in individual courses but are hard to generalize.
• Heterogeneity has been a major factor in hindering cross-course or context generalization.

What this paper adds

• Intra-individual (within-person) variations can be modelled using repeated measures data.
• Hybrid between–within-person models offer more explanatory and predictive power of students' performance.
• Intra-individual variations do not mirror interindividual variations, and thus, generalization is not warranted.
• Regularity is a robust predictor of student performance at both the individual and the group levels.

Implications for practice

• The study offers a method for teachers to better understand and predict students' performance.
• The study offers a method of identifying what works on a group or personal level.
• Intervention at the personal level can be more effective when using within-person predictors and at the group level when using between-person predictors.