Multilevel Rasch Modeling: Does Misfit to the Rasch Model Impact the Regression Model?

Abstract

Multilevel Rasch models are increasingly used to estimate the relationships between test scores and student and school factors. Response data were generated to follow one-, two-, and three-parameter logistic (1PL, 2PL, 3PL) models, but the Rasch model was used to estimate the latent regression parameters. When the response functions followed 2PL or 3PL models, the proportion of variance explained in test scores by the simulated student or school predictors was estimated accurately with a Rasch model. Proportion of variance within and between schools was also estimated accurately. The regression coefficients were misestimated unless they were rescaled out of logit units. However, item-level parameters, such as DIF effects, were biased when the Rasch model was violated, similar to single-level models.     

教学质量增值评价中数学模型的选择与应用

教学质量增值评价能够改善我国传统的学校评估方法,提高学校教学质量评价的公正性。本文搜集整理了某地区学生中、高考总成绩,在对数据进行标准化处理的基础上,通过分析数据特征,确定了一元线性回归、反比例回归、幂回归、指数回归和二次回归等五个比较适合的回归增值评价模型。设置"双差法"标准,对模型的优劣进行比较,得出一类学生和二类学生对应的最优模型为二次回归模型,三类学生对应的最优模型为指数回归模型,并计算了该地区160所学校的教学质量增值排名。最后,给出增值评价模型选择的相关建议。     

基于IRT模型的数学学业成就水平测试分析

对不同类型学校的774名有效被试实施数学学业成就水平测试,并应用IRT参数模型方法进行分析,得出四点判断:(1)测验分数、最优分数呈负偏态分布;(2)测验信息函数负向偏移,大体呈现双峰波形;(3)主观性试题与逻辑斯蒂模型的拟合性较差;(4)不同类型学校学生的数学学业成就水平存在显著性差异。     

A PROBABILISTIC MODEL FOR STUDENTS’ ERRORS AND MISCONCEPTIONS ON THE STRUCTURE OF MATTER IN RELATION TO THREE COGNITIVE VARIABLES

In this study, the effect of 3 cognitive variables such as logical thinking, field dependence/field independence, and convergent/divergent thinking on some specific students’ answers related to the particulate nature of matter was investigated by means of probabilistic models. Besides recording and tabulating the students’ responses, a combination of binomial and multinomial logistic regression techniques was used to analyze the data. Thus, students’ misconceptions as well as the compatible-with-the-scientific-view student’s answers were explored one by one in relation to the above 3 cognitive variables. The study took place with the participation of 329 ninth-grade junior high school pupils (aged 14–15). The results showed that mostly logical thinking and sporadically the other 2 cognitive variables were significantly associated with students’ answers. Interpretation of the results and implications for science education are discussed.     

Factors and conditions promoting academic resilience: a TIMSS-based analysis of five Asian education systems

It is well documented that academic achievement of students from families of low socioeconomic status (SES) tends to be below their more socially advantaged peers. Several studies have identified factors and conditions that facilitate academic success for disadvantaged students (i.e., promote academic resilience). However, one of the main criticisms of this body of research is in the set of variables that explain academic success for low-SES students and which is not very different from the variables that would explain academic success for all students. The objectives of this article are dual: firstly, to identify factors and conditions associated with academic success, regardless of student SES, and secondly, to identify factors and conditions associated with academic resilience, that is, exclusively for low-SES students. To this end, we used data from Singapore, South Korea, Hong Kong, Chinese Taipei, and Japan in the Trends in International Mathematics and Science Study (TIMSS) 2011. The study sample covered 23,354 students in 720 schools in the five countries. The strategy for analysis was driven by fit of logistic regression models, first predicting the probability of academic success and then subsequent identification of variables significant as predictors for success within the pool of low-SES students. Results indicated that variables, such as positive student attitude to mathematics, teacher confidence in student performance and the test language being spoken at home, were associated with greater chances of academic success. High academic expectations and time spent on mathematics at home demonstrated a differential effect between disadvantaged and non-disadvantaged students in Singapore. In Korea, being male (gender) and in Taipei, low levels of bullying at school, increased the likelihood of resilience. Results suggested that interventions impacting behavior reflected in differentially associated variables could help disadvantaged students to become academically resilient.     

Effective schools,school segregation,and the link with school achievement

This study examines whether 3 teacher-rated aspects of school effectiveness differ across school segregation profiles in Stockholm, and to what extent these indicators are associated with the academic achievement of 9th-grade students. Analyses were based on 2 cross-sectional data collections performed in 2014 and 2016, respectively (147 school units), one among teachers (n = 2,024) and the other among 9th-grade students (n = 9,151). Multilevel analysis was applied, estimating 2-level random intercept linear regression models. Results show that teachers’ ratings of school leadership, teacher cooperation, and school ethos, as well as student-reported marks differ across school segregation profiles. Findings further reveal significant associations between these school effectiveness indicators and student performance, even when taking student family background and the school’s student body composition into consideration. In part, these associations are also identified within segregation profiles. Moreover, results show that school ethos acts as a mediator between school segregation profile and student achievement.     

Exploring Person Fit with an Approach Based on Multilevel Logistic Regression

The idea that test scores may not be valid representations of what students know, can do, and should learn next is well known. Person fit provides an important aspect of validity evidence. Person fit analyses at the individual student level are not typically conducted and person fit information is not communicated to educational stakeholders. In this study, we focus on a promising method for detecting and conveying person fit for large-scale educational assessments. This method uses multilevel logistic regression (MLR) to model the slopes of the person response functions, a potential source of person misfit for IRT models. We apply the method to a representative sample of students who took the writing section of the SAT (N = 19,341). The findings suggest that the MLR approach is useful for providing supplemental evidence of model–data fit in large-scale educational test settings. MLR can be useful for detecting general misfit at global and individual levels. However, as with other model–data fit indices, the MLR approach is limited in providing information regarding only some types of person misfit.     

Freshman Year Dropouts: Interactions Between Student and School Characteristics and Student Dropout Status

Data from a large school district in the southwestern United States were analyzed to investigate relations between student and school characteristics and high school freshman dropout patterns. Application of a multilevel logistic regression model to student dropout data revealed evidence of school-to-school differences in student dropout rates and school-to-school differences in the relation between student characteristics and student dropout status. Investigation of the school differences revealed that school organization and schools' social context were statistically significant predictors of several of the dropout outcomes. Evidence of school context and school practice effects on student dropout outcomes suggests that school personnel may need to consider how characteristics of the school environment interact with the background characteristics of students when developing and focusing dropout prevention strategies. Implications for school policy and practice are discussed.     

Schulformwechsel im zweigliedrigen Schulsystem

This study focuses on students who change from the academic to the non-academic school track after grade 7 (probationary year) in Berlin. We examined which individual student characteristics (e.?g., performance, school biography, family background) predict school change after the probationary year. We were particularly interested as to whether there are secondary effects of social background, after controlling for differences in performance. Our analyses were based on data from a study which included a whole cohort of school track changers (N?=?754) who were compared with students that remained at the academic track (N?=?1470). In order to predict the change of school type, multivariate logistic regression analyses were performed. Results revealed significant differences between both groups regarding all observed characteristics. Performance-related characteristics were the best predictors of track change. After controlling for these characteristics, no further significant effects for other predictors were observable (i.?e., no secondary effects of social background were present).     

Socioeconomic trends in school bullying among Finnish adolescents from 2000 to 2015

Bullying at school has far-reaching impacts on adolescent well-being and health. The aim of this study was to examine trends in bullying at school according to socioeconomic adversities among Finnish adolescents from 2000 to 2015. A population-based school survey was conducted biennially among 14–16-year-old Finns between 2000 and 2015 (n = 761,278). Distributions for bullying, being bullied and socioeconomic adversities were calculated. Associations between bullying involvement, time and socioeconomic adversities were studied using binomial logistic regression with results shown by odds ratios with 95% confidence intervals. At the population level, the likelihoods of bullying and being bullied varied only slightly between 2000 and 2015. Bullying and being bullied were associated with socioeconomic adversities (low parental education, not living with both parents and parental unemployment in the past year). Unlike in the general population, the likelihoods of bullying and being bullied increased markedly among adolescents with most socioeconomic adversities. The increased socioeconomic differences in bullying involvement observed in this study add to the mounting evidence of polarization of adolescent health and well-being. Socioeconomic adversities should be considered in the prevention of bullying at school. In addition, socio-political measures are needed to decrease socioeconomic inequalities among Finnish adolescents.     

A Bayesian Hierarchical Selection Model for Academic Growth With Missing Data

Using a sample of schools testing annually in grades 9–11 with a vertically linked series of assessments, a latent growth curve model is used to model test scores with student intercepts and slopes nested within school. Missed assessments can occur because of student mobility, student dropout, absenteeism, and other reasons. Missing data indicators are modeled using logistic regression, with grade 9 and potentially unobserved growth scores used as covariates. Under a hierarchical selection model, estimates of school effects on academic growth and missingness are obtained. The results from the selection model are compared to a model that ignores the missing data process.     

Using Spreadsheets to Calculate Prob (X+Y=W)

Using the spreadsheet function SUMPRODUCT which calculates dot products, it is possible to compute the probabilities associated with the sums of independent discrete random variables. This allows the student to confirm properties of the sum of binomial and Poisson random variables. It also provides a method for computing the distribution of the sum of two (or more) arbitrary random variables. In addition, it also allows the student to compute the probabilities associated with the difference of random variables, and thus find the probability that one random variable exceeds another (or exceeds by a given amount).     

Factors behind low reading literacy achievement

The initial results of the Programme for International Student Assessment (PISA) indicated that Finnish and Swedish students' are among the best readers in all OECD countries. However, the literacy performance of 7% of Finnish and 12% of Swedish students' remains at a level which is not sufficient for further studies or active citizenship. This article reports a further comparative study which explores, compares and contrasts, by means of two‐level logistic regression models, students' personal, socio‐economic and cultural factors and their effects on low as opposed to average reading literacy achievement in Finland and Sweden. The results indicate that the risk of being a low achiever is strongly determined by gender and by several sociocultural factors as well as by students' personal characteristics, attitudes and activities both at and outside school. The constructed model was relatively similar and predicted with approximately equal degrees of probability membership in the risk group in both countries. This lays a solid foundation for joint pedagogic developmental efforts.     

An Investigation of Classification Consistency Indexes Estimated Under Alternative Strong True Score Models

Using several data sets, the authors examine the relative performance of the beta binomial model and two other more general strong true score models in estimating several indexes of classification consistency. It is shown that the beta binomial model can provide inadequate fits to raw score distributions compared to more general models. This lack of fit is reflected in differences in decision consistency indexes computed using the beta binomial model and the other models. It is recommended that the adequacy of a model in fitting the data be assessed before the model is used to estimate decision consistency indexes. When the beta binomial model does not fit the data, the more general models discussed here may provide an adequate fit and, in such cases, would be more appropriate for computing decision consistency indexes.     

两类零膨胀负二项回归模型在汽车保险定价中的应用

讨论了两种分布形式的零膨胀负二项回归模型,并应用一组实际汽车保险损失数据对两类模型进行了实证比较．结果表明,对于具有零膨胀特征的损失数据,零膨胀负二项回归模型的拟合结果优于普通索赔频率回归模型．     

To teach or not to teach: Negative selection into the teaching profession in Uruguay

Recruiting highly motivated and capable individuals to train and later work as teachers is key for improving instructional quality. Yet, research suggests students who performed best in secondary school often prefer other professions. We use a longitudinal retrospective survey of high-school graduates who completed the PISA-2009 assessment to determine, through multi-variate logistic regression models, to what extent the negative selection hypothesis applies to the Uruguayan case. Results show negative selection into teacher training exists in Uruguay, with individual, family and previous schooling factors playing a determinant role in this choice of tertiary studies.     

Investigating the effects of furloughing public school teachers on juvenile crime in Hawaii

Policymakers have long been concerned about the large social costs of juvenile crime. Detecting the causes of juvenile crime is an important educational policy concern as many of these crimes happen during the school day. In the 2009–10 school year, the State of Hawaii responded to fiscal strains by furloughing all school teachers employed by the Department of Education and canceling classes for seventeen instructional days. We examine the effects of these non-holiday school closure days to draw conclusions about the relationship between time in school and juvenile arrests in the State of Hawaii on the island of Oahu. We calculate marginal effects from a negative binomial model and find that time off from school is associated with significantly fewer juvenile assault and drug-related arrests, although there are no changes in other types of crimes, such as burglaries. The declines in arrests for assaults are the most pronounced in poorer regions of the island while the decline in drug-related arrests is larger in the relatively more prosperous regions.     

Generalized Linear Factor Score Regression: A Comparison of Four Methods

Factor score regression has recently received growing interest as an alternative for structural equation modeling. However, many applications are left without guidance because of the focus on normally distributed outcomes in the literature. We perform a simulation study to examine how a selection of factor scoring methods compare when estimating regression coefficients in generalized linear factor score regression. The current study evaluates the regression method and the correlation-preserving method as well as two sum score methods in ordinary, logistic, and Poisson factor score regression. Our results show that scoring method performance can differ notably across the considered regression models. In addition, the results indicate that the choice of scoring method can substantially influence research conclusions. The regression method generally performs the best in terms of coefficient and standard error bias, accuracy, and empirical Type I error rates. Moreover, the regression method and the correlation-preserving method mostly outperform the sum score methods.     

Predicting Student Success in a Major’s Introductory Biology Course via Logistic Regression Analysis of Scientific Reasoning Ability and Mathematics Scores

Studies over the last 30 years have considered various factors related to student success in introductory biology courses. While much of the available literature suggests that the best predictors of success in a college course are prior college grade point average (GPA) and class attendance, faculty often require a valuable predictor of success in those courses wherein the majority of students are in the first semester and have no previous record of college GPA or attendance. In this study, we evaluated the efficacy of the ACT Mathematics subject exam and Lawson’s Classroom Test of Scientific Reasoning in predicting success in a major’s introductory biology course. A logistic regression was utilized to determine the effectiveness of a combination of scientific reasoning (SR) scores and ACT math (ACT-M) scores to predict student success. In summary, we found that the model—with both SR and ACT-M as significant predictors—could be an effective predictor of student success and thus could potentially be useful in practical decision making for the course, such as directing students to support services at an early point in the semester.     

用普阿松分布逼近二项分布的误差估计

在概率论中有大量的实际问题需要用二项分布来求解,但二项分布的计算有时相当麻烦,计算量相当大,往往同普阿松分布来逼近二项分布以简化计算.对于这种逼近的近似程度如何,怎样估计其误差以达到预定的准确度等问题作了一些探讨.