Globalised student achievement? A longitudinal and cross-country analysis of convergence in mathematics performance

ABSTRACT

With the aid of longitudinal country-level data from five IEA TIMSS assessments (1995–2011), the current study addresses the issue of the globalisation of curricula and achievement. To explore the hypothesis of global convergence, we study performance in four subdomains of mathematics. Using regression with fixed effects for countries, we consider whether the variation of subdomain scores decreases globally over time. Additionally, we explore qualitative differences in performance profiles using latent class analysis. Our results provide little evidence for a global harmonisation of student achievement. Rather, for regions with a similar language and culture, we observe similar strengths and weaknesses in mathematics content areas. Furthermore, these patterns remain stable over time. Directions for future research include the exploration of global trends in aspects of attained curricula for other subjects, and the use of information on school achievement.     

Exploring the Utility of Background and Cognitive Variables in Explaining Latent Differential Item Functioning: An Example of the PISA 2009 Reading Assessment

Differential item functioning (DIF) may be caused by an interaction of multiple manifest grouping variables or unexplored manifest variables, which cannot be detected by conventional DIF detection methods that are based on a single manifest grouping variable. Such DIF may be detected by a latent approach using the mixture item response theory model and subsequently explained by multiple manifest variables. This study facilitates the interpretation of latent DIF with the use of background and cognitive variables. The PISA 2009 reading assessment and student survey are analyzed. Results show that members in manifest groups were not homogenously advantaged or disadvantaged and that a single manifest grouping variable did not suffice to be a proxy of latent DIF. This study also demonstrates that DIF items arising from the interaction of multiple variables can be effectively screened by the latent DIF analysis approach. Background and cognitive variables jointly well predicted latent class membership.     

Relationship between affect and achievement in science and mathematics in Malaysia and Singapore

Background : The Trends in International Mathematics and Science Study (TIMSS) assesses the quality of the teaching and learning of science and mathematics among Grades 4 and 8 students across participating countries.

Purpose : This study explored the relationship between positive affect towards science and mathematics and achievement in science and mathematics among Malaysian and Singaporean Grade 8 students.

Sample : In total, 4466 Malaysia students and 4599 Singaporean students from Grade 8 who participated in TIMSS 2007 were involved in this study.

Design and method : Students’ achievement scores on eight items in the survey instrument that were reported in TIMSS 2007 were used as the dependent variable in the analysis. Students’ scores on four items in the TIMSS 2007 survey instrument pertaining to students’ affect towards science and mathematics together with students’ gender, language spoken at home and parental education were used as the independent variables.

Results : Positive affect towards science and mathematics indicated statistically significant predictive effects on achievement in the two subjects for both Malaysian and Singaporean Grade 8 students. There were statistically significant predictive effects on mathematics achievement for the students’ gender, language spoken at home and parental education for both Malaysian and Singaporean students, with R ² = 0.18 and 0.21, respectively. However, only parental education showed statistically significant predictive effects on science achievement for both countries. For Singapore, language spoken at home also demonstrated statistically significant predictive effects on science achievement, whereas gender did not. For Malaysia, neither gender nor language spoken at home had statistically significant predictive effects on science achievement.

Conclusions : It is important for educators to consider implementing self-concept enhancement intervention programmes by incorporating ‘affect’ components of academic self-concept in order to develop students’ talents and promote academic excellence in science and mathematics.     

Bayesian Analysis of Multivariate Latent Curve Models With Nonlinear Longitudinal Latent Effects

In longitudinal studies, investigators often measure multiple variables at multiple time points and are interested in investigating individual differences in patterns of change on those variables. Furthermore, in behavioral, social, psychological, and medical research, investigators often deal with latent variables that cannot be observed directly and should be measured by 2 or more manifest variables. Longitudinal latent variables occur when the corresponding manifest variables are measured at multiple time points. Our primary interests are in studying the dynamic change of longitudinal latent variables and exploring the possible interactive effect among the latent variables.

Much of the existing research in longitudinal studies focuses on studying change in a single observed variable at different time points. In this article, we propose a novel latent curve model (LCM) for studying the dynamic change of multivariate manifest and latent variables and their linear and interaction relationships. The proposed LCM has the following useful features: First, it can handle multivariate variables for exploring the dynamic change of their relationships, whereas conventional LCMs usually consider change in a univariate variable. Second, it accommodates both first- and second-order latent variables and their interactions to explore how changes in latent attributes interact to produce a joint effect on the growth of an outcome variable. Third, it accommodates both continuous and ordered categorical data, and missing data.     

Mediation Analysis in a Latent Growth Curve Modeling Framework

This article presents several longitudinal mediation models in the framework of latent growth curve modeling and provides a detailed account of how such models can be constructed. Logical and statistical challenges that might arise when such analyses are conducted are also discussed. Specifically, we discuss how the initial status (intercept) and change (slope) of the putative mediator variable can be appropriately included in the causal chain between the independent and dependent variables in longitudinal mediation models. We further address whether the slope of the dependent variable should be controlled for the dependent variable's intercept to improve the conceptual relevance of the mediation models. The models proposed are illustrated by analyzing a longitudinal data set. We conclude that for certain research questions in developmental science, a multiple mediation model where the dependent variable's slope is controlled for its intercept can be considered an adequate analytical model. However, such models also show several limitations.     

The Utility of Third International Mathematics and Science Study Scales in Predicting Students' State Examination Performance

To examine the predictive utility of three scales provided in the released database of the Third International Mathematics and Science Study (TIMSS) (international plausible values, standardized percent correct score, and national Rasch score), information was obtained on the performance in state examinations in mathematics and science in 1996 (2,969 Grade 8 students) and in 1997 (2,898 Grade 7 students) of students in the Republic of Ireland who had participated in TIMSS in 1995. Performance on TIMSS was related to later performance in the state examinations using normal and nonparametric maximum likelihood (NPML) random effects models. In every case, standardized percent correct scores were found to be the best predictors of later performance, followed by national Rasch scores, and lastly, by international plausible values. The estimates for normal mixing distributions are close to those estimated by the NPML approach, lending support to the validity of estimates.     

Validity of a Residualized Dependent Variable After Pretest Covariance Adjustments: Still the Same Variable?

The authors empirically examined whether the validity of a residualized dependent variable after covariance adjustment is comparable to that of the original variable of interest. When variance of a dependent variable is removed as a result of one or more covariates, the residual variance may not reflect the same meaning. Using the pretest–posttest design as a general framework, the authors compared the nomological validity network for the (a) original dependent variable scores and (b) residualized dependent variable scores after having covaried-out variance explainable by a pretest. Heuristic and empirical examples are provided that demonstrate potential variation in construct validity of residualized dependent variables is a function of correlations among dependent, covariate, and validity variables.     

Trends in gender differences in mathematics and science (TIMSS 1995–2003)

This article investigates changes in gender differences evident in the performance of grade 8th grade students participating in the Trends in International Mathematics and Science Study (TIMSS) between 1995 and 2003. Gender specific results and patterns found in TIMSS 1995 were compared with later cycles of the study in order to address the question of how far the mathematics and science gender gap has narrowed over time. Using a regression approach to compare the trend data, the findings indicated no major changes for mathematics but it appears that the gap in science may be closing, especially in the previously male dominated content areas of chemistry and physics.     

A study of the effect of HyperCard and pen-paper performance assessment methods on expert-novice chemistry problem solving

This study investigated HyperCard as a tool for assessment in science education and determined whether or not a HyperCard assessment instrument could differentiate between expert and novice student performance (balancing stoichiometric equations) in science education. Five chemical equations were presented by traditional pen-paper and by a HyperCard (Hyperequation) program. Thirty honors (expert) and 30 regular (novice) chemistry students were randomly divided into HyperCard and traditional pen-paper groups of 15 students each. Scoring was based on five dependent variables: performance scores, number of attempts, rate of attempts, time on task, and correctness. Correlation results indicated that students with high performance scores correctly balanced more equations, required fewer attempts to balance equations, and required less time per attempt than did students with low performance scores. MANOVA results showed that performance scores and correctness scores for both experts and novice were significantly higher on HyperCard compared to pen-paper assessment; the novice scores on HyperCard nearly equaled the expert pen-paper assessment scores. Significant interactions were found for time on task and for correctness. The results suggest that HyperCard can be a suitable tool for assessment in science education and that such an instrument can differentiate between expert and novice student performance.     

MODELING RELATIONSHIPS AMONG LEARNING,ATTITUDE, SELF-PERCEPTION,AND SCIENCE ACHIEVEMENT FOR GRADE 8 SAUDI STUDENTS

The purpose of the present study was to examine the validity of modeling science achievement in terms of 3 social psychological variables (school connectedness, science attitude, and active learning) and 2 self-perception variables (self-confidence and science value). Two models were tested: full mediation and partial mediation. In the full-mediation model, effects of the 3 social psychological variables upon science achievement were hypothesized to be completely mediated through science value and self-confidence. In the partial-mediation model, however, those 3 variables were hypothesized to affect achievement directly as well as indirectly through the mediating roles of science value and self-confidence. Data were obtained from Grade 8 Saudi students (N = 4,099) who participated in TIMSS 2007. The relationships among constructs were examined with the use of structural equation modeling software Mplus7. Results indicated that both models performed adequately in terms of fit indices, but the partial-mediation model was retained due to its superiority over the full-mediation model in representing the sample covariance matrix as tested through chi-square difference test. The mediating role of self-confidence in the relationships of science attitude and active learning to achievement was substantiated, but the mediating role of science value was not supported.     

What Can Mathematics Educators in England Learn from TIMSS?

In most of the countries taking part in TIMSS, students scored at similar levels for mathematics and science. England was one of the few countries where the results did not conform to this pattern. The key question for mathematics educators in England is: why did students in England perform relatively well in science but relatively badly in mathematics? The results for 9-year-olds were particularly intriguing since the majority of students at this age in England were taught mathematics and science by their class teacher. In order to seek answers to the question posed above, this article compares the responses to the TIMSS context questionnaires made by 9-year-olds and their teachers in the 13 European countries taking part in the TIMSS survey of that age group (Population 1). Issues examined include: curriculum content; lesson time; homework; class size; use of calculators in mathematics; practical activities in science; classroom organisation and students’ attitudes.     

Differences between multiple choice items and constructed response items in the IEA timss surveys

In international large-scale surveys, constructed response (CR) items are increasingly being used and multiple-choice (MC) items are being used less frequently. In this article the two item types will be compared in terms of any differences they have on national mean scores. TIMSS 1995 and TIMSS 1999 data have been used. Are there different effects of the question types for mathematics and science? Does the introduction of open-ended items into the math and science tests affect the math and science achievement results?     

Teacher education variables as correlates of primary science ratings in thirteen TIMSS systems

This paper presents a comparative overview of teacher education variables associated with primary science in 13 TIMSS educational systems. While using TIMSS mean cohort performances at the Year 4 level to rank the sample systems, the study went beyond TIMSS in that it was at the whole-system level and took into account developments since those tests. The study reinforced the view that primary teacher training ideally occurs in a university, and involves a 4-year degree programme that preferably adheres to common standards across institutions. Teachers’ attainment at high school emerged as a principal correlate with TIMSS rankings. Better rankings were also associated with the existence of mandatory science ‘content’ studies as part of teacher training. These observations are consistent with the axiom that teachers’ competence in primary science arises largely from their own mastery of scientific concepts. The authors propose that candidates for primary teacher training programmes should have been awarded passes in science, including physical science, at minimally the middle secondary level, and urge primary teacher training institutions to include compulsory science ‘content’ as well as science pedagogy courses in their programmes.     

Science self-beliefs and science achievement of adolescents in Gulf Cooperation Council countries

This study explored the predictive effects of science self-beliefs on science achievement for 24,680 13-year-old students from Gulf Cooperation Council member countries – Bahrain, Kuwait, Oman, Qatar, Saudi Arabia and the United Arab Emirates – who participated in the Trends in International Mathematics and Science Study (TIMSS) 2007. The performance of adolescent students in Qatar and Saudi Arabia on the TIMSS 2007 science assessment was significantly below the TIMSS scale average. Adolescent students’ science beliefs had both positive and negative predictive effects on science achievement across the Gulf Cooperation Council member countries.     

International assessment: A Rasch model and teachers' evaluation of TIMSS science achievement items

The Trends in International Mathematics and Science Study (TIMSS) is a comparative assessment of the achievement of students in many countries. In the present study, a rigorous independent evaluation was conducted of a representative sample of TIMSS science test items because item quality influences the validity of the scores used to inform educational policy in those countries. The items had been administered internationally to 16,009 students in their eighth year of formal schooling. The evaluation had three components. First, the Rasch model, which emphasizes high quality items, was used to evaluate the items psychometrically. Second, readability and vocabulary analyses were used to evaluate the wording of the items to ensure they were comprehensible to the students. And third, item development guidelines were used by a focus group of science teachers to evaluate the items in light of the TIMSS assessment framework, which specified the format, content, and cognitive domains of the items. The evaluation components indicated that the majority of the items were of high quality, thereby contributing to the validity of TIMSS scores. These items had good psychometric characteristics, readability, vocabulary, and compliance with the assessment framework. Overall, the items tended to be difficult: constructed response items assessing reasoning or application were the most difficult, and multiple choice items assessing knowledge or application were less difficult. The teachers revised some of the sampled items to improve their clarity of content, conciseness of wording, and fit with format specifications. For TIMSS, the findings imply that some of the non‐sampled items may need revision, too. For researchers and teachers, the findings imply that the TIMSS science items and the Rasch model are valuable resources for assessing the achievement of students. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 1321–1344, 2012     

Inference and Interval Estimation Methods for Indirect Effects With Latent Variable Models

Although much is known about the performance of recent methods for inference and interval estimation for indirect or mediated effects with observed variables, little is known about their performance in latent variable models. This article presents an extensive Monte Carlo study of 11 different leading or popular methods adapted to structural equation models with latent variables. Manipulated variables included sample size, number of indicators per latent variable, internal consistency per set of indicators, and 16 different path combinations between latent variables. Results indicate that some popular or previously recommended methods, such as the bias-corrected bootstrap and asymptotic standard errors had poorly calibrated Type I error and coverage rates in some conditions. Likelihood-based confidence intervals, the distribution of the product method, and the percentile bootstrap emerged as leading methods for both interval estimation and inference, whereas joint significance tests and the partial posterior method performed well for inference.     

School Science Achievement in Japan and Singapore: A Tale of Two Cities

This paper compares and contrasts school science achievement between two top scoring nations, Japan and Singapore, on the Third International Mathematics and Science Study (TIMSS) assessments. The first part of the study is devoted to examining cross-national comparisons on selected background questions administered in the TIMSS survey, while the second part examines selected educational attributes and practices that might help explain their consistently high achievement in science. Attention to TIMSS data has chiefly focused on the achievement gap between US and other nations. This report moves beyond US deficit comparisons to examine results and programs of high achieving nations to better inform efforts to close the gaps.     

Structural pattern differences in course enrollment rates among community college students

This study tested a structural equation model of enrollment patterns of white and Hispanic males and females in two-year institutions and the invariance of parameter estimates among the different subgroups in the study. The model represented a multiequation model with three latent endogenous variables, high school academic preparation in mathematics and science, mathematics and science attitudes, and the dependent variable, enrollment patterns in mathematics and science courses. Exogenous variables included parents' education, levels of encouragement by others, and high school grades. Structural equation modeling was used to examine the structural and measurement coefficients of the hypothesized causal model for all subgroups in the study. In summary, an examination of the direct and total effect coefficients revealed different underlying patterns of factors for white and Hispanic females. No convergence on the model was found for white and Hispanic males. Equality constraints on all structural coefficients for both white and Hispanic females were tested and results indicated that all parameter estimates in the structural models for both subgroups were significantly different from each other.     

Motivation to Learn Science: Differences Related to Gender,Class Type,and Ability

Abstract

Motivation differences of gender, science class type (biological vs. physical), and ability level of 242 high school students were investigated. High achievers and physical science students had higher scores than did low achievers and biological science students on academic goals, valuing science, and perceived ability. Boys had higher scores than did girls on perceived ability and stereotyped views of science. For only a subset of variables, these main effects were moderated by class type using achievement-level interaction. The class type main effect was moderated by gender in only one instance. Gender did not interact with achievement level for any variable. Instructional implications are discussed.     

The effect of teacher–student gender matching: Evidence from OECD countries

While some educators argue that teacher–student gender matching improves student performance, there is little empirical evidence to support this hypothesis. This paper assesses the impact of teacher–student gender matching on academic achievement across fifteen OECD countries using data from the Trends in International Mathematics and Science Study (TIMSS). One attractive feature of TIMSS is that it provides information on test scores and teacher characteristics, including gender, for both math and science thereby allowing for student fixed effects estimation. The results provide little support for the conjecture that students benefit from teacher–student gender matching.