Investigating the Sensitivity of Goodness-of-Fit Indices to Detect Measurement Invariance in a Bifactor Model

A Monte Carlo simulation study was conducted to evaluate the sensitivities of the likelihood ratio test and five commonly used delta goodness-of-fit (ΔGOF) indices (i.e., ΔGamma, ΔMcDonald’s, ΔCFI, ΔRMSEA, and ΔSRMR) to detect a lack of metric invariance in a bifactor model. Experimental conditions included factor loading differences, location and number of noninvariant items, and sample size. The results indicated all ΔGOF indices held Type I error to a minimum and overall had adequate power for the study. For detecting the violation of metric invariance, only ΔGamma and ΔCFI, in addition to Δχ², are recommended to use in the bifactor model with values of ?.016 to ?.023 and ?.003 to ?.004, respectively. Moreover, in the variance component analysis, the magnitude of the factor loading differences contributed the most variation to all ΔGOF indices, whereas sample size affected Δχ² the most.     

Psychometric properties of the Brunel Mood Scale in Chinese adolescents and adults

Abstract

Building on the work of Terry and colleagues (Terry, P. C., Lane, A. M., Lane, H. J., & Keohane, L. (1999). Development and validation of a mood measure for adolescents. Journal of Sports Sciences, 17, 861–872; Terry, P. C., Lane, A. M., & Fogarty, G. J. (2003). Construct validity of the Profile of Mood States-Adolescents for use with adults. Psychology of Sport & Exercise, 4, 125–139.), the present study examined the validity and internal consistency reliability of the Chinese version of the Brunel Mood Scale (BRUMS-C) among 2,548 participants, comprising adolescent athletes (n = 520), adult athletes (n = 434), adolescent students (n = 673), and adult students (n = 921). Both adolescent and adult athletes completed the BRUMS-C before, during, or after regular training and both adolescent and adult students completed the BRUMS-C in a classroom setting. Confirmatory factor analyses (CFAs) provided support for the factorial validity of a 23-item six-factor model, with one item removed from the hypothesised measurement model. Internal consistency reliabilities were satisfactory for all subscales across each of the four samples. Criterion validity was supported with strong relationships between the BRUMS-C, abbreviated POMS, and Chinese Affect Scale consistent with theoretical predictions. Multi-sample CFAs showed the BRUMS-C to be invariant at the configural, metric, strong, and structural levels for all samples. Furthermore, latent mean difference analyses showed that athletes reported significantly higher levels of fatigue than students while maintaining almost the same levels of vigour, and adolescent students reported significantly higher levels of depressed mood than the other three samples.     

Sensitivity of the Soccer Defending Skill Scale: A comparison between teams

Abstract

The aim of the present study was to examine the sensitivity of the Soccer Defending Skill Scale (SDSS) using multiple-group analysis and a structured mean model. The data were 469 defending performances in the finals of the FIFA World Cup Korea/Japan 2002^TM, which were measured as distances, angles between attackers and defenders, and the number of players using 6-point interval scales. The results of multiple-group analysis indicated that the model, which had factor loading, intercept and factor correlation, and F3/F4/F6 variances invariant across Brazil and Germany, represented the best fit to the data (e.g. Tucker-Lewis index = 1.001, confirmatory fit index = 1.000, root mean square error of approximation = 0.000). Furthermore, the latent mean model comparison between teams revealed significant differences in latent mean with respect to “defence to attackers without the ball” (Brazil < Germany). We conclude that our results provide strong support for the sensitivity of the SDSS.     

Improving the Measurement of School Climate Using Item Response Theory

The U.S. government has become increasingly focused on school climate, as recently evidenced by its inclusion as an accountability indicator in the Every Student Succeeds Act. Yet, there remains considerable variability in both conceptualizing and measuring school climate. To better inform the research and practice related to school climate and its measurement, we leveraged item response theory (IRT), a commonly used psychometric approach for the design of achievement assessments, to create a parsimonious measure of school climate that operates across varying individual characteristics. Students (n = 69,513) in 111 secondary schools completed a school climate assessment focused on three domains of climate (i.e., safety, engagement, and environment), as defined by the U.S. Department of Education. Item and test characteristics were estimated using the mirt package in R using unidimensional IRT. Analyses revealed measurement difficulties that resulted in a greater ability to assess less favorable perspectives on school climate. Differential item functioning analyses indicated measurement differences based on student academic success. These findings support the development of a broad measure of school climate but also highlight the importance of work to ensure precision in measuring school climate, particularly when considering use as an accountability measure.     

Evaluation of Structural Relationships in Autoregressive Cross-Lagged Models Under Longitudinal Approximate Invariance:A Bayesian Analysis

To infer longitudinal relationships among latent factors, traditional analyses assume that the measurement model is invariant across measurement occasions. Alternative to placing cross-occasion equality constraints on parameters, approximate measurement invariance (MI) can be analyzed by specifying informative priors on parameter differences between occasions. This study evaluated the estimation of structural coefficients in multiple-indicator autoregressive cross-lagged models under various conditions of approximate MI using Bayesian structural equation modeling. Design factors included factor structures, conditions of non-invariance, sizes of structural coefficients, and sample sizes. Models were analyzed using two sets of small-variance priors on select model parameters. Results showed that autoregressive coefficient estimates were more accurate for the mixed pattern than the decreasing pattern of non-invariance. When a model included cross-loadings, an interaction was found between the cross-lagged estimates and the non-invariance conditions. Implications of findings and future research directions are discussed.     

Reexamining the factor structure of the test of gross motor development – second edition: Application of exploratory structural equation modeling

The Test of Gross Motor Development – Second Edition (TGMD-2) is a widely used evaluation tool of children’s fundamental motor skills (FMS). This study illustrates how exploratory structural equation modeling (ESEM) addresses current limitations associated with TGMD-2 factor structure. Using the normative dataset from the TGMD-2 manual, we test alternative measurement models using ESEM and confirmatory factor analysis (CFA) and assess measurement invariance between boys and girls and across age. Findings highlight complexity and poor discriminant validity in the two-factor solution that is accentuated when relying on CFA independent clusters assumption. Results also demonstrate differential item functioning (DIF) across age for three of the 12 TGMD-2 indicators. Taken together, this study provides valuable information on the need to reevaluate the TGMD-2 factor structure and can guide revisions for the third edition of the TGMD.     

Log-periodic quantum magneto-oscillations and discrete-scale invariance in topological material HfTe5

Discrete-scale invariance (DSI) is a phenomenon featuring intriguing log-periodicity that can be rarely observed in quantum systems. Here, we report the log-periodic quantum oscillations in the longitudinal magnetoresistivity (ρ_xx) and the Hall traces (ρ_yx) of HfTe₅ crystals, which reveal the DSI in the transport-coefficients matrix. The oscillations in ρ_xx and ρ_yx show the consistent logB-periodicity with a phase shift. The finding of the logB oscillations in the Hall resistance supports the physical mechanism as a general quantum effect originating from the resonant scattering. Combined with theoretical simulations, we further clarify the origin of the log-periodic oscillations and the DSI in the topological materials. This work evidences the universality of the DSI in the Dirac materials and provides indispensable information for a full understanding of this novel phenomenon.     

Examining factorial validity and measurement invariance of the Student-Teacher Relationship Scale

The purposes of this study were to (a) test the hypothesized factor structure of the Student-Teacher Relationship Scale (STRS; Pianta, 2001) for 308 African American (AA) and European American (EA) children using confirmatory factor analysis (CFA) and (b) examine the measurement invariance of the factor structure across AA and EA children. CFA of the hypothesized three-factor model with correlated latent factors did not yield an optimal model fit. Parameter estimates obtained from CFA identified items with low factor loadings and R² values, suggesting that content revision is required for those items on the STRS. Deletion of two items from the scale yielded a good model fit, suggesting that the remaining 26 items reliably and validly measure the constructs for the whole sample. Tests for configural invariance, however, revealed that the underlying constructs may differ for AA and EA groups. Subsequent exploratory factor analyses (EFAs) for AA and EA children were carried out to investigate the comparability of the measurement model of the STRS across the groups. The results of EFAs provided evidence suggesting differential factor models of the STRS across AA and EA groups. This study provides implications for construct validity research and substantive research using the STRS given that the STRS is extensively used in intervention and research in early childhood education.     

Five Methods to Score the Teacher Observation of Classroom Adaptation Checklist and to Examine Group Differences

This study compared 5 scoring methods in terms of their statistical assumptions. They were then used to score the Teacher Observation of Classroom Adaptation Checklist, a measure consisting of 3 subscales and 21 Likert-type items. The 5 methods used were (a) sum/average scores of items, (b) latent factor scores with continuous indicators, (c) latent factor scores with ordered categorical indicators using the mean- and variance-adjusted weighted least squares estimation method, (d) latent factor scores with ordered categorical indicators using the full information maximum likelihood estimation method, and (e) multidimensional graded response model using the Bock-Aitkin expectation-maximization estimation procedure. Measurement invariance between gender groups and between free/reduced-price lunch status groups was evaluated with the second, third, fourth, and fifth methods. Group mean differences based on the 5 methods were calculated and compared.     

Measurement invariance of the Servant Leadership Questionnaire across K-12 principal gender

Measurement invariance of the five-factor Servant Leadership Questionnaire between female and male K-12 principals was tested using multi-group confirmatory factor analysis. A sample of 956 principals (56.9% were females and 43.1% were males) was analysed in this study. The hierarchical multi-step measurement invariance test supported the measurement invariance of the five-factor model across gender. Latent factor means were compared between females and males when measurement invariance was established. Results showed that females were significantly higher than males on emotional healing, wisdom, persuasive mapping and organisational stewardship, and they were not statistically different on altruistic calling.