Reliability and Validity Evidence for Achievement Goal Models in High School Physical Education Settings

Although empirical research in academic areas provides support for both a 3-factor as well as a 4-factor achievement goal model, both models were proposed and tested with a collegiate sample. Little is known about the generalizability of either model with high school level samples. This study was designed to examine whether the 3-factor model (Mastery Goals, Performance-Approach Goals, and Performance-Avoidance Goals) or the 4-factor model (Mastery-Approach Goals, Mastery-Avoidance Goals, Performance-Approach Goals, and Performance-Avoidance Goals) is appropriate in high school physical education settings. The factorial validity of the models and internal consistency reliability were tested with confirmatory factor analysis, invariance testing, and tests of internal consistency across 2 samples. The results reveal that the items from the 4-factor achievement goal model can produce internally consistent and valid scores for high school students in physical education settings; the 4-factor model provides a better fit to the data than the 3-factor model. The multistep invariance analysis, however, reveals only metric invariance across 2 school samples.     

Testing Factorial Invariance of the Amotivation Inventory-Physical Education Across Gender for Urban Adolescents

Lack of motivation, or amotiation, is emerging as a critical issue in high school physical education. The Amotivation Inventory-Physical Education (Shen, Winger, Li, Sun, & Rukavina, 2010) was developed to measure the multidimensional nature of amotivation construct in physical education. This study was designed to examine the consistency of the metric properties of Amotivation Inventory-Physical Education scores by evaluating their factorial structure for invariance across gender in a sample of 985 urban inner-city high school adolescents. Latent mean invariance was also tested. Results revealed that constraining factor loadings and intercepts in sequential configural, metric, and scalar invariances had no meaningful impact on model fit. However, gender might play a role in the magnitude of factor scores. Findings suggest that the four-factor structure of the Amotivation Inventory-Physical Education is a satisfactory representation of motivational deficits for urban inner-city adolescents and provide significant validity evidence for the scale scores in urban high school settings.     

Factorial equivalence of social cognitive theory: educational levels × time differences

The study of social cognitive theory has involved a number of inquiries, notably one of which concerns the formation and development of self-efficacy beliefs. Social cognitive theory indicates that we form our self-efficacy beliefs from four major sources of information: enactive performance accomplishments, vicarious experiences, verbal persuasion and emotional and physiological states. We advance this tenet by exploring across four occasions, and the four sources of information in the context of elementary school learning. Two cohorts of participants, at level 2 (N?=?352 3rd–4th grade students) and level 3 (N?=?264 5th–6th grade students), answered a Likert’s-scale inventory that we developed and used in a previous study. We proposed and tested a number of a priori models using LISREL 8.80. Furthermore, factorial invariance analyses of the inventory were performed, a posteriori, to determine the stability/instability of the four sources of information.     

Assessing construct of DIBELS: Measurement invariance of DIBELS subscales,from kindergarten to the first grade

A challenge using the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) in studying reading growth is that reading skills children exhibit change by age. In order to study growth using changing subscales, it is necessary to examine measurement invariance and measurement structure underlying the different subscales. The purpose of this paper is to examine the measurement structure of the DIBELS subscales, particular measurement invariance. The results indicate that the DIBELS subscales do not seem to have metric invariance but they do share a common factor over time, suggesting that the same construct of reading skills were measured but they manifested in the different fashion over time.     

The Impact of Varying the Number of Measurement Invariance Constraints on the Assessment of Between-Group Differences of Latent Means

The objective was to offer guidelines for applied researchers on how to weigh the consequences of errors made in evaluating measurement invariance (MI) on the assessment of factor mean differences. We conducted a simulation study to supplement the MI literature by focusing on choosing among analysis models with different number of between-group constraints imposed on loadings and intercepts of indicators. Data were generated with varying proportions, patterns, and magnitudes of differences in loadings and intercepts as well as factor mean differences and sample size. Based on the findings, we concluded that researchers who conduct MI analyses should recognize that relaxing as well as imposing constraints can affect Type I error rate, power, and bias of estimates in factor mean differences. In addition, fit indexes can be misleading in making decisions about constraints of loadings and intercepts. We offer suggestions for making MI decisions under uncertainty when assessing factor mean differences.     

Comparing BSEM and EUPD Estimates for Two-Group SB-MTMM Experiments

In 2018, two different approaches have been suggested to solve the estimation problems that have been detected during the analysis of the data of the two-group split ballot multi-trait multi-method (SB-MTMM experiments performed in many countries between 2002 and 2010 in the European Social Survey). One group suggested using the Bayesian estimation. The other group suggested a new estimation procedure (EUPD) that makes use of the pooled data across the different countries. In this note, we present a comparison of the results of the two approaches on the same generated data, which are comparable with the data that created problems in the ESS.     

Book reviews

Statistical Models for Ordinal Variables. C. C. Clogg and E. S. Shihadeh. Thousand Oaks, CA: Sage, 1994, 192 pages.

Graphical Multivariate Analysis with AMOS, EQS and LISREL: A Visual Approach to Covariance Structure Analysis (in Japanese). Yutaka Kano. Kyoto, Japan: Gendai‐Sugakusha, 1997,235 pages.     

Investigating Factorial Invariance of Latent Variables Across Populations When Manifest Variables Are Missing Completely

Difficulties arise in multiple-group evaluations of factorial invariance if particular manifest variables are missing completely in certain groups. Ad hoc analytic alternatives can be used in such situations (e.g., deleting manifest variables), but some common approaches, such as multiple imputation, are not viable. At least 3 solutions to this problem are viable: analyzing differing sets of variables across groups, using pattern mixture approaches, and a new method using random number generation. The latter solution, proposed in this article, is to generate pseudo-random normal deviates for all observations for manifest variables that are missing completely in a given sample and then to specify multiple-group models in a way that respects the random nature of these values. An empirical example is presented in detail comparing the 3 approaches. The proposed solution can enable quantitative comparisons at the latent variable level between groups using programs that require the same number of manifest variables in each group.     

Testing Factorial Invariance in Multilevel Data: A Monte Carlo Study

Testing factorial invariance has recently gained more attention in different social science disciplines. Nevertheless, when examining factorial invariance, it is generally assumed that the observations are independent of each other, which might not be always true. In this study, we examined the impact of testing factorial invariance in multilevel data, especially when the dependency issue is not taken into account. We considered a set of design factors, including number of clusters, cluster size, and intraclass correlation (ICC) at different levels. The simulation results showed that the test of factorial invariance became more liberal (or had inflated Type I error rate) in terms of rejecting the null hypothesis of invariance held between groups when the dependency was not considered in the analysis. Additionally, the magnitude of the inflation in the Type I error rate was a function of both ICC and cluster size. Implications of the findings and limitations are discussed.