Multivariate Meta-Analysis as Structural Equation Models

Multivariate meta-analysis has become increasingly popular in the educational, social, and medical sciences. It is because the outcome measures in a meta-analysis can involve more than one effect size. This article proposes 2 mathematically equivalent models to implement multivariate meta-analysis in structural equation modeling (SEM). Specifically, this article shows how multivariate fixed-, random- and mixed-effects meta-analyses can be formulated as structural equation models. metaSEM (a free R package based on OpenMx) and Mplus are used to implement the proposed procedures. A real data set is used to illustrate the procedures. Formulating multivariate meta-analysis as structural equation models provides many new research opportunities for methodological development in both meta-analysis and SEM. Issues related to and extensions on the SEM-based meta-analysis are discussed.     

On the growth of structural equation modeling in psychological journals

This study investigates the distribution of technical and substantive structural equation modeling articles (SEM) that were published in psychological journals from 1987 to 1994. An inspection of more than 1050 abstracts on PsycLit 1987–1995 (PsycINFO, 1973–1995) revealed a number of clear trends: (a) an increase by year of articles concerned with SEM, (b) an increase in the number of journals that publish structural equation modeling articles, (c) a relatively stable output of technical articles across years, and (d) an increase of substantive articles across years. Furthermore, when the substantive articles are classified as either causal models or confirmatory factor analyses, a similar “growth” trend across years occurs for both categories. We further inspected the growth trend by considering the ratio of SEM articles to the total number of psychology articles and by comparing these results to distributions of analysis of variance, multivariate analysis of variance, regression, and factor analyses articles for the period of 1973 to 1994.     

Structural Equation Models and Mixture Models With Continuous Nonnormal Skewed Distributions

In this article we describe a structural equation modeling (SEM) framework that allows nonnormal skewed distributions for the continuous observed and latent variables. This framework is based on the multivariate restricted skew t distribution. We demonstrate the advantages of skewed SEM over standard SEM modeling and challenge the notion that structural equation models should be based only on sample means and covariances. The skewed continuous distributions are also very useful in finite mixture modeling as they prevent the formation of spurious classes formed purely to compensate for deviations in the distributions from the standard bell curve distribution. This framework is implemented in Mplus Version 7.2.     

Statistical Power of the Multiple Domain Latent Growth Model for Detecting Group Differences

The latent growth model (LGM) in structural equation modeling (SEM) may be extended to allow for the modeling of associations among multiple latent growth trajectories, resulting in a multiple domain latent growth model (MDLGM). While the MDLGM is conceived as a more powerful multivariate analysis technique, the examination of its methodological performance is very limited. Hence, the present study compared the power of the MDLGM with that of a set of univariate LGMs for detecting group differences in growth rates over time using a Monte Carlo study with a two-group and two-domain design. The results indicated that there were different scenarios where the power rates for the MDLGM were greater than that of the set of LGMs (and vice versa) due to a joint function of the two domains’ intercorrelation size and the group difference effect size.     

Implementing Restricted Maximum Likelihood Estimation in Structural Equation Models

Structural equation modeling (SEM) is now a generic modeling framework for many multivariate techniques applied in the social and behavioral sciences. Many statistical models can be considered either as special cases of SEM or as part of the latent variable modeling framework. One popular extension is the use of SEM to conduct linear mixed-effects modeling (LMM) such as cross-sectional multilevel modeling and latent growth modeling. It is well known that LMM can be formulated as structural equation models. However, one main difference between the implementations in SEM and LMM is that maximum likelihood (ML) estimation is usually used in SEM, whereas restricted (or residual) maximum likelihood (REML) estimation is the default method in most LMM packages. This article shows how REML estimation can be implemented in SEM. Two empirical examples on latent growth model and meta-analysis are used to illustrate the procedures implemented in OpenMx. Issues related to implementing REML in SEM are discussed.     

Approach–Method Interaction: The role of teaching method on the effect of context-based approach in physics instruction

The purpose of this study is to explore not only the effect of context-based physics instruction on students’ achievement and motivation in physics, but also how the use of different teaching methods influences it (interaction effect). Therefore, two two-level-independent variables were defined, teaching approach (contextual and non-contextual approaches) and teaching method (traditional and learning cycle methods). Thus, a 2?×?2 factorial design was performed with four treatment groups, including 131 11th-grade students: (1) a traditional method with the non-contextual approach group, (2) a traditional method with the contextual approach group, (3) a learning cycle with the non-contextual approach group, and (4) a learning cycle with the contextual approach group. Prior to and just after the treatments, which took 5 weeks, pre-tests and post-tests on student achievement and motivation were administered. For the analysis of data, multivariate analysis of covariance, simple regressions and follow-up analysis of covariances were performed. Consequently, it was found that the effect of context-based approach on physics achievement is dependent upon the teaching method implemented. That is, the traditional method was observed to increase the effect of the contextual approach while the learning cycle decreased it. Related to the effects on motivation in physics, no evidence was found to claim a significant difference. Based on the findings of this study, further research is suggested for determining which teaching methods are more effective with the context-based approach on students' achievement and motivation in physics.     

The Model-Size Effect on Traditional and Modified Tests of Covariance Structures

The integration of modern methods for causal inference with latent class analysis (LCA) allows social, behavioral, and health researchers to address important questions about the determinants of latent class membership. In this article, 2 propensity score techniques, matching and inverse propensity weighting, are demonstrated for conducting causal inference in LCA. The different causal questions that can be addressed with these techniques are carefully delineated. An empirical analysis based on data from the National Longitudinal Survey of Youth 1979 is presented, where college enrollment is examined as the exposure (i.e., treatment) variable and its causal effect on adult substance use latent class membership is estimated. A step-by-step procedure for conducting causal inference in LCA, including multiple imputation of missing data on the confounders, exposure variable, and multivariate outcome, is included. Sample syntax for carrying out the analysis using SAS and R is given in an appendix.     

A Structural Equation Modeling Approach for the Analysis of Cortisol Data Collected Using Pre–Post–Post Designs

Abstract

This study introduces a novel application of structural equation modeling (SEM) for the analysis of cortisol data that are collected using a pre–post–post design. By way of an extended example, an SEM model is developed that permits an examination of both the overall level of cortisol, as well as changes in cortisol (reactivity and regulation), as predictors of cognitive (executive) and behavioral functioning in 3- to 5-year-old children (N = 171) attending Head Start. The SEM model makes use of the parameterization of latent curve models. Throughout the extended example, the strengths of using an SEM approach for the analysis of cortisol data that are collected using pre–post–post designs is highlighted.     

Covariance and Discriminant Analysis

The formal equiValence between tests of the effect of deleting variables in a multiple response setting based on distances and based on multivariate analysis of covariance (MANCOVA) is shown for the two-group case. Implications of this result for the interpretation of results of multiple-group analyses are discussed, as well as how MANCOV A may be useful for ordering and selecting variables that contribute to group discriminations.     

Part-time faculty status and student evaluation of teaching: An analysis of criminal justice classes

Although part-time (p/t) faculties constitute a growing proportion of college instructors, there is little work on their level of teaching effectiveness relative to full-time (f/t) faculty. Previous work on a key indicator of perceived teaching effectiveness, student evaluation of teaching (SET), and faculty status (p/t/ vs f/t) is marked by a series of shortcomings including lack of a systematic theoretical framework and lack of multivariate statistical analysis techniques to check for possible spuriousness. The present study corrects for these shortcomings. Data consist of SETs from 175 sections of criminal justice classes taught at a Midwestern urban university. Controls are introduced for variables drawn from the literature and include ascribed characteristics of the professor, grade distribution, and structural features of the course (e.g., level, size). The results of a multivariate regression analysis indicate that even after controlling for the other predictors of SETs, p/t faculty receive significantly higher student evaluation scores than f/t faculty. Further, faculty status was the most important predictor of SETs. The results present the first systematic evidence on faculty status and SETs.     

The association between principal leadership and student achievement: A multivariate meta-meta-analysis

A multivariate random-effects meta-meta-analysis was conducted to synthesize the association between principal leadership and student achievement. A total of 12 prior meta-analyses with 18 effect sizes were included in this meta-meta-analysis. The quantitative analysis showed that principal leadership has a statistically significant positive relationship with student achievement (Cohen's d = 0.34). The qualitative analyses revealed that: (a) with the accumulation of knowledge, there appears to have been a trend toward more consistent and precise estimates of principal leadership's effect on student achievement; (b) there was still not enough evidence to argue a specific leadership model or practice is more effective in improving student achievement than others; and (c) the educational contexts seem to moderate the effect of principal leadership. The significance and limitations of the current study, as well as the recommendations for future research, are discussed.     

Maximum Likelihood Dynamic Factor Modeling for Arbitrary N and T Using SEM

This article has 3 objectives that build on each other. First, we demonstrate how to obtain maximum likelihood estimates for dynamic factor models (the direct autoregressive factor score model) with arbitrary T and N by means of structural equation modeling (SEM) and compare the approach to existing methods. Second, we go beyond standard time series analysis (T large and N = 1) and conventional SEM (N large and T = 1 or small) by integrating both approaches. The resulting combined model offers a variety of new modeling options including a direct test of the ergodicity hypothesis, according to which the factorial structure of an individual observed at many time points is identical to the factorial structure of a group of individuals observed at a single point in time. Third, we illustrate the flexibility of SEM time series modeling by extending the approach to account for complex error structures. We end with a discussion of current limitations and future applications of SEM-based time series modeling for arbitrary T and N.     

Determinants of classroom environment in Queensland secondary schools: a multilevel reanalysis

This paper reports the reanalysis of data collected in a study of 3 determinants of classroom environment (viz. year level, subject, and school type) using multivariate analysis of variance and multilevel analysis. Data were collected from 2,211 students in Queensland Catholic and government schools. The Catholic School Classroom Environment Questionnaire, which assesses student affiliation, interactions, cooperation, task orientation, order and organisation, individualisation, and teacher control, was administered to the sample. The original multivariate analyses which used the class as the unit of analysis were supplemented by similar analyses using the student as the unit of analysis and multilevel analyses. While multilevel analyses yielded tests of significance results similar to multivariate analyses conducted with the class as the unit of analysis, effect sizes for the multilevel analyses were similar to those reported for multivariate tests conducted with the student as the unit of analysis.     

Multivariate error assessment of response time histories method for dynamic systems

In this paper, an integrated validation method and process are developed for multivariate dynamic systems. The principal component analysis approach is used to address multivariate correlation and dimensionality reduction, the dynamic time warping and correlation coefficient are used for error assessment, and the subject matter experts (SMEs)’ opinions and principal component analysis coefficients are incorporated to provide the overall rating of the dynamic system. The proposed method and process are successfully demonstrated through a vehicle dynamic system problem.     

Multiple-Group Analysis for Structural Equation Modeling With Dependent Samples

Multigroup structural equation modeling (SEM) plays a key role in studying measurement invariance and in group comparison. However, existing methods for multigroup SEM assume that different samples are independent. This article develops a method for multigroup SEM with correlated samples. Parallel to that for independent samples, the focus here is on the cross-group stability of the within-group structure and parameters. In particular, the method does not require the specification of any between-group relationship. Rescaled and adjusted statistics as well as sandwich-type covariance matrices make the developed method work for possibly nonnormal variables with finite 4th-order moments. The method is applied to a longitudinal data set on the development of entrepreneurial teams across 4 phases. Detailed analysis is provided regarding the stability of the effect of psychological compatibility on team performance, as it is mediated by fairness perception and team cohesion.     

Comparing Indirect Effects in SEM: A Sequential Model Fitting Method Using Covariance-Equivalent Specifications

In social science research, an indirect effect occurs when the influence of an antecedent variable on the effect variable is mediated by an intervening variable. To compare indirect effects within a sample or across different samples, structural equation modeling (SEM) can be used if the computer program supports model fitting with nonlinear constraints. However, such an option is not routinely available in every popular software program. In this study, the basic idea of generating covariance-equivalent models in SEM is given and a sequential model fitting method is proposed as an alternative without the need to use nonlinear constraints. Under this method, the hypothesized model is transformed into a set of successive covariance-equivalent models so that an indirect effect is reparameterized as a single model parameter in the final transformed model. Real examples are given to illustrate how the proposed method is implemented using EQS, a SEM program that currently does not support the analysis with nonlinear constraints.     

Growth goals and growth mindset from a methodological-synergistic perspective: lessons learned from a quantitative correlational research program

This review explores predictors and consequences of students’ growth goals and growth mindset in school with particular emphasis on how correlational statistical methods can be applied to illuminate key issues and implications. Study 1 used cross-sectional data and employed structural equation modelling (SEM) to investigate the role of growth goals in mediating the link between interpersonal relationships and academic engagement. Study 2 conducted multi-group path analysis to investigate the role of growth goals in the academic outcomes of two groups of students (ADHD and non-ADHD). Study 3 used longitudinal data and SEM to test a cross-lagged panel design to investigate reciprocal links between growth goals and growth mindset. Study 4 conducted multi-level SEM where the effects of a growth orientation on engagement and achievement were investigated at the student-level (level 1) and the classroom-level (level 2). Taking these four studies together, we aim to show how correlational data and multivariate correlational analyses have been effective in answering research questions in a way that have practical and theoretical implications for students’ academic growth. We also position this review as a substantive-methodological synergy – an approach recently recommended in response to concerns about the increasing polarization of substantive and methodological research and researchers.     

Propensity Score Analysis of Complex Survey Data with Structural Equation Modeling: A Tutorial with Mplus

Propensity score (PS) analysis aims to reduce bias in treatment effect estimates obtained from observational studies, which may occur due to non-random differences between treated and untreated groups with respect to covariates related to the outcome. We demonstrate how to use structural equation modeling (SEM) for PS analysis to remove selection bias due to latent covariates and estimate treatment effects on latent outcomes. Following the discussion of the design and analysis stages of PS analysis with SEM, an example is presented which uses the Mplus software to analyze data from the 1999 School and Staffing Survey (SASS) and 2000 Teacher Follow-up Survey (TFS) to estimate the effects teacher’s participation in a network of teachers on the teacher’s perception of workload manageability.     

Comparing Exploratory Structural Equation Modeling and Existing Approaches for Multiple Regression with Latent Variables

Exploratory structural equation modeling (ESEM) is an approach for analysis of latent variables using exploratory factor analysis to evaluate the measurement model. This study compared ESEM with two dominant approaches for multiple regression with latent variables, structural equation modeling (SEM) and manifest regression analysis (MRA). Main findings included: (1) ESEM in general provided the least biased estimation of the regression coefficients; SEM was more biased than MRA given large cross-factor loadings. (2) MRA produced the most precise estimation, followed by ESEM and then SEM. (3) SEM was the least powerful in the significance tests; statistical power was lower for ESEM than MRA with relatively small target-factor loadings, but higher for ESEM than MRA with relatively large target-factor loadings. (4) ESEM showed difficulties in convergence and occasionally created an inflated type I error rate under some conditions. ESEM is recommended when non-ignorable cross-factor loadings exist.