Issues in applied structural equation modeling research

When using the popular structural equation modeling (SEM) methodology, the issues of sample size, method of parameter estimation, assessment of model fit, and capitalization on chance are of great importance in the process of evaluating the results of an empirical study. We focus first on implications of the large‐sample theory underlying applications of the methodology. The utility for applied contexts of the asymptotically distribution‐free parameter estimation and model testing method is discussed next. We then argue for wider use of a recently developed, non conventional model‐fit assessment strategy in SEM. We conclude by discussing the issue of capitalization on chance, primarily in situations in which exploratory and confirmatory analyses are conducted on the same data set.     

Managing knowledge performance: testing the components of a knowledge management system on organizational performance

The main purpose of the current study is to validate the framework of knowledge management (KM) capabilities created by Gold (Towards a theory of organizational knowledge management capabilities. Doctoral dissertation, University of North Carolina, Chapel Hill) 2001) in a study of South Korean companies. However, the original framework did not provide a thorough explanation of the effect of incentives, which motivate and encourage the knowledge management process. In this study, the modified framework that includes incentives in the knowledge infrastructure capability was tested. Moreover, since there is a weak linkage between KM and organizational performance, this study used empirical evidence to identify the relationship between KM capabilities (KMC) and four perspectives of organizational performance. Since structural equation modeling (SEM) is mostly used to describe causal relationships among unobserved (latent) and observed variables, this study used SEM procedures to determine whether there were any structural relationships between knowledge management capabilities and four perspectives of organizational performance. Moreover, the SEM procedure is “a statistical test to find whether a model fits a set of data, whether it matches a theoretical expectation” (Vogt, Dictionary of statistics & methodology. Sage Publications Inc., Thousand Oaks, CA, p 135, 2005). Therefore, this study also used SEM procedures to test a hypothesized model that had a good fit indicates that the model adequately describes the sample data. This study assumed that knowledge management capabilities could be divided into two types: knowledge infrastructure and process capabilities. The original hypothesized model showed that there was a positive relationship between knowledge management capabilities and organizational performance, but the overall model fit was insufficient to be accepted, because knowledge infrastructure and process capabilities were highly correlated. This study proposed two alternative models to find the best fit and found that knowledge infrastructure and process capabilities should be combined under the higher-order latent variable as subordinate latent variables. Lastly, there was a positive relationship between KMC and organizational performance. This study might not be free from common method bias to some degrees. It would be better to divide participants into two groups to respond to either the knowledge management capabilities survey or the organizational performance survey and to investigate the correlation between them. There are two main contributions for the field of knowledge management. First, this study attempted to integrate the fragmented literature of knowledge management into a holistic view and develop a framework for knowledge management. Moreover, this study found that there is a strong and positive relationship between KM infrastructure and process, which could refer that, to improve organizational performance, an organization should support KM processes, as well as build decent KM infrastructure. The results of this study would help KM practitioners to advocate the importance of KM to top managements.     

A Structural Equation Modeling Evaluation of the General Model of Instructional Communication

The General Model of Instructional Communication introduced by McCroskey, Valencic, and Richmond (2004) is supported in its original conception by canonical data. This study, however, uses structural equation modeling (SEM) to provide a more detailed analysis. Although the model as originally hypothesized fits the data poorly, analysis of the SEM results suggests adjustments to the original model that substantially improve the model's fit. The revised model accounts for significant portions of the variance in the outcome variables, provides a more detailed explanation of the relationships involved, and has implications for future research. Bootstrapped parameter estimates suggest that the results are replicable.     

A Note on Structural Equation Modeling Estimates of Reliability

Reliability can be estimated using structural equation modeling (SEM). Two potential problems with this approach are that estimates may be unstable with small sample sizes and biased with misspecified models. A Monte Carlo study was conducted to investigate the quality of SEM estimates of reliability by themselves and relative to coefficient alpha. The SEM approach showed minimal bias when the model was correctly specified if items were relatively well defined by their underlying factor(s). They tended to demonstrate somewhat greater bias when the model was misspecified, particularly underspecified. Overall, SEM estimates were more stable than anticipated. Researchers are more likely to obtain accurate estimates of reliability using SEM by conducting large-sample studies with well-constructed scales and critically assessing model fit.     

A Comparison of Method Effects in Two Confirmatory Factor Models for Structurally Different Methods

It is well known that measurement error in observable variables induces bias in estimates in standard regression analysis and that structural equation models are a typical solution to this problem. Often, multiple indicator equations are subsumed as part of the structural equation model, allowing for consistent estimation of the relevant regression parameters. In many instances, however, embedding the measurement model into structural equation models is not possible because the model would not be identified. To correct for measurement error one has no other recourse than to provide the exact values of the variances of the measurement error terms of the model, although in practice such variances cannot be ascertained exactly, but only estimated from an independent study. The usual approach so far has been to treat the estimated values of error variances as if they were known exact population values in the subsequent structural equation modeling (SEM) analysis. In this article we show that fixing measurement error variance estimates as if they were true values can make the reported standard errors of the structural parameters of the model smaller than they should be. Inferences about the parameters of interest will be incorrect if the estimated nature of the variances is not taken into account. For general SEM, we derive an explicit expression that provides the terms to be added to the standard errors provided by the standard SEM software that treats the estimated variances as exact population values. Interestingly, we find there is a differential impact of the corrections to be added to the standard errors depending on which parameter of the model is estimated. The theoretical results are illustrated with simulations and also with empirical data on a typical SEM model.     

Fit for a Bayesian: An Evaluation of PPP and DIC for Structural Equation Modeling

Despite its importance to structural equation modeling, model evaluation remains underdeveloped in the Bayesian SEM framework. Posterior predictive p-values (PPP) and deviance information criteria (DIC) are now available in popular software for Bayesian model evaluation, but they remain underutilized. This is largely due to the lack of recommendations for their use. To address this problem, PPP and DIC were evaluated in a series of Monte Carlo simulation studies. The results show that both PPP and DIC are influenced by severity of model misspecification, sample size, model size, and choice of prior. The cutoffs PPP < 0.10 and ?DIC > 7 work best in the conditions and models tested here to maintain low false detection rates and misspecified model selection rates, respectively. The recommendations provided in this study will help researchers evaluate their models in a Bayesian SEM analysis and set the stage for future development and evaluation of Bayesian SEM fit indices.     

Combining IRT and SEM: A Hybrid Model for Fitting Responses and Response Certainties

This article proposes a model-based procedure, intended for personality measures, for exploiting the auxiliary information provided by the certainty with which individuals answer every item (response certainty). This information is used to (a) obtain more accurate estimates of individual trait levels, and (b) provide a more detailed assessment of the consistency with which the individual responds to the test. The basis model consists of 2 submodels: an item response theory submodel for the responses, and a linear-in-the-coefficients submodel that describes the response certainties. The latter is based on the distance-difficulty hypothesis, and is parameterized as a factor-analytic model. Procedures for (a) estimating the structural parameters, (b) assessing model–data fit, (c) estimating the individual parameters, and (d) assessing individual fit are discussed. The proposal was used in an empirical study. Model–data fit was acceptable and estimates were meaningful. Furthermore, the precision of the individual trait estimates and the assessment of the individual consistency improved noticeably.     

Bayesian Meta-Analytic SEM: A One-Stage Approach to Modeling Between-Studies Heterogeneity in Structural Parameters

Meta-analytic structural equation modeling (MASEM) refers to a set of meta-analysis techniques for combining and comparing structural equation modeling (SEM) results from multiple studies. Existing approaches to MASEM cannot appropriately model between-studies heterogeneity in structural parameters because of missing correlations, lack model fit assessment, and suffer from several theoretical limitations. In this study, we address the major shortcomings of existing approaches by proposing a novel Bayesian multilevel SEM approach. Simulation results showed that the proposed approach performed satisfactorily in terms of parameter estimation and model fit evaluation when the number of studies and the within-study sample size were sufficiently large and when correlations were missing completely at random. An empirical example about the structure of personality based on a subset of data was provided. Results favored the third factor structure over the hierarchical structure. We end the article with discussions and future directions.     

Effects of sample size,estimation methods,and model specification on structural equation modeling fit indexes

A Monte Carlo simulation study was conducted to investigate the effects on structural equation modeling (SEM) fit indexes of sample size, estimation method, and model specification. Based on a balanced experimental design, samples were generated from a prespecified population covariance matrix and fitted to structural equation models with different degrees of model misspecification. Ten SEM fit indexes were studied. Two primary conclusions were suggested: (a) some fit indexes appear to be noncomparable in terms of the information they provide about model fit for misspecified models and (b) estimation method strongly influenced almost all the fit indexes examined, especially for misspecified models. These 2 issues do not seem to have drawn enough attention from SEM practitioners. Future research should study not only different models vis‐à‐vis model complexity, but a wider range of model specification conditions, including correctly specified models and models specified incorrectly to varying degrees.     

A comprehensive reanalysis of the metacognitive self-regulation scale from the MSLQ

The metacognitive self-regulation (MSR) scale is among the most widely used measures of metacognition in educational research. However, the psychometric properties and validity of the scale have not been well established. A series of analyses on a college sample were performed to address this issue. In Study 1, a split-sample exploratory (EFA) and confirmatory factor analysis (CFA) was performed to test the one-factor specification of the MSR scale. Time and study environment (TSE), total study time, and cumulative grade performance average (cGPA) were introduced as outcome variables in a structural equation model (SEM) to examine the factors suggested by the EFA. The results of Study 1 indicated poor one-factor model fit and suggested two and three-factor models provided improved fits of the sample data. Results from the SEM indicated the novel factors from the two and three-factor models had different relationships with the outcome variables than the originally specified one-factor model. In Study 2, a modified one-factor model was introduced that consisted of nine items and was named metacognitive self-regulation revised (MSR-R). Five additional samples were included to replicate the model fit for the revised model specification. Finally, a path analysis was performed to examine the relationship of the MSR-R to variables from Study 1. The results of Study 2 revealed improved psychometric properties and reliability for the MSR-R. An indirect relationship emerged between MSR-R and cGPA through TSE. In conclusion, convincing evidence for replacing the MSR was found and implications of the revised scale for future studies was discussed.     

Bayesian Data-Model Fit Assessment for Structural Equation Modeling

Bayesian approaches to modeling are receiving an increasing amount of attention in the areas of model construction and estimation in factor analysis, structural equation modeling (SEM), and related latent variable models. However, model diagnostics and model criticism remain relatively understudied aspects of Bayesian SEM. This article describes and illustrates key features of Bayesian approaches to model diagnostics and assessing data–model fit of structural equation models, discussing their merits relative to traditional procedures.     

Structural equation modeling: Back to basics

Major technological advances incorporated into structural equation modeling (SEM) computer programs now make it possible for practitioners who are basically unfamiliar with the purposes and limitations of SEM to use this tool within their research contexts. The current move by program developers to market more user friendly software packages is a welcomed trend in the social and behavioral science research community. The quest to simplify the data analysis step in the research process has—at least with regard to SEM—created a situation that allows practitioners to apply SEM but forgetting, knowingly ignoring, or most dangerously, being ignorant of some basic philosophical and statistical issues that must be addressed before sound SEM analyses should be conducted. This article focuses on some of the almost forgotten topics taken here from each step in the SEM process: model conceptualization, identification and parameter estimation, and data‐model fit assessment and model modification. The main objective is to raise awareness among researchers new to SEM of a few basic but key philosophical and statistical issues. These should be addressed before launching into any one of the new generation of SEM software packages and being led astray by the seemingly irresistible temptation to prematurely start “playing” with the data.     

The Overlap of Child and Adult Writing Vocabularies

Many mechanistic rules of thumb for evaluating the goodness of fit of structural equation models (SEM) emphasize model parsimony; all other things being equal, a simpler, more parsimonious model with fewer estimated parameters is better than a more complex model Although this is usually good advice, in the present article a heuristic counterexample is demonstrated in which parsimony as typically operationalized in indices of fit may be undesirable. Specifically, in simplex models of longitudinal data, the failure to include correlated uniquenesses relating the same indicators administered on different occasions will typically lead to systematically inflated estimates of stability. Although simplex models with correlated uniquenesses are substantially less parsimonious and may be unacceptable according to mechanistic decision rules that penalize model complexity, it can be argued a priori that these additional parameter estimates should be included. Simulated data . are used to support this claim and to evaluate the behavior of a variety of fit indices and decision rules. The results demonstrate the validity of Bollen and Long’s (1993) conclusion that “test statistics and fit indices are very beneficial, but they are no replacement for sound judgment and substantive expertise” (p. 8).     

An SAS Macro for Implementing the Modified Bollen-Stine Bootstrap for Missing Data: Implementing the Bootstrap Using Existing Structural Equation Modeling Software

The Bollen-Stine bootstrap can be used to correct for standard error and fit statistic bias that occurs in structural equation modeling (SEM) applications due to nonnormal data. The purpose of this article is to demonstrate the use of a custom SAS macro program that can be used to implement the Bollen-Stine bootstrap with existing SEM software. Although this article focuses on missing data, the macro can be used with complete data sets as well. A series of heuristic analyses are presented, along with detailed programming instructions for each of the commercial SEM software packages.     

Exploration of the Developmental Components Contributing to Elementary School Children's Reading Comprehension

The purpose of this study was to compare rauding theory (Carver, 1995) to our own evolving model of reading acquisition, which supports stage and phase theories of reading development. The relations among rauding variables--cognitive power, auditory-accuracy level, word-recognition level, comprehension-accuracy level, reading-rate level, and reading-comprehension-rate level-were examined using structural equation modeling. The Kaufman Assessment Battery for Children and the Kaufman Test of Educational Achievement subtests were used to operationalize the various constructs. Carver's (1993) model was assessed at each grade in two parts, with overlapping paths allowing cross-validation of some coefficients. For Grades 1 and 2, the fit indices were above. 95, indicating a good model fit. One additional path was supported-from word recognition to reading comprehension rate. The fit indices for Grades 3 and 4 were above. 95, supporting the modified Carver model. Analyses for Grades 5 and 6 produced fit indices above. 90, indicating drops in the level of association among the variables compared to earlier grades. The results of this study offer support to Carver's (1993) rauding theory and further advance the theories that children go through stages or phases of reading development. Although word recognition is a notable component of reading development throughout the elementary grades, its contribution to comprehension begins to lessen at Grades 5 and 6. There is, however, an increase with grade of the influence of cognitive power on reading comprehension, which is greater for crystallized (Peabody Picture Vocabulary Test-Revised) intelligence than for global (fluid and crystallized) or fluid intelligence.     

Teaching problem solving as viewed through a theory of models

Summary This paper presents a small amount of theory including an analysis of the task of teaching problem solving and the act of problem solving derived from a theory of models of operations. It also presents the results of an exploratory investigation which tends to add credence to elements of the analysis.Two of the general problem solving strategies which are suggested are (1) classification of word problems as instances of models previously learned (taught) and (2) a variety of techniques involving mapping individually word problems onto instances of familiar models. A third problem solving strategy which has not been mentioned is indicated by the theory: In this strategy, from the given problematical situation (word problem), a model is generalized which subsumes the given situation, and search for an isomorphism between the model and various operations (i.e., multiplication) is conducted. If an isomorphism is observed, then the selected operation and attending computational algorithms can be used to solve the problem. Of course, it is unlikely that this type of activity would be appropriate for primary grade children, but perhaps it could be taught at a later stage of the curriculum.The extensive mathematics education literature devoted to the study of teaching problem solving in the elementary school includes numerous theoretical and methodological approaches to investigating problem solving. This paper, the author believes, presents a unique, systematic, and reasonable perspective on the matter. Both a systematic view of the act of solving one-step word problems and a two stage general level analysis of practices found in teaching problem solving have been presented as they were derived from a theory of models.     

Binomial index of model fit: An elaboration

This article is an elaboration on the use of the binomial test of model fit value, which in this article will be referred to as the binomial index of model fit value, to gauge the degree that the data fit a path analytic or structural equation model. In addition, this article responds to the criticisms and comments made by Hsu (this issue), Drezner and Drezner (this issue), and Raykov and Penev (this issue) regarding the use of this approach to measuring the degree of model fit. We appreciate the comments provided by these authors. Their comments have assisted us in clarifying our reason for developing the binomial index of model fit procedure as well as our perception of its use.     

Modeling Nonlinear Structural Equation Models: A Comparison of the Two-Stage Generalized Additive Models and the Finite Mixture Structural Equation Model

Researchers have devoted some time and effort to developing methods for fitting nonlinear relationships among latent variables. In particular, most of these have focused on correctly modeling interactions between 2 exogenous latent variables, and quadratic relationships between exogenous and endogenous variables. All of these approaches require prespecification of the nonlinearity by the researcher, and are limited to fairly simple nonlinear relationships. Other work has been done using mixture structural equation models (SEMM) in an attempt to fit more complex nonlinear relationships. This study expands on this earlier work by introducing the 2-stage generalized additive model (2SGAM) approach for fitting regression splines in the context of structural equation models. The model is first described and then investigated through the use of simulated data, in which it was compared with the SEMM approach. Results demonstrate that the 2SGAM is an effective tool for fitting a variety of nonlinear relationships between latent variables, and can be easily and accurately extended to models including multiple latent variables. Implications of these results are discussed.     

The Mediating Effect of Happiness on the Job Satisfaction of Aging Filipino Workers: A Structural Equation Model (SEM)

Job satisfaction is a frequently studied topic among scholars, but only few have taken into account the job satisfaction of older workers. This study aims to develop a model that describes the mediating effect of happiness on job satisfaction specifically on a select group of aging Filipino workers. This paper utilized structural equation modeling (SEM) for its data analysis to test what impact happiness has on job satisfaction. Three hundred aging Filipino workers, both from public and private organizations in the Philippines, took part in this study by completing a five-part research tool that consists of a Robotfoto (a Dutch term that describes a photo-like picture drawn by police to describe a suspect from a witness's illustration) and adopted questionnaires for physical and mental well-being, employee recognition, happiness, and job satisfaction. The structural equation model revealed that physical and mental well-being and employee recognition significantly affect—and are predictors of—happiness. Moreover, it was found that happiness has an impact on the job satisfaction of older workers. Findings generated in this study cater relevant ideas in developing programs and practices for the aging workforce in the field of Human Resource Management.