Exploratory Structural Equation Modeling,Integrating CFA and EFA: Application to Students' Evaluations of University Teaching

This study is a methodological-substantive synergy, demonstrating the power and flexibility of exploratory structural equation modeling (ESEM) methods that integrate confirmatory and exploratory factor analyses (CFA and EFA), as applied to substantively important questions based on multidimentional students' evaluations of university teaching (SETs). For these data, there is a well established ESEM structure but typical CFA models do not fit the data and substantially inflate correlations among the nine SET factors (median rs = .34 for ESEM, .72 for CFA) in a way that undermines discriminant validity and usefulness as diagnostic feedback. A 13-model taxonomy of ESEM measurement invariance is proposed, showing complete invariance (factor loadings, factor correlations, item uniquenesses, item intercepts, latent means) over multiple groups based on the SETs collected in the first and second halves of a 13-year period. Fully latent ESEM growth models that unconfounded measurement error from communality showed almost no linear or quadratic effects over this 13-year period. Latent multiple indicators multiple causes models showed that relations with background variables (workload/difficulty, class size, prior subject interest, expected grades) were small in size and varied systematically for different ESEM SET factors, supporting their discriminant validity and a construct validity interpretation of the relations. A new approach to higher order ESEM was demonstrated, but was not fully appropriate for these data. Based on ESEM methodology, substantively important questions were addressed that could not be appropriately addressed with a traditional CFA approach.     

A synthesis model of sustainable market orientation: conceptualization,measurement, and influence on academic accreditation – a case study of Egyptian-accredited faculties

Higher education institutions are increasingly concerned about accreditation. Although sustainable market orientation (SMO) bears on academic accreditation, to date, no study has developed a valid scale of SMO or assessed its influence on accreditation. The purpose of this paper is to construct and validate an SMO scale that was developed in Egyptian faculties. SMO is identified as a one-dimensional construct consisting of four overlapping components. Using a survey, data were collected from 204 respondents in 6 Egyptian-accredited governmental faculties. Both item analysis and split-half methods were used to purify the measurement scale and assess its stability. Exploratory factor analysis was used to assess dimensionality, and confirmatory factor analysis was used to examine the construct and convergent/discriminant validity. Nomological validity was assessed with a structural equation model. Results suggest both a validated scale and empirical evidence of the influence of SMO on academic accreditation.     

Self-regulation in three types of online interaction: a scale development

The purpose of this study was to develop a scale with which to examine students’ self-regulation (SR) in three types of online interaction. Using scale development steps, we constructed the online self-regulation questionnaire (OSRQ), a self-report survey. A total of 799 online students participated in the study. Data from 400 randomly selected participants were used for exploratory factor analysis (EFA), and data of the remaining 399 participants were used for confirmatory factor analysis (CFA). The EFA yielded three factors as hypothesized: SR in interaction between student and content, SR in interaction between student and student and SR in interaction between student and teacher. The CFA demonstrated that the factor structures appearing in the EFA were also observed with different participants. In addition, we found convergent validity in the OSRQ. The study contributes to understanding SR in online learning settings.     

Self-assessment instrument to measure the competencies of Namibian graduates: Testing of validity and reliability

Namibia has been reported to be one of the countries with the highest unemployment rates. In this work, the reliability and validity of the self-assessment instrument used to measure competencies of graduates in Namibia were assessed using exploratory factor analysis (EFA) and second-order confirmatory factor analysis (CFA). The EFA results demonstrated that the twenty indicators can be categorized into five factors, namely, “management and resilience”, “professional and communication”, “teamwork and critical thinking”, “self-control”, and “achievement motive”. The CFA results showed that all of the factors and indicators are highly reliable with good construct validity. Students and graduates could employ this validated self-assessment instrument to assess or diagnose a pattern of strengths and weaknesses in their own competencies and provide themselves with a realistic and objective estimate of their employability, as well as help them increase effectiveness in their workplace.     

Bicultural Work Motivation Scale for Asian American college students

The bicultural work motivations of Asian Americans have not yet been comprehensively captured by contemporary vocational constructs and scales. For this study, we conducted two studies on the initial reliability and validity of the Bicultural Work Motivation Scale (BWMS) by combining qualitative and quantitative methods. First, a pilot study was conducted with 14 Asian American college students to assess their motivation to work, and items were generated from the themes developed in the qualitative analysis. Results from the exploratory and confirmatory factor analyses supported a stable four-factor structure of the BWMS: Honoring Parents, Family Financial Obligation, Fulfilling Personal Interest, and Being Independent. Adequate reliability and evidence for concurrent and discriminant validity are presented. Implications of the scale for further vocational assessment and research are discussed.     

A Simulation Study of Polychoric Instrumental Variable Estimation in Structural Equation Models

Data collected from questionnaires are often in ordinal scale. Unweighted least squares (ULS), diagonally weighted least squares (DWLS) and normal-theory maximum likelihood (ML) are commonly used methods to fit structural equation models. Consistency of these estimators demands no structural misspecification. In this article, we conduct a simulation study to compare the equation-by-equation polychoric instrumental variable (PIV) estimation with ULS, DWLS, and ML. Accuracy of PIV for the correctly specified model and robustness of PIV for misspecified models are investigated through a confirmatory factor analysis (CFA) model and a structural equation model with ordinal indicators. The effects of sample size and nonnormality of the underlying continuous variables are also examined. The simulation results show that PIV produces robust factor loading estimates in the CFA model and in structural equation models. PIV also produces robust path coefficient estimates in the model where valid instruments are used. However, robustness highly depends on the validity of instruments.     

The development and validation of the emotion labor in families scale: Associations with emotion regulation,feeling caught,and relational satisfaction in parent-child relationships

In this study, we developed and validated a new inventory that assesses emotion labor in parent-child relationships. Participants included 230 young adults who reported on the frequency with which they engaged in emotion labor with both parents. Results produced a 24-item inventory that assesses young adults’ surface acting, deep acting, and emotional effort when having conversations with their mother and father, labeled the Emotion Labor in Families (ELF) scale. Tests of concurrent and discriminant validity supported the construct validity of the ELF scale, although the dimension of emotional effort was largely unassociated with the validity measures included in this report.     

CONVERGENT AND DISCRIMINANT VALIDITY OF THREE MEASURES OF ABILITY,ASPIRATION-LEVEL,ACHIEVEMENT, ADJUSTMENT AND DOMINANCE1

The convergent and discriminant validity of three measures of the concepts of aspiration level, ability, achievement, adjustment, and dominance were examined in the context of a multitrait-multimethod matrix. Self-reports and peer-reports on 75 Ss were employed as two measures of each trait. In addition, aspiration level was measured by the Edwards Personal Preference Schedule (EPPS) Nach scale, dominance by the EPPS (dom scale), achievement by cumulative college grade point ratio (GPR), ability by the Ohio State Psychological Examination (OSPE), and adjustment by the Bell Adjustment Inventory. Of the paper and pencil instruments, only the OSPE and EPPS (dominance scale) exhibited satisfactory convergent validity. No measure met all the requirements of discriminant validity. The desirability of establishing adequate validational evidence prior to using “trait” measures in studies relating theoretical variables was emphasized.     

A Multilevel CFA–MTMM Approach for Multisource Feedback Instruments: Presentation and Application of a New Statistical Model

Multisource feedback instruments are a widely used tool in human resource management. However, comprehensive validation studies remain scarce and there is a lack of statistical models that account appropriately for the complex data structure. Because both peers and subordinates are nested within the target but stem from different populations, the assumption of traditional multilevel structural equation models that the sample on a lower level stems from the same population is violated. We present a multilevel confirmatory factor analysis multitrait–multimethod (ML–CFA–MTMM) model that considers this peculiarity of multisource feedback instruments. The model is applied to 2 scales of the Benchmarks® instrument and it is demonstrated how measures of reliability and of convergent and discriminant validity can be obtained using multilevel structural equation modeling software. We discuss the results as well as some implications and guidelines for the use of the model.     

Convergent and discriminant validity of the mental processing scales of the Kaufman Assessment Battery for Children

Convergent and discriminant validity of the Mental Processing Scales of the Kaufman Assessment Battery for Children (K-ABC) were examined using 51 first-grade children. Convergent validity was assessed using the Reading Recognition and Comprehension subtests of the Peabody Individual Achievement Test. Discriminant validity was assessed using a measure of anxiety/self-esteem, the Child Anxiety Scale, and a measure of hyperactive behavior, the Hyperactivity scale of the Achenbach Child Behavior Checklist. Results supported the convergent validity of the K-ABC; correlations with reading achievement were fairly large. The discriminant validity received only partial support. The K-ABC did not correlate with the Child Anxiety Scale, but did show rather large correlations with the measure of hyperactive behavior. Implications for understanding what the K-ABC Mental Processing Scales are measuring are discussed.     

What is technology? Development and evaluation of a simple instrument for measuring children's conceptions of technology

Technology is important to all aspects of our lives, so helping students develop an accurate understanding of technology should be an educational goal at the K-12 level. Assessments are important tools in reaching this goal. We developed an instrument to measure the technology conceptions of children ages 8–11, the ‘What is Technology’ (WT) instrument. We gathered evidence for the validity of using the WT instrument to measure children's conceptions of technology, including changes due to an intervention, and we describe those changes. We used exploratory factor analysis (EFA) to establish scales, and confirmatory factor analysis (CFA) with new samples of subjects to confirm our hypothesised model of children's conceptions. For further evidence, we analyzed and coded the written responses children gave to open-ended questions asking them to explain their understanding of technology, then calculated correlations between these codes and the CFA-confirmed scale measures. We found that children tend to think of technology as artifacts that are powered by electrical energy; however, after instruction most children's conceptions become more consistent with definitions given in educational standards. The instrument is shown to be valid and reliable for its intended use, to assess preadolescent children's conceptions of technology, and evaluate the impact of an intervention.     

Phonological processing and emergent literacy in Spanish-speaking preschool children

Phonological awareness (PA), phonological memory (PM), and phonological access to lexical storage (also known as RAN), play important roles in acquiring literacy. We examined the convergent, discriminant, and predictive validity of these phonological processing abilities (PPAs) in 147 3-, 4-, and 5-year-old children whose native language was Spanish. Confirmatory factor analysis (CFA) supported the validity of each PPA as separate from general cognitive ability and separate from each other. Moreover, structural equation modeling found RAN uniquely associated with knowledge of Spanish letter names and Spanish letter sounds. PA was found the best predictor of children’s ability to distinguish alphabetic text from nonalphabetic text. Finally, general cognitive ability was only indirectly associated with emergent literacy skills via PPAs. These results highlight the importance of PPAs in the early literacy development of native Spanish speaking preschool children.     

Assessing the psychometric properties of Kember and Leung’s Reflection Questionnaire

Reflective thinking is often stated as a learning outcome of baccalaureate nursing education, and as a characteristic of a competent professional; however, no consistent method exists to assess the extent to which students engage in reflective thinking. To address this need, Kember and Leung developed and tested a self-report questionnaire based on Mezirow’s conceptualisation of levels of reflective thinking. The purpose of this study was to test the psychometric properties of the Reflection Questionnaire, developed by Kember and Leung. A convenience sample (n?=?538) of third-year baccalaureate nursing students from four collaborative nursing programmes in Ontario was used. Ethical approval was secured from 10 sites. Second-order confirmatory factor analyses (CFA) were used to test the factor structure of the Reflection Questionnaire. This research was part of a larger study on reflective thinking and is a first step in validating a four-level measure of reflective thinking, in educational environments, with baccalaureate nursing students. The results of the second-order CFA provide support for the construct validity of reflective thinking. Results of this study contribute to the evidence supporting the reliability and validity of the questionnaire. Nurse educators can use this information when implementing the questionnaire, and learning the extent to which students are engaging in the reflective thinking process.     

Validity of alternative approaches for the identification of learning disabilities: operationalizing unexpected underachievement

This article reviews the validity of models based on (a) aptitude-achievement discrepancies, (b) low achievement, (c) intraindividual differences, and (d) response to instruction for the classification and identification of learning disabilities (LD). Models based on aptitude-achievement discrepancies and intraindividual differences showed little evidence of discriminant validity. Low achievement models had stronger discriminant validity but do not adequately assess the most significant component of the LD construct, unexpected underachievement. All three of these status models have limited reliability because of their reliance on a measurement at a single time point. Models that incorporate response to instruction have stronger reliability and validity but cannot represent the sole criterion for LD identification. Hybrid models combining low achievement and response to instruction most clearly capture the LD construct and have the most direct relation to instruction. The assessment of students for LD must reflect a stronger underlying classification that takes into account relations with other developmental disorders as well as the reliability and validity of the underlying classification and resultant identification system.     

Adaptation and validation of the teacher emotional labour strategy scale in China

The present study reports the adaptation and validation of the Teacher Emotional Labour Strategy Scale (TELSS) as tested on samples of 633 Beijing teachers and 648 Chongqing teachers in Chinese mainland. Results show that the 13-item TELSS adapted for this study had good internal consistency on three subscales which measure three types of teacher emotional labour strategy, namely, surface acting, deep acting and expression of naturally felt emotions. Confirmatory factor analysis using different data-sets provided support for the construct validity of this TELSS. In addition, the convergent, discriminant and concurrent validity of this newly-adapted TELSS were examined in the present study.     

大学英语形成性评估效度实证研究

课堂形成性评估指教学过程中通过及时、有效的反馈促进语言教学健康发展的评估手段。课堂形成性评估能否作为终结性标准化测试的有效补充,在很大程度上取决于评估的效度。大学英语课堂形成性评估内容效度、结构效度和互存效度的分析表明：课堂形成性评估能提供更全面、更有意义的信息来描述学生的学习行为、能力发展和成绩进步,促进教学目标的实现。     

师范生职业认同感量表的初步编制

目的：编制师范生职业认同感量表。方法：运用文献法、专家咨询法设计原始量表,多次施测修改量表,采用相关分析、探索性因素分析和验证性因素分析对量表的信、效度进行检验。结果：师范生职业认同感量表包含职业意愿与期望、职业意志、职业价值和职业效能四个维度,累计贡献率63.053%,共12项题目;问卷的信度和效度均符合心理测量学的要求。结论：师范生职业认同感量表可作为测评师范生职业认同感的工具。     

Validity of the parenting inventory: Young children

The Parenting Inventory: Young Children (PI) is a rating scale that measures the developmental expectations and behaviors of parents of children who are between the ages of 1 and 4 years, 11 months. Using a representative urban sample of 1,056 mothers, the PI was found to discriminate successfully between parents with children of different ages and to have substantial item-construct correlations (content validity). The PI was not strongly related to the Adult-Adolescent Parenting Inventory, a measure of parenting attitudes (discriminant validity). Reasons for this latter finding are discussed. Potential uses of the PI for practitioners working with parents in the schools are described.     

自我监控量表辨别效度的研究

该研究用355名被试,考验了新自我监控量表的辨别效度。根据自我监控量表三个子量表：他人导向、自我导向和高自我监控,把被试分成四组：他人导向、自我导向、高自我监控和低自我监控组,验证他们在三类效标上是否存在预期的差异,效标分别为：对内外线索敏感、行为控制点、社会适应性。研究结果表明：新自我监控量表具有较好的辨别效度。     

The Academic Diligence Task (ADT): assessing individual differences in effort on tedious but important schoolwork

The current study reports on the development and validation of the Academic Diligence Task (ADT), designed to assess the tendency to expend effort on academic tasks which are tedious in the moment but valued in the long-term. In this novel online task, students allocate their time between solving simple math problems (framed as beneficial for problem solving skills) and, alternatively, playing Tetris or watching entertaining videos. Using a large sample of high school seniors (N = 921), the ADT demonstrated convergent validity with self-report ratings of Big Five conscientiousness and its facets, self-control and grit, as well as discriminant validity from theoretically unrelated constructs, such as Big Five extraversion, openness, and emotional stability, test anxiety, life satisfaction, and positive and negative affect. The ADT also demonstrated incremental predictive validity for objectively measured GPA, standardized math and reading achievement test scores, high school graduation, and college enrollment, over and beyond demographics and intelligence. Collectively, findings suggest the feasibility of online behavioral measures to assess noncognitive individual differences that predict academic outcomes.