Application of Exploratory Structural Equation Modeling to Evaluate the Academic Motivation Scale

In this research, the authors examined the construct validity of scores of the Academic Motivation Scale using exploratory structural equation modeling. Study 1 and Study 2 involved 1,416 college students and 4,498 high school students, respectively. First, results of both studies indicated that the factor structure tested with exploratory structural equation modeling provides better fit to the data than the one tested with confirmatory factor analysis. Second, the factor structure was gender invariant in the exploratory structural equation modeling framework. Third, the pattern of convergent and divergent correlations among Academic Motivation Scale factors was more in line with theoretical expectations when computed with exploratory structural equation modeling rather than confirmatory factor analysis. Fourth, the configuration of convergent and divergent correlations connecting each Academic Motivation Scale factors to a validity criterion was more in line with theoretical expectations with exploratory structural equation modeling than with confirmatory factor analysis.     

The Development and Validation of the University Belonging Questionnaire

Although belonging in K–12 school settings has been abundantly researched and clearly defined, at the university level the research and construct definition is still in its infancy (Tovar & Simon, 2010). The present study sought to develop and validate an instrument measuring university belonging—the University Belonging Questionnaire (UBQ). In Study 1, an exploratory factor analysis was conducted with a sample of university students (N = 421), finding a reliable scale with three factors: (a) university affiliation, (b) university support and acceptance, and (c) faculty and staff relations. In Study 2, a confirmatory factor analysis on a new sample (N = 290), confirmed the final 3-factor, 24-item model. Further analyses demonstrated the convergent and incremental validity of the UBQ, as it positively correlated with measures of perceived social support, social connectedness, and general belonging. Implications and recommendations for university belonging research are discussed.     

Emotions in medical education: Examining the validity of the Medical Emotion Scale (MES) across authentic medical learning environments

To advance emotion research in education, there is a need to develop practical and context-relevant measures of emotion and to test the applicability of emotion theories using these measures. In two studies, we examined validity evidence of a self-report scale (the Medical Emotion Scale, MES) designed to measure the unique range of emotions activated within medical education. In Study 1, we administered the MES and conducted interviews with medical trainees (N = 15). Content analysis of interviews demonstrated that the MES captured an appropriate range of emotions and that there was alignment between scale responses and interview responses. In Study 2, we measured medical trainees' (N = 60) emotions using the MES for three learning environments. Results from principal components analysis revealed a structure of emotions according to valence (negative, positive) and novelty. The findings have implications for the measurement of emotions within technology-rich learning environments and beyond.     

INTERNAL FACTOR STRUCTURE AND CONVERGENT VALIDITY EVIDENCE: THE SELF‐REPORT VERSION OF SELF‐REGULATION STRATEGY INVENTORY

Using a sample of 348 middle school students, we gathered evidence regarding the internal consistency of scores, as well as the internal factor structure and convergent validity evidence for inferences from a self‐report questionnaire called the Self‐Regulation Strategy Inventory–Self Report. Confirmatory factor analysis revealed that the fit indexes for a hierarchical model (composite, three factors) and a single‐level, three‐factor model were highly similar but mixed. Respecification of the hierarchical model based on conceptual overlap of items led to substantial improvement in the overall fit of the model, as indicated by the root mean square error of approximation, chi‐square/df, and the comparative fit index. Correlational analyses also provided strong convergent validity evidence, as the three subscales exhibited statistically significant relations with four motivation beliefs (i.e., self‐efficacy, perceived instrumentality, task interest, perceived responsibility) and two distinct markers of regulation‐related behaviors (i.e., teacher ratings, office discipline referrals).     

An international validation of the engaged teacher scale

The purpose of this study was to test and validate the Engaged Teacher Scale (ETS) in a Turkish context (ETS-TR). In order to test the construct validity of the ETS, data were collected from 388 teachers in two northeast cities of Turkey. First-order confirmatory factor analysis results supported the 16-item and four-factor model of ETS while second-order confirmatory factor analysis suggested that a single factor was also appropriate for representing teacher engagement. Additionally, four multiple linear regression analyses were conducted to provide further validation evidence. Results showed that subscales of the ETS-TR were found to be positively correlated with teacher self-efficacy. Given our evidence of validity and reliability, we recommend researchers interested in measuring the engagement of Turkish teachers to consider using the ETS-TR. The adaptation of ETS into Turkish also provides a measure for use when conducting research examining cultural comparisons between english-speaking and Turkish teachers.     

Individual differences in trait motivation: development of the Motivational Trait Questionnaire

The development and initial evaluation of a measure of motivational traits, the Motivational Trait Questionnaire (MTQ), is described. Based upon theorizing by Kanfer and Heggestad (In B.M. Staw, & L.L. Cummings (Eds.), Research in organizational behaviour, vol. 19 (pp. 1–56). Greenwich, CT: JAI Press, Inc.) development of the MTQ began by identifying and defining five motivational traits. Item pools were generated for each of the proposed traits, and initial facets were developed through a content-sorting procedure. Two studies were conducted to evaluate the MTQ at the item, facet, and scale levels. In Study 1, the facet scales were refined based on item-level factor analyses and item characteristics. An exploratory factor analysis of the refined MTQ facets provided support for three of the proposed traits. In Study 2, the facets were re-evaluated at the item-level. The factor structure of the MTQ facets was similar to that found in Study 1. An extension analysis from the three trait factors to extant measures of achievement, test and trait anxiety, and personality provided construct validity evidence for the MTQ scales. Results from these studies support the multidimensional structure of motivational traits proposed by Kanfer and Heggestad (In B.M. Staw, & L.L. Cummings (Eds.), Research in organizational behaviour, vol. 19 (pp. 1–56). Greenwich, CT: JAI Press, Inc.). Implications for motivation research in education are discussed.     

Bicultural Work Motivation Scale for Asian American college students

The bicultural work motivations of Asian Americans have not yet been comprehensively captured by contemporary vocational constructs and scales. For this study, we conducted two studies on the initial reliability and validity of the Bicultural Work Motivation Scale (BWMS) by combining qualitative and quantitative methods. First, a pilot study was conducted with 14 Asian American college students to assess their motivation to work, and items were generated from the themes developed in the qualitative analysis. Results from the exploratory and confirmatory factor analyses supported a stable four-factor structure of the BWMS: Honoring Parents, Family Financial Obligation, Fulfilling Personal Interest, and Being Independent. Adequate reliability and evidence for concurrent and discriminant validity are presented. Implications of the scale for further vocational assessment and research are discussed.     

Assessment of attention in school children: teachers' ratings related to tests of attention

The paper provides (1) a teacher-administered rating instrument for inattention without confounding the rating with hyperactivity and conduct disorder, and (2) evidence that the ratings correlate with the scores obtained from cognitive tests of attention. In Study I, the first objective was to investigate the construct validity and the inter-rater reliability of the Attention Checklist (ACL) by factor analysing the teacher ratings of 110 Grade 4 children, obtained by using the ACL. The second objective was to investigate the predictive validity of the ACL by examining the relationship between the scores obtained for the participants from teachers' ratings using the ACL and the scores obtained by participants in the lab-type attention tests. The results of factor analysis showed that a single factor labelled ‘inattention’ underlies the 12 items in the ACL. Examining the differences in performance on attention tests, the ‘low attention’ children as rated by the teachers on the ACL scored lower than the ‘high attention’ children on the objective tests of attention. These findings were replicated in Study II, which was conducted to test further the construct validity and predictive validity of the ACL. This time, only those two tests (Auditory Attention and Visual Attention) that had shown relatively poor discrimination between the high and low attention groups in Study I were, again, administered to another cohort of 97 Grade 4 children, as it was our intention to further challenge the reliability of the ACL. Overall, the results of both studies suggest that comprehensive assessment of attention skills should include both ACL and objective measures of selective attention.     

Development of the human interaction dimension of the Self‐Regulated Learning Questionnaire in asynchronous online learning environments

Two studies focusing on the development and validation of the Online Self‐Regulated Learning Inventory (OSRLI) were conducted. The OSRLI is a self‐report instrument assessing the human interaction dimension of online self‐regulated learning. It consists of an affect/motivation scale and an interaction strategies scale. In Study 1, exploratory factor analysis of an initial affect/motivation item pool yielded four factors: enjoyment of human interaction, self‐efficacy for interaction with instructors, concern for interaction with students, and self‐efficacy for contributing to the online community. Exploratory factor analysis of an initial learning strategies item pool revealed three factors: writing strategies, responding strategies, and reflection strategies. In Study 2, confirmatory factor analysis was conducted in order to evaluate the stability of multidimensional factor structures. These exploratory and confirmatory factor analyses showed the OSRLI to be statistically moderate in terms of reliability and validity.     

Evaluating existing and new validity evidence for the Academic Motivation Scale

The current study evaluates existing and new validity evidence for the Academic Motivation Scale (AMS; Vallerand et al., 1992). We first provide a narrative review synthesizing past research, and then conduct a validity investigation of the scores from the measure. Data analysis using a sample of 1406 American college students provided construct validity evidence in the form of a well-fitting seven-factor model and adequate internal consistency of the item responses. Convergent and discriminant validity evidence provided insight into the distinctiveness of the seven subscales. However, support for the scale’s simplex structure, which would represent the self-determination theory continuum, was not fully substantiated. Implications for theory and the scale’s use in the current form are discussed.     

Academic delay of gratification

Academic delay of gratification (ADOG) refers to students' postponement of immediately available opportunities to satisfy impulses in favor of pursuing chosen important academic rewards or goals that are temporally remote but ostensibly more valuable. In Study 1, we developed a course-specific academic delay of gratification scale (ADOGS) with acceptable psychometric properties. Results of Study 2 supported the hypothesized association between ADOG and students' self-regulated learning, which consisted of academic motivation and the use of cognitive, metacognitive, and resource management learning strategies. Discriminant validity of the ADOGS is indicated by its stronger correlations with students' self-regulation than found for instruments that measure closelyrelated constructs (generalized deferment of gratification and impulsivity). The conceptual status of ADOG as a strategy and outcome of successful strategy use is discussed.     

A comprehensive reanalysis of the metacognitive self-regulation scale from the MSLQ

The metacognitive self-regulation (MSR) scale is among the most widely used measures of metacognition in educational research. However, the psychometric properties and validity of the scale have not been well established. A series of analyses on a college sample were performed to address this issue. In Study 1, a split-sample exploratory (EFA) and confirmatory factor analysis (CFA) was performed to test the one-factor specification of the MSR scale. Time and study environment (TSE), total study time, and cumulative grade performance average (cGPA) were introduced as outcome variables in a structural equation model (SEM) to examine the factors suggested by the EFA. The results of Study 1 indicated poor one-factor model fit and suggested two and three-factor models provided improved fits of the sample data. Results from the SEM indicated the novel factors from the two and three-factor models had different relationships with the outcome variables than the originally specified one-factor model. In Study 2, a modified one-factor model was introduced that consisted of nine items and was named metacognitive self-regulation revised (MSR-R). Five additional samples were included to replicate the model fit for the revised model specification. Finally, a path analysis was performed to examine the relationship of the MSR-R to variables from Study 1. The results of Study 2 revealed improved psychometric properties and reliability for the MSR-R. An indirect relationship emerged between MSR-R and cGPA through TSE. In conclusion, convincing evidence for replacing the MSR was found and implications of the revised scale for future studies was discussed.     

Measurement invariance across information and communication technology development index and gender: The case of the Pearson Test of English Academic reading

The computerization of reading assessments has presented a set of new challenges to test designers. From the vantage point of measurement invariance, test designers must investigate whether the traditionally recognized causes for violating invariance are still a concern in computer-mediated assessments. In addition, it is necessary to understand the technology-related causes of measurement invariance among test-taking populations. In this study, we used the available data (n = 800) from the previous administrations of the Pearson Test of English Academic (PTE Academic) reading, an international test of English comprising 10 test items, to investigate measurement invariance across gender and the Information and Communication Technology Development index (IDI). We conducted a multi-group confirmatory factor analysis (CFA) to assess invariance at four levels: configural, metric, scalar, and structural. Overall, we were able to confirm structural invariance for the PTE Academic, which is a necessary condition for conducting fair assessments. Implications for computer-based education and the assessment of reading are discussed.     

Moving beyond academic achievement goal measures: A study of social achievement goals

The current research explored the theory of social goal orientation. More specifically, we conducted three studies utilizing six-independent university student samples to evaluate the construct validity of the Social Achievement Goal Orientation Scale (SAGOS; Ryan & Hopkins, 2003), a measure representing the construct of social goal orientation. The purpose of Study 1 was to: (1) compare the three-dimensional (mastery, performance-approach, and performance-avoidance) model of social goal orientation to three theoretically based competing models, (2) examine item functioning, and (3) assess generalizability of the factor structure. The fit of the proposed three-factor model was promising; however, areas of misfit and problematic items were identified. Stronger support for the three-factor structure of goal orientation was found using scores from an abbreviated 13-item SAGOS. In Study 2, item wording was altered slightly to evaluate a revised Social Achievement Goal Scale (SAGS), yet resulted in similar findings. Study 3 examined external validity evidence for the SAGS, garnering some support for the meaning of the scores. Although continued refinement of the SAGOS and SAGS is recommended, the findings help contribute to our general understanding and conceptualization of social goal theory and the role that social goals may play in academic contexts.     

Self-Efficacy: An Essential Motive to Learn

During the past two decades, self-efficacy has emerged as a highly effective predictor of students' motivation and learning. As a performance-based measure of perceived capability, self-efficacy differs conceptually and psychometrically from related motivational constructs, such as outcome expectations, self-concept, or locus of control. Researchers have succeeded in verifying its discriminant validity as well as convergent validity in predicting common motivational outcomes, such as students' activity choices, effort, persistence, and emotional reactions. Self-efficacy beliefs have been found to be sensitive to subtle changes in students' performance context, to interact with self-regulated learning processes, and to mediate students' academic achievement. Copyright 2000 Academic Press.     

Beliefs about Schooled Knowledge: Domain Specific or Domain General?

This study explored the domain specificity of students' beliefs about academic knowledge in three related studies. Using a four-factor model as an initial framework, a series of domain-specific items about mathematics and history was developed. In Study I, these items were administered to 182 undergraduates, and the psychometric properties and underlying factor structure were examined via exploratory factor analysis. In Study II, the modified instrument, the Domain-Specific Beliefs Questionnaire (DSBQ), was administered to 633 students and a confirmatory factor analysis was conducted. A subsample of participants' responses on the DSBQ was also compared to responses on Schommer's epistemological questionnaire. Study III involved a second confirmatory factor analysis using data from a new sample (n = 523). Those data were examined for potential gender differences. Overall, the results of this investigation suggested that students possess certain domain-specific beliefs about knowledge in mathematics and history. Further, a significant moderate relationship between the DSBQ and Schommer's questionnaire provided some evidence of domain-generality in undergraduates' epistemological beliefs.     

The happy-fish-little-pond effect on enjoyment: Generalizability across multiple domains and countries

Academic enjoyment is an important educational construct given that it benefits students' engagement, persistence, wellbeing, and mental health. In this study, we examine two factors that determine this crucial emotion, namely student- and class-level achievement. Past research has been restricted to single-country or single-domain examinations of secondary school students, limiting generalizability of findings. To bridge this gap, we utilize the Trends in International Mathematics and Science Study and the Progress in International Reading Literacy Study (TIMSS-PIRLS) combined 2011 data (N = 180,084 4th-grade students, 37 countries). Our results provide robust evidence that student-level achievement positively predicts enjoyment in math, science, and reading, while the effects of class-level achievement are negative—the Happy-Fish-Little-Pond Effect. These results showed relative universality across the domains and countries examined.     

The Mediating Effects of Approaches to Learning on the Academic Success of First-Year College Students

Students’ personal predictors of academic success are particularly relevant for first-year college students, given the specific challenges that these students face when entering higher education (HE). Academic success in HE has been related to multiple factors, including the students’ approaches to learning (SAL), satisfaction (linked to commitment and persistence), study time (effort), and prior academic achievement. This study analyzes the combined effect of these predictors on perceived academic success. Data from 247 students was collected using the Approaches and Study Skills Inventory for Students and other specific measures to assess presage and process variables of academic success. Although academic success is multidimensional and difficult to explain, factors such as prior academic achievement, satisfaction with the course, SAL, and study time contribute to explain perceived academic success in first-year college students.     

Measuring perceived social support: Development of the child and adolescent social support scale (CASSS)

The measurement of students' social support has become a popular topic in education and psychology, yet measurement tools in this area are limited. In this study, we use a large, representative sample to conduct confirmatory factor, reliability, and correlational analyses of scores on the Child and Adolescent Social Support Scale (CASSS; Malecki, Demaray, Elliott, & Nolten, 1999). These analyses revealed evidence of reliability, a four‐factor structure (Parent, Teacher, Classmate, and Close Friend subscales), and construct validity. The results of this study indicate that the CASSS covaries as predicted with the clinically important constructs of self‐concept, social skills, and behavioral indicators. There is evidence that the CASSS can be used to understand children and adolescents' perceived social support. © 2002 John Wiley & Sons, Inc.     

Gender Gaps in North American Research Productivity: Examining Faculty Publication Rates in Mexico,Canada, and the U.S.

The present study addresses gender gaps in North American research productivity, which may be influenced by personal and family variables, as well as professional and work-related variables. The study was conducted as part of the Changing Academic Profession (CAP) International Survey, conducted in 2007–08. Using articles as indicator of research productivity, we analyzed the gender gap in publication rates among full-time higher education faculty in our combined sample (Canada, Mexico, and the United States). This analysis has implications for higher education policy. In terms of research productivity, the relative productivity rates of male and female academics have been a policy priority for many years to increase the cumulative rates of research activity. We found that the variables related to research intensity varied by country, providing a more nuanced understanding of the gender gap between male and female faculty.