Measurement invariance across information and communication technology development index and gender: The case of the Pearson Test of English Academic reading

The computerization of reading assessments has presented a set of new challenges to test designers. From the vantage point of measurement invariance, test designers must investigate whether the traditionally recognized causes for violating invariance are still a concern in computer-mediated assessments. In addition, it is necessary to understand the technology-related causes of measurement invariance among test-taking populations. In this study, we used the available data (n = 800) from the previous administrations of the Pearson Test of English Academic (PTE Academic) reading, an international test of English comprising 10 test items, to investigate measurement invariance across gender and the Information and Communication Technology Development index (IDI). We conducted a multi-group confirmatory factor analysis (CFA) to assess invariance at four levels: configural, metric, scalar, and structural. Overall, we were able to confirm structural invariance for the PTE Academic, which is a necessary condition for conducting fair assessments. Implications for computer-based education and the assessment of reading are discussed.     

Testing for factorial invariance of the Modified Leadership Scale for Sports: using a Japanese version

The objective of this study was to provide empirical evidence to support psychometric properties of a modified four-dimensional model of the Leadership Scale for Sports (LSS). The study tested invariance of all parameters (i.e., factor loadings, error variances, and factor variances–covariances) in the four-dimensional measurement model between two groups of student-athletes. For testing multi-group invariance of the proposed scale, 335 middle school and 320 high school student-athletes in Japan participated in this study. The modified version of the LSS consists of 35 items representing training instruction, democratic behaviour, positive feedback, and social support. A chi-square difference test was employed for model comparisons. The results supported configural, metric, scalar and factor variance–covariance invariance in the modified LSS across the two student-athlete groups.     

密苏里同伴关系问卷在中国未成年犯中的信效度检验

采用分层随机抽样原则对257名男性未成年犯进行问卷调查,结果表明：修订后的问卷只包括因素一情感维系度和因素二攻击性,总的信度为0.7351。验证性因素分析的结果为χ2/df=4.42,NFI=0.81,CFI=0.86,RSMEA=0.00。中文版的《密苏里同伴关系问卷》有良好的信度和效度,可用于未成年人犯罪同伴关系的研究。     

Structural validity and invariance of the Feedback Perceptions Questionnaire

Despite a growing interest in instructional feedback, students’ feedback perceptions received limited attention. We examined the structural validity and measurement invariance of the Feedback Perceptions Questionnaire (FPQ). The FPQ measures feedback perceptions in terms of perceived fairness, usefulness, acceptance, willingness to improve, and affect. Secondary school students (N = 1486) received a fictional scenario containing Concise General Feedback or Elaborated Specific Feedback by a fictional peer. Students rated their perceptions as if they had received the feedback themselves. Confirmatory Factor Analysis (CFA) supports the structural validity of the FPQ and its invariance for the two types of peer feedback, gender, four grade-levels and two tracks. Perceived fairness of peer feedback was a strong positive predictor of willingness to improve and affect, whereas perceived usefulness and acceptance of peer feedback showed a more complex pattern in predicting willingness to improve and affect.     

The potential of anchoring vignettes to increase intercultural comparability of non-cognitive factors

ABSTRACT

We investigate whether Anchoring Vignettes (AV) improve intercultural comparability of non-cognitive student-directed factors (e.g., procrastination). So far, correlation analyses for anchored and non-anchored scores with a criterion have been used to demonstrate the effectiveness of AV in improving data quality. However, correlation analyses are often used to investigate external validity of a scale. Nonetheless, before testing for validity, the reliability of the measurement of a construct should be examined. In the present study, we tested for measurement invariance across countries and languages and compared anchored and non-anchored student-directed self-reports that are highly relevant for the students’ self and their behaviour and performance. In addition, we apply further criteria for testing reliability. The results indicate that the data quality for some of the constructs can – in fact – be improved slightly by anchoring; whereas, for other self-reports, anchoring is less successful than was hoped. We discuss with regard to possible consequences for research methodology.     

Development and validation of a short form of the Teacher Efficacy for Inclusive Practices Scale (TEIP-SF)

High self-efficacy is a marker of successful teaching and is, therefore, a subject of great interest to research on inclusive education. One of the most frequently used instruments to assess such beliefs is the Teacher Efficacy for Inclusive Practice (TEIP) scale. Although used widely, some studies did not precisely replicate the original factor structure, and no short form of the TEIP scale currently exists, although this could enhance measurement efficiency. This study (1) systematically assessed the TEIP scale's factor structure and psychometric properties, (2) identified potentially problematic items and developed a more concise short form of the scale, and (3) evaluated its dimensionality and criterion and convergent validities using three validation samples of teachers in three different countries (486 in Switzerland, 189 in Australia and 276 in Canada). Compared to the full-length TEIP scale, the TEIP-SF uses half the items, demonstrates better model fit and reveals a clearer distinction of domain-specific factors. In conclusion, the TEIP-SF represents a concise, efficient means of assessing teachers' self-efficacy about teaching in inclusive classrooms.     

Preliminary development of the Brief–California School Climate Survey: dimensionality and measurement invariance across teachers and administrators

A brief 15-item version of the California School Climate Scale (Brief-CSCS) is presented to fill a need for a measure that could be used for periodic monitoring of school personnel's general perception of the climate of their school campus. From a sample of 81,261 California school personnel, random subsamples of 2,400 teachers and 2,400 administrators were used in the analyses. Confirmatory factor analyses supported a model in which general school climate was a second-order latent factor composed of 2 first-order latent traits, organizational supports and relational supports. Measurement invariance of factor loadings for teachers and administrators was found. Additional analyses revealed that administrators held more positive perceptions of school climate than teachers, with this difference increasing from primary through high school. The implications for these findings for educational research and policy reform are outlined.     

Model Misspecification and Invariance Testing Using Confirmatory Factor Analytic Procedures

Confirmatory factor analytic procedures are routinely implemented to provide evidence of measurement invariance. Current lines of research focus on the accuracy of common analytic steps used in confirmatory factor analysis for invariance testing. However, the few studies that have examined this procedure have done so with perfectly or near perfectly fitting models. In the present study, the authors examined procedures for detecting simulated test structure differences across groups under model misspecification conditions. In particular, they manipulated sample size, number of factors, number of indicators per factor, percentage of a lack of invariance, and model misspecification. Model misspecification was introduced at the factor loading level. They evaluated three criteria for detection of invariance, including the chi-square difference test, the difference in comparative fit index values, and the combination of the two. Results indicate that misspecification was associated with elevated Type I error rates in measurement invariance testing.     

The Impact of Varying the Number of Measurement Invariance Constraints on the Assessment of Between-Group Differences of Latent Means

The objective was to offer guidelines for applied researchers on how to weigh the consequences of errors made in evaluating measurement invariance (MI) on the assessment of factor mean differences. We conducted a simulation study to supplement the MI literature by focusing on choosing among analysis models with different number of between-group constraints imposed on loadings and intercepts of indicators. Data were generated with varying proportions, patterns, and magnitudes of differences in loadings and intercepts as well as factor mean differences and sample size. Based on the findings, we concluded that researchers who conduct MI analyses should recognize that relaxing as well as imposing constraints can affect Type I error rate, power, and bias of estimates in factor mean differences. In addition, fit indexes can be misleading in making decisions about constraints of loadings and intercepts. We offer suggestions for making MI decisions under uncertainty when assessing factor mean differences.     

美国校长信息化领导力提升项目的特点与启示

为了让社会、学校、教师对教育信息化领导力的培养方式有全面认识,使校长信息化领导力提升有更具体可参照的方案,本研究通过网络等方式获取文献资料,分析美国乔治华盛顿大学的教育信息化领导力项目、美国肯塔基大学教育领导力研究院的学校信息化领导力项目及美国教育信息化领导力认证考试项目,发现其特点包括：多层次培养目标,满足学习者个别化选择;灵活的入学形式和网络授课,便于学习者安排学习活动;多学科交叉的课程设置,体现教育、技术和管理的融合;参照国际标准,培养和构建世界一流师资队伍。我国校长信息化领导力提升项目应该根据国家标准,体现校长信息化领导力的特点;根据学校发展战略,发挥校长信息化领导力的宏观作用;选择适当院校,开展校长信息化领导力水平认证;凸显课程的交叉融合特征,探讨校长信息化领导力的学位授予。     

积极人格特质问卷PPTQ在中国大学生中的因素结构研究

探索积极人格特质问卷(PPTQ)在中国大学生中的因素结构。采用积极人格特质问卷(Positive Personality Traits Questionnaire)中文版,对648名大学生施测,对其中一半数据使用PASWStatistics18进行探索性因素分析,另一半数据使用AMOS16.0进行验证性因素分析(CFA)。探索性因素分析得出积极自我意象、外向性和文化认同三因素结构。累计解释率为50.457%,验证性因素分析结果显示:χ2/df=2.230,GFI=0.841。AGFI=0.816,CFI=0.822,RMSEA=0.062,中文版三因素结构在中国大学生人群中较为合理。     

父亲攻击型教养方式问卷的初步编制

为了解家庭教育中“攻击性”这一特征,有针对性地为实施健康家庭教育提供依据,在文献分析、访谈、开放式问卷等方法的基础上收集项目,采用探索性因素分析获得父亲攻击型教养方式七因子模型：督促攻击、默许对朋辈的攻击、责罚、惩罚对长辈的攻击、默许对长辈的攻击、攻击示范、攻击性教养观。此模型在验证性因素分析中得到了验证,具有良好的信度和效度,这对于我国家庭教养方式本土化研究提供了一定理论基础。     

Socioeconomic and gender group differences in early literacy skills: a multiple-group confirmatory factor analysis approach

Socioeconomic status and gender are important demographic variables that strongly relate to academic achievement. This study examined the early literacy skills differences between 4 sociodemographic groups, namely, boys ineligible for free or reduced-price lunch (FRL), girls ineligible for FRL, boys eligible for FRL, and girls eligible for FRL. Data on kindergarteners (N = 462) were analysed using multiple-group confirmatory factory analysis. Early literacy skill differences between boys and girls are more nuanced than previously reported; subsidy status and gender interact. Both boys and girls from high-poverty households performed significantly lower than the girls from low-poverty households in alphabet knowledge, phonological awareness, and spelling. There were gender gaps, with a female advantage, among children from high-poverty households in alphabet knowledge and spelling and among children from low-poverty households in alphabet knowledge. These results highlight the importance of employing methodologically sound techniques to ascertain group differences in componential early literacy skills.     

An Investigation of the Alignment Method With Polytomous Indicators Under Conditions of Partial Measurement Invariance

The alignment method (Asparouhov & Muthén, 2014) is an alternative to multiple-group factor analysis for estimating measurement models and testing for measurement invariance across groups. Simulation studies evaluating the performance of the alignment for estimating measurement models across groups show promising results for continuous indicators. This simulation study builds on previous research by investigating the performance of the alignment method’s measurement models estimates with polytomous indicators under conditions of systematically increasing, partial measurement invariance. We also present an evaluation of the testing procedure, which has not been the focus of previous simulation studies. Results indicate that the alignment adequately recovers parameter estimates under small and moderate amounts of noninvariance, with issues only arising in extreme conditions. In addition, the statistical tests of invariance were fairly conservative, and had less power for items with more extreme skew. We include recommendations for using the alignment method based on these results.     

《大学生主观幸福感问卷》的编制

过去国内对主观幸福感(包括大学生)的研究更多是直接引用或修订国外编制的问卷或量表.所以引用过程中民族个性差异,人口统计量的差异,文化差异以及翻译等额外变量使得研究结论的准确性受到质疑,所以编制适用于中国大学生特点的问卷对主观幸福感的本土化研究有着重要意义.     

Challenging the Universality of Achievement Goal Models: A Comparison of Two Culturally Distinct Countries

ABSTRACT

Achievement goal theory is one of the most widespread motivation models within education research. Strong empirical support exists for the trichotomous model, comprising mastery-approach, performance-approach, and performance-avoidance goals. However, research also indicates problems with model transferability between contexts. In this study, based on questionnaire data from 4201 students, we use confirmatory factor analysis to compare the factor structures of students’ achievement goals in two culturally distinct countries. Factor structures for Grades 5–11 within the two countries were also compared. Results show that the separation between performance-approach and performance-avoidance goals differs between the two countries, and that this difference is consistent over the grades. Hence, results indicate that the model is not freely transferable between countries. The results are discussed in relation to differences in national culture and other proposed explanations such as age, perceived competence, and questionnaire characteristics.     

Exploratory and Confirmatory Factor Analysis of the School Level Environment Questionnaire (SLEQ)

The purpose of the present study was to validate an existing school environment instrument, the School Level Environment Questionnaire (SLEQ). The SLEQ consists of 56 items, with seven items in each of eight scales. One thousand, one hundred and six (1106) teachers in 59 elementary schools in a southwestern USA public school district completed the instrument. An exploratory factor analysis was undertaken for a random sample of half of the completed surveys. Using principal axis factoring with oblique rotation, this analysis suggested that 13 items should be dropped and that the remaining 43 items could best be represented by seven rather than eight factors. A confirmatory factor analysis was run with the other half of the original sample using structural equation modeling. Examination of the fit indices indicated that the model came close to fitting the data, with goodness-of-fit (GOF) coefficients just below recommended levels. A second model was then run with two of the seven factors, with their associated items removed. That left five factors with 35 items. Model fit was improved. A third model was tried, using the same five factors with 35 items but with correlated residuals between some of the items within a factor. This model seemed to fit the data well, with GOF coefficients in recommended ranges. These results led to a refined, more parsimonious version of the SLEQ that was then used in a larger study. Future research is needed to see if this model would fit other samples in different elementary schools and in secondary schools both in the USA and in other countries. This revised version was published online in July 2006 with corrections to the Cover Date.     

Measurement Invariance in Confirmatory Factor Analysis: An Illustration Using IQ Test Performance of Minorities

Measurement invariance with respect to groups is an essential aspect of the fair use of scores of intelligence tests and other psychological measurements. It is widely believed that equal factor loadings are sufficient to establish measurement invariance in confirmatory factor analysis. Here, it is shown why establishing measurement invariance with confirmatory factor analysis requires a statistical test of the equality over groups of measurement intercepts. Without this essential test, measurement bias may be overlooked. A re-analysis of a study by Te Nijenhuis, Tolboom, Resing, and Bleichrodt (2004) on ethnic differences on the RAKIT IQ test illustrates that ignoring intercept differences may lead to the conclusion that bias of IQ tests with respect to minorities is small, while in reality bias is quite severe.     

Measurement invariance and validity of a brief questionnaire on reading motivation in elementary students

In order to initiate more research on the role of reading motivation during the initial stages of learning to comprehend texts, we developed the Reading Motivation Questionnaire for Elementary Students (RMQ‐E). The sample comprised 1497 elementary students in Grades 1–3. By means of exploratory and confirmatory factor analyses, three factors were determined: Curiosity, involvement and competition. The three‐factor structure of the RMQ‐E was found to be invariant across grade levels (scalar invariance) and across female and male students (strict invariance). As was anticipated, students in higher grades and male students were lower in curiosity and involvement than students in lower grades and female students. Whereas competitive reading motivation did not differ across grade levels, it was higher for boys than for girls. Moreover, the contributions of involvement and competition to reading amount and reading competence were in accordance with the hypotheses. The predictive validity of curiosity, however, was not confirmed.