Conceptualisation and development of the physics related personal epistemology questionnaire (PPEQ)

This study was designed to develop an assessment tool to measure high school students’ personal epistemology (PE) in the physics domain by using validation processes. Based on theoretical foundations of PE, the PPEQ was conceptualised on six hypothetical dimensions: structure of knowledge [SK], justification of knowledge and knowing [JK], changeability of knowledge [CK], equations in physics [EQ], quick learning [QL], and source of knowledge [source]. In total, 42-items were developed deductively for these dimensions, and a panel of experts assessed the content validity of the instrument. Subsequently, factor analyses were performed to obtain construct-related evidence. First, exploratory factor analysis (EFA) was performed on the data collected from 362 ninth graders. EFA yielded a six-factor solution which excluded the EQ and divided SK into two dimensions: coherent SK and hierarchical SK. Second, confirmatory factor analysis (CFA) was conducted with the remaining 27-items on data collected from a new sample of 350 ninth graders. The CFA results confirmed that the PPEQ assesses six factors emerged from EFA. Internal consistency of the PPEQ was also found to be very high with Cronbach’s α reliability coefficient of .92, and the coefficients of dimensions ranged from .71 to .83.     

Self-regulation in three types of online interaction: a scale development

The purpose of this study was to develop a scale with which to examine students’ self-regulation (SR) in three types of online interaction. Using scale development steps, we constructed the online self-regulation questionnaire (OSRQ), a self-report survey. A total of 799 online students participated in the study. Data from 400 randomly selected participants were used for exploratory factor analysis (EFA), and data of the remaining 399 participants were used for confirmatory factor analysis (CFA). The EFA yielded three factors as hypothesized: SR in interaction between student and content, SR in interaction between student and student and SR in interaction between student and teacher. The CFA demonstrated that the factor structures appearing in the EFA were also observed with different participants. In addition, we found convergent validity in the OSRQ. The study contributes to understanding SR in online learning settings.     

Investigating the quality of the school technology needs assessment (STNA) 3.0: A validity and reliability study

Schools and districts should use a well-designed needs assessment to inform important decisions about a range of technology program areas. Presently, there is a lack of valid and reliable instruments available and accessible to schools to effectively assess their educational needs to better design and evaluate their projects and initiatives. The School Technology Needs Assessment (STNA) is a free, user-friendly online survey tool that meets this need for planning and formative evaluation of technology projects in educational settings. This study used existing data from a robust sample (n = 1918) of educators from across North Carolina to examine the reliability and validity of STNA. A collective review of study results including the literature review, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and internal consistently reliability analysis indicated that STNA was a high-quality instrument.     

青少年个人成长主动性问卷的初步编制与检验

Nine hundred and forty-four effective participants from middle schools are enrolled in this research and divided into two subgroups: sample 1 (N=472) is used in the exploratory factor analysis (EFA), and sample 2 (N=472) in the certified factor analysis (CFA). Results show that an Adolescents’ Personal Growth Initiative Scale contains 24 items in 5 factors, with expected indices of reliability and validity.     

大学生求职资源动员行为的结构与测量——基于北京高校毕业生的实证研究

本文依据北京13所不同层级高校1341名毕业生的实证调研数据,运用主成分分析法与结构方程模型比较,从资源动员视角分析了大学生求职行为的结构特征.研究发现,大学生求职行为的结构由四因素构成;中国大学生动员所属高校就业资源,是其重要的求职行为特征.     

Exploratory Structural Equation Modeling,Integrating CFA and EFA: Application to Students' Evaluations of University Teaching

This study is a methodological-substantive synergy, demonstrating the power and flexibility of exploratory structural equation modeling (ESEM) methods that integrate confirmatory and exploratory factor analyses (CFA and EFA), as applied to substantively important questions based on multidimentional students' evaluations of university teaching (SETs). For these data, there is a well established ESEM structure but typical CFA models do not fit the data and substantially inflate correlations among the nine SET factors (median rs = .34 for ESEM, .72 for CFA) in a way that undermines discriminant validity and usefulness as diagnostic feedback. A 13-model taxonomy of ESEM measurement invariance is proposed, showing complete invariance (factor loadings, factor correlations, item uniquenesses, item intercepts, latent means) over multiple groups based on the SETs collected in the first and second halves of a 13-year period. Fully latent ESEM growth models that unconfounded measurement error from communality showed almost no linear or quadratic effects over this 13-year period. Latent multiple indicators multiple causes models showed that relations with background variables (workload/difficulty, class size, prior subject interest, expected grades) were small in size and varied systematically for different ESEM SET factors, supporting their discriminant validity and a construct validity interpretation of the relations. A new approach to higher order ESEM was demonstrated, but was not fully appropriate for these data. Based on ESEM methodology, substantively important questions were addressed that could not be appropriately addressed with a traditional CFA approach.     

Cultural,social, and economic capital constructs in international assessments: an evaluation using exploratory structural equation modeling

The article employs exploratory structural equation modeling (ESEM) to evaluate constructs of economic, cultural, and social capital in international large-scale assessment (LSA) data from the Progress in International Reading Literacy Study (PIRLS) 2006 and the Programme for International Student Assessment (PISA) 2009. ESEM integrates the theory-generating approach of exploratory factor analysis (EFA) and theory-testing approach of confirmatory factor analysis (CFA). It relaxes the zero-loading restriction in CFA, allowing items to load on different factors simultaneously, and it provides measurement invariance tests across countries not available in EFA. A main criticism of international LSA studies is the extended use of indicators poorly grounded in theory, like socioeconomic status, that prevent the study of mechanisms underlying associations with student outcomes. This article contributes to addressing this criticism by providing statistical criteria to evaluate the fit of well-defined sociological constructs with the empirical data.     

《走向优质——中国幼儿园教育质量评价标准》的测量学属性分析——基于我国100所幼儿园与1670名儿童的测评数据

幼儿园教育质量评价标准的编制具有重要的理论与实践意义。中国学前教育研究会组建了一个包含了来自国内6所高校的专家的跨学科研究团队,遵循严谨的测量学研制程序,经过两年扎实的研究,编制了《走向优质——中国幼儿园教育质量评价标准》(简称《优质标准》)。为验证该标准的有效性,课题组采用分层抽样的方法,在位于我国不同区域的5个省区抽取了不同性质与等级的城乡幼儿园共计100所,采用《优质标准》进行质量评价,同时从300个样本班级中随机抽取了1670名儿童(男女各半),对其语言、数学认知、情感社会性发展水平进行测评。基于幼儿园教育质量和儿童发展水平测评数据,课题组对《优质标准》进行了信效度分析。结果显示,《优质标准》总体及其各领域内部一致性均达到较高水平;绝大部分项目具有良好的区分度;《优质标准》具有良好的结构效度,包含两个潜在的质量因子,分别是课程教学与学习环境、管理支持与师资保障;幼儿园教育质量评价结果与儿童发展水平测评结果总体上呈显著相关,不同质量领域与儿童不同发展领域之间的相关性存在一定差异。综合以上证据表明,《优质标准》是适用于我国幼儿园教育情境的有效评估工具,其评价结果是可靠、可信的。     

The Revised Two-Factor Study Process Questionnaire (R-SPQ-2F): Exploratory and confirmatory factor analyses at item level

The underlying structure of the Revised Two factor version of the Study Process Questionnaire (R-SPQ-2F), a 20-item instrument for the evaluation of students’ approaches to learning (SAL), was examined at item level using two independent groups of undergraduate students enrolled in the first (n=314) and last (n=522) years of their studies. The methods used were (a) Exploratory factor analysis (EFA) assisted by rigorous procedures such as Velicer’s MAP test, parallel analysis and the Schmid Leiman solution with the first sample; and (b) confirmatory factor analysis (CFA) with the second sample. The results of EFA indicated that the latent structure of the R-SPQ-2F is best described by two factors and the results of CFA suggested that out of four a priori structural models, the best fit was achieved by a simple first-order two-factor model. Taken together, these results seemed to converge, suggesting (a) that SAL might be defined as a co-variation between a motive and its intended strategy, these not necessarily being divided into separate first-order factors (subscales), and (b) that the underlying structure of the R-SPQ-2F is apparently non-hierarchical, being best described by a parsimonious first-order two-factor model in which Deep and Surface learning approaches are each measured by their ten corresponding items.     

中学生班级集体效能感量表的初步编制

根据研究和访谈结果,编制中学生班级集体效能感初测问卷,对194名中学生进行测试,结果用于探索性分析。对1773名中学生的测试结果用于验证性因素分析。159人完成重测,359人同时完成校标检验。探索性因素分析获得获得四个因素：合作、能力、预期和努力,解释了总变异的52．62％;验证性因素分析显示,四因素模型的各项参数达到可接受的水平;量表的内在一致性α系数和分半信度分别为0．878和0．859,重测信度为0．703,采用高峰强等人的集体效能信念量表作为班级集体效能感测量的效标,结果相关系数为0．541。因此,该量表具有较好的信效度,可应用于中学生群体。     

高校毕业生就业公正感问卷的编制

该研究在相关研究与开放式问卷调查的基础上,编制了高校毕业生就业公正感初始问卷.对186名毕业生第一次施测,221名毕业生第二次施测,均进行探索性因素分析,然后对872名高校毕业生正式施测的结果进行验证性因素分析.结果表明,高校毕业生就业公正感问卷包括八个维度：就业机会、信息发布、录用程序、招录依据、所受待遇、潜在干扰、自我展示、综合评价.验证性因素分析的结果表明,各指标均达到可接受水平,八维度模型拟合度较好.问卷具有良好的信效度,可以作为高校毕业生就业公正感的测量工具。     

To assess or not to assess holistic competencies – Student perspectives in Hong Kong

Despite the importance of holistic competencies, universities have not yet been able to successfully embrace holistic competencies (HC) as part of students’ assessment in their curricula. HC are often part of a hidden curriculum – they tend to be assessed together with academic knowledge, and not assessed explicitly. Based in Hong Kong, this research developed a questionnaire to study university students' perceptions of the formal assessment of HC and how students’ perceptions could be bound to the sociocultural context. Through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), two dimensions of perceptions were derived from the responses of 300 respondents: (a) students’ acceptance of formal HC assessment, and (b) their preference for quantitative scoring in the formal assessment of holistic competencies. Findings reveal that students’ perceptions of formal HC assessment were generally considered positive. However, the majority of students appeared to favour pass/fail grading compared to quantitative assessment. As assessment drives learning, this study reinforces the need for universities to incorporate students’ perspectives in policies regarding HC assessment and highlights the role of culture in influencing student perceptions.     

The content and construct development of the PACT survey

The authors created a psychometrically sound instrument that could be used to examine K-12 teachers’ perceptions of arts integration. Ten content experts reviewed the initial survey items for construct-fit and readability. Scores from 354 K-12 teachers were examined for the survey’s initial factor structure through exploratory factor analysis (EFA). Then, scores from 1,072 K-12 teachers were analyzed to complete a confirmatory factor analysis (CFA) of the Putting the Arts and the Classroom Together (PACT) data. EFA results indicated a three-factor structure (i.e., value, willingness, and barriers), and CFA results produced model indices showing good fit of the three-factor model. For purposes of examining teachers’ practices in integrating the arts, establishing teacher receptiveness to arts integrated curriculum is important in designing and implementing any professional development program. This survey could be used to establish baseline participant perceptions for staging targeted professional development to support educators in creating appropriate arts integrated curricula.     

What is technology? Development and evaluation of a simple instrument for measuring children's conceptions of technology

Technology is important to all aspects of our lives, so helping students develop an accurate understanding of technology should be an educational goal at the K-12 level. Assessments are important tools in reaching this goal. We developed an instrument to measure the technology conceptions of children ages 8–11, the ‘What is Technology’ (WT) instrument. We gathered evidence for the validity of using the WT instrument to measure children's conceptions of technology, including changes due to an intervention, and we describe those changes. We used exploratory factor analysis (EFA) to establish scales, and confirmatory factor analysis (CFA) with new samples of subjects to confirm our hypothesised model of children's conceptions. For further evidence, we analyzed and coded the written responses children gave to open-ended questions asking them to explain their understanding of technology, then calculated correlations between these codes and the CFA-confirmed scale measures. We found that children tend to think of technology as artifacts that are powered by electrical energy; however, after instruction most children's conceptions become more consistent with definitions given in educational standards. The instrument is shown to be valid and reliable for its intended use, to assess preadolescent children's conceptions of technology, and evaluate the impact of an intervention.     

Psychometric reevaluation of the scientific attitude inventory‐revised (SAI‐II)

The central purposes of this study were to review the development and evolution of the Scientific Attitude Inventory (SAI) and then reevaluate the psychometric properties of the revised form of the SAI, the Scientific Attitude Inventory II (SAI‐II). The SAI‐II was administered to a convenience sample of 543 middle and high school students from five teachers in four schools in four school districts in San Antonio, Texas, at the beginning of the 2004–2005 school year. Confirmatory factor analysis on the full data set failed to support the existence of a 12‐factor structure (as proposed by the scale developers) or a one‐factor structure. The data were then randomly divided into exploratory [exploratory factor analysis (EFA)] validation and confirmatory [confirmatory factor analysis (CFA)] cross‐validation sets. Exploratory and confirmatory models yielded a three‐factor solution that did not fit the data well [χ² (321) = 646, p < .001; RMSEA = .061 (.90 CI = .054–.068); and CFI = .81]. The three factors were labeled “Science is About Understanding and Explaining” (13 items), “Science is Rigid” (6 items), and “I Want to Be a Scientist” (8 items). The α‐coefficients for these three factors ranged from 0.59 to 0.85. Whether these identified subscales are valid will require independent investigation. In this sample, and consistent with prior publications, the SAI‐II in its current form did not have satisfactory psychometric properties and cannot be recommended for further use. © 2008 Wiley Periodicals, Inc. J Res Sci Teach 45: 600–616, 2008     

Validation and profile of Chinese pre-service teachers’ technological pedagogical content knowledge scale

Researchers state that teachers in different contexts reported different technological pedagogical content knowledge (TPACK). This phenomenon may partially be explained by cultural differences. Based on this consideration, the development and validation of the Chinese pre-service teachers’ technological pedagogical content knowledge (CTPCK) scale is described in this article. The sample was split into two subsamples on random basis (n1 = 229, n2 = 207) for the purpose of conducting (1) exploratory factor analysis (EFA) and (2) confirmatory factor analysis (CFA), respectively. After the EFA, the CTPCK scale excluded six items and included eight factors with 42 items. Reliability and correlations were discussed. The findings revealed that the CTPCK scale was a valid and reliable instrument for measuring the TPACK of Chinese pre-service teachers’ knowledge with or without linking educational technology.     

The development of a questionnaire for predicting online learning achievement

The study reported here concerns the development and predictive validation of an instrument to assess the achievement outcomes of DE/online learning success. A 38‐item questionnaire was developed and administered to 167 students who were about to embark on an online course. Factor analysis indicated a four‐factor solution, interpreted as “general beliefs about DE,” “confidence in prerequisite skills,” “self‐direction and initiative” and “desire for interaction.” Using multiple regression we found that two of these factors predicted achievement performance (i.e., Cumulative Course Grade). Comparisons of pretest and posttest administrations of the questionnaire revealed that some changes in opinion occurred between the beginning and the end of the course. Also, categories of demographic characteristics were compared on the four factors. The overall results suggest that this instrument has some predictive validity in terms of achievement, but that Cumulative Grade Point Average (i.e., the university's record of overall achievement) is a much better predictor.     

Confirmatory factor analyses of Chinese students' evaluations of university teaching

Chinese University of Hong Kong students (N = 844) selected a “good” and a “poor” teacher, and rated each using a Chinese translation of the Students' Evaluations of Educational Quality (SEEQ) instrument. Multigroup confirmatory factor analysis (CFA) models, based on a 3 × 2 design, were constructed to test the invariance of the SEEQ factor structure across 3 discipline groups (a between‐group comparison of ratings by students in arts, social sciences, and education; in business administration; and in engineering, medicine, and science) and across ratings of good and poor teachers (via within‐subjects comparison). The selected model imposed between‐group invariance constraints on factor loadings, factor correlations, and factor variances across the 3 discipline groups and within‐subjects invariance constraints on factor loadings across ratings of good and poor teachers. The results support the use of SEEQ in this Chinese setting, demonstrating the generality of North American research findings and the usefulness of CFA in this research area.     

On the Exploratory Road to Unraveling Factor Loading Non-invariance: A New Multigroup Rotation Approach

Multigroup exploratory factor analysis (EFA) has gained popularity to address measurement invariance for two reasons. Firstly, repeatedly respecifying confirmatory factor analysis (CFA) models strongly capitalizes on chance and using EFA as a precursor works better. Secondly, the fixed zero loadings of CFA are often too restrictive. In multigroup EFA, factor loading invariance is rejected if the fit decreases significantly when fixing the loadings to be equal across groups. To locate the precise factor loading non-invariances by means of hypothesis testing, the factors’ rotational freedom needs to be resolved per group. In the literature, a solution exists for identifying optimal rotations for one group or invariant loadings across groups. Building on this, we present multigroup factor rotation (MGFR) for identifying loading non-invariances. Specifically, MGFR rotates group-specific loadings both to simple structure and between-group agreement, while disentangling loading differences from differences in the structural model (i.e., factor (co)variances).     

Vocational Education Students’ Generic Working Life Competencies: Developing a Self-Assessment Instrument

The current study reports on the process of developing a self-assessment instrument for vocational education students’ generic working life competencies. The instrument was developed based on a competence framework and in close collaboration with several vocational education teachers and intermediary organisations offering various human resource services. A first version of the questionnaire was presented to 26 students and 5 recent graduates who were asked to comment on the items. The pilot version of the questionnaire was completed by 826 students. Half of the data were used to explore the structure of the questionnaire (n?=?413) and the other half were used to confirm the structure (n?=?413). The results showed that 8 factors could be distinguished. Further analysis reduced this to 7 usable factors: empathy, listening, assertiveness, professional attitude, problem solving, cooperation ability and, planning and prioritising. The revised questionnaire containing 44 items was tested a second time to determine the stability and measurement invariance of the instrument. In total, 456 students from the first sample completed the questionnaire. The structure was confirmed and measurement invariance across students with and without working experience was established.