Pedagogical and Social School Climate: Psychometric Evaluation and Validation of the Student Edition of PESOC

Previous studies indicate that school climate is important for student health and academic achievement. This study concerns the validity and reliability of the student edition a Swedish instrument for measuring pedagogical and social school climate (PESOC). Data were collected from 5,745 students at 97 Swedish secondary schools. Multilevel confirmatory factor analyses were conducted, and multilevel composite reliability estimates, as well as correlations with school-level achievement indicators, were calculated. The results supported an 8-factor structure at the student level and 1 general factor at the school level. Factor loadings and composite reliability estimates were acceptable at both levels. The school-level factor was moderately and positively correlated with school-level academic achievement. The student PESOC is a promising instrument for studying school climate.     

Literacy as a Unidimensional Multilevel Construct: Validation,Sources of Influence,and Implications in a Longitudinal Study in Grades 1 to 4

This study examined the extent to which literacy is a unitary construct, the differences between literacy and general language competence, and the relative roles of teachers and students in predicting literacy outcomes. Much of past research failed to make a distinction between variability in outcomes for individual students and variability for outcomes in the classrooms students share (i.e., the classroom level). Utilizing data from 1,342 students in 127 classrooms in Grades 1 to 4 in 17 high-poverty schools, confirmatory factor models were fit with single- and two-factor structures at both student and classroom levels. Results support a unitary literacy factor for reading and spelling, with the role of phonological awareness as an indicator of literacy declining across the grades. Writing was the least related to the literacy factor but the most impacted by teacher effects. Language competence was distinct at the student level but perfectly correlated with literacy at the classroom level. Implications for instruction and assessment of reading comprehension are discussed.     

Psychometric Properties of an Instrument to Measure Social and Pedagogical School Climate Among Teachers (PESOC)

This study investigated the psychometric properties of a teacher-reported version of a Swedish school climate instrument called the Pedagogical and Social Climate (PESOC), which consists of 95 items covering cultural, structural and social factors. A sample of 348 teachers from 19 Swedish secondary schools was used. Multilevel confirmatory factor analysis conducted within a structural equation modelling framework indicated that the PESOC had a two-factor structure at the teacher level and a one-factor at the school level. The PESOC’s convergent validity was supported by the school-level correlations between PESOC and another established instrument (i.e., the Team Climate Inventory). Further validation studies of PESOC are needed with larger, more representative samples, and with information on important outcomes such as student achievement and wellbeing.     

Evaluating the perception of disability and the inclusive education of teachers: the Italian validation of the Sacie-R (Sentiments,Attitudes, and Concerns about Inclusive Education – Revised Scale)

This study aimed to verify the factorial structure, internal validity, reliability, and criterion validity of The Sentiments, Attitudes and Concerns about Inclusive Education Revised Scale (SACIE-R) in an Italian sample. The SACIE-R is a useful tool to measure three aspects of teachers’ perceptions about inclusion, which are Sentiments, Attitudes and Concerns, providing valuable information to address the needs of teachers working with special students. A measure of this construct is currently not available in Italian, and a validation process is required. In order to provide an initial psychometric evaluation of the Italian adaptation, the SACIE-R was translated for use in Italian-speaking populations. Four hundred Italian teachers enrolled in educational teaching courses completed the SACIE-R. Confirmatory factor analyses demonstrated a good fit to the hypothesised 3-factor model, and adequate internal consistency for the scale was evidenced. A confirmatory factor analysis showed the same factor structural validity as the original instrument (although some items in the original version were deleted). Furthermore, taken together, the results showed a good level of reliability, and convergent and discriminant validity. Finally, the Italian version of the SACIE-R has good psychometric characteristics that can make it a useful measure of teacher attitude toward inclusion in the Italian educational context.     

An international validation of the engaged teacher scale

The purpose of this study was to test and validate the Engaged Teacher Scale (ETS) in a Turkish context (ETS-TR). In order to test the construct validity of the ETS, data were collected from 388 teachers in two northeast cities of Turkey. First-order confirmatory factor analysis results supported the 16-item and four-factor model of ETS while second-order confirmatory factor analysis suggested that a single factor was also appropriate for representing teacher engagement. Additionally, four multiple linear regression analyses were conducted to provide further validation evidence. Results showed that subscales of the ETS-TR were found to be positively correlated with teacher self-efficacy. Given our evidence of validity and reliability, we recommend researchers interested in measuring the engagement of Turkish teachers to consider using the ETS-TR. The adaptation of ETS into Turkish also provides a measure for use when conducting research examining cultural comparisons between english-speaking and Turkish teachers.     

Load reduction instruction in mathematics and English classrooms: A multilevel study of student and teacher reports

Load reduction instruction (LRI) is a practical instructional framework aimed at managing the cognitive demands experienced by students as they learn. LRI comprises five key instructional principles: (1) difficulty reduction during initial learning, as appropriate to students’ prior learning, (2) support and scaffolding, (3) structured practice, (4) feedback-feedforward, and (5) guided independent application. The present investigation explored student- and teacher-reports of LRI in both mathematics and English. The Load Reduction Instruction Scale – Short was administered to 1773 students and their teachers in 93 mathematics classrooms and 94 English classrooms. Multilevel (Level 1 student; Level 2 classroom) confirmatory factor analysis (MCFA) supported a Level 1 student LRI factor and a Level 2 class-average LRI factor in each of mathematics and English. However, two LRI factors emerged for teachers in each of mathematics and English: one factor related to Principle 1 (difficulty reduction) and one factor related to Principles 2–5 (scaffolding to autonomy). Follow-up multilevel structural equation modeling (MSEM) revealed that teachers adjusted their application of LRI Principle 1 (but not Principles 2–5) as a function of class-average prior learning (lower prior learning was associated with greater application of Principle 1). MCFA also showed that correlations between student- and teacher-reported LRI were low. Follow-up MSEM revealed that student- and teacher-reports of LRI uniquely predicted students’ effort and achievement—suggesting that students and teachers provide distinct insights into instruction (hence the low correlation between them) and highlighting the important role of both informants in capturing a comprehensive perspective on instruction in the classroom.     

Construct validation of the Behavior and Instructional Management Scale

Beliefs related to classroom management vary among teachers and play an important role in classrooms. Despite the importance of this construct, valid measures have proven difficult to develop. This study evaluated the psychometric properties of the Behavior and Instructional Management Scale (BIMS), a short but valid measure of teachers' approaches to behavioral and instructional management. Results revealed a two-factor solution that possessed a good model fit, with large estimated factor loadings using confirmatory factor analysis. Evidence of validity was obtained with the Ohio State Teacher Efficacy Scale. Internal consistency for both subscales was adequate. Implications for future research are discussed.     

Development and psychometric evidence of an engagement instrument in the pre-service teacher education context

The present study details the development and validation of an engagement instrument in the pre-service teacher education context. Exploratory and Confirmatory Factor Analyses were used to obtain the factor structures performed on a sample of 196 and 212 pre-service teachers respectively. Exploratory factor analysis demonstrated four empirically interpretable and reliable dimensions: academic engagement, peer engagement, intellectual engagement, and online engagement. The confirmatory factor analysis results also supported the correlational four-dimensional model with adequate goodness-of-fit indices. Results also depicted convergent and discriminant validity of the instrument. Future research involved in the iterative process of instrument development is discussed.     

Reliability and validity of a Turkish version of the DELES

The primary aim of this study was to examine the reliability and validity of a Turkish version of the Distance Education Learning Environment Survey (DELES) in post-secondary distance education. The second aim was to investigate empirically the conceptualisation of the distance education learning environment as a singular latent construct, within a second-order factorial structure that consists of six first-order factors. The DELES is a 34-item instrument for assessing social-psychological perceptions among distance education students. The tested hypotheses concerned key antecedents and consequences of the factorial structure of DELES in the Turkish context using confirmatory factor analyses. Principal component analysis of the Turkish data revealed six main factors of interest, which corresponded to the hypothesised constructs in the original DELES. The data suggested good reliability and validity for the Turkish DELES (DELES-TR) relative to the original version. Further, second-order analyses were conducted with two complementary samples to demonstrate the validity of six dimensions of the DELES-TR and their contributions to an overall factorial structure of social-psychological learning environment in distance education. Initial support for the second-order factorial validity of DELES-TR was found. Results indicate that a dimensional model of DELES-TR with the six first-order factors provided a superior fit to the data than first-order DELES-TR with six factors. Additionally, an investigation was conducted regarding associations between the DELES-TR’s psychosocial scales and a student satisfaction scale. This study is unique in its focus on the learning environment in distance education in the Turkish setting.     

Teaching skills of student teachers: Calibration of an evaluation instrument and its value in predicting student academic engagement

Student teachers are expected to develop their teaching skills sooner and more rapidly. However, a sound evaluation instrument that can be used to diagnose and monitor the skilfulness level to aid formative assessment of student teachers is still limited. This article is aimed to calibrate and validate a teaching skill evaluation instrument for use in secondary education. A total of 264 student teachers in the Netherlands participated in the study. Rasch and multilevel analyses were used. Results suggest that the evaluation instrument meets the restrictive assumptions of the Rasch model and has predictive value for academic engagement. This adds validation evidence and justifies the calibration of the evaluation instrument to be used for monitoring the development of teacher's teaching skills.     

Survey of classroom use of representations: development,field test and multilevel analysis

Because of the multimodal nature of learning, doing and reporting science, it is important that students learn how to interpret, construct, relate and translate scientific representations or, in other words, to develop representational competence. Explicit instruction about multimodal representations is needed to foster students’ representational competence in the classroom. However, only a handful studies have surveyed how representations are actually used in science classes. This might be because of the fact that economical instruments for assessing the use of representations in classrooms are not available. To bridge that gap, an instrument was developed, field-tested in biology classes with 175 and 931 students, respectively, and analysed using exploratory and (multilevel) confirmatory factor analyses. Results supported an instrument with six scales and 21 items at the individual and classroom levels covering the following dimensions: (1) interpretation of visual representations, (2) construction of visual representations, (3) use of scientific texts (verbal representations), (4) use of symbolic representations, (5) number of terms used in class, and (6) the extent to which active social construction of knowledge is possible in the class. The scales showed satisfactory discriminant validity and reliability at each level. Further applications of this instrument for researchers and teachers are discussed.     

The Group Reactions of Over 1300 College,University, and Teachers College Instructors to Sixty-Three Curriculum Proposals Related to the Education of Teachers

This study explored the validity of the Utrecht Work Engagement Scale in a sample of 853 practicing teachers from Australia, Canada, China (Hong Kong), Indonesia, and Oman. The authors used multigroup confirmatory factor analysis to test the factor structure and measurement invariance across settings, after which they examined the relationships between work engagement, workplace well-being (job satisfaction and quitting intention), and contextual variables (socioeconomic status, experience, and gender). The 1-factor version of the Utrecht Work Engagement Scale was deemed preferable to the 3-factor version and showed acceptable fit to the cross-national data. The 1-factor Utrecht Work Engagement Scale showed good internal consistency and similar relationships with workplace well-being and contextual variables across settings. The Utrecht Work Engagement Scale was invariant within broadly construed Western and non-Western groups but not across Western and non-Western groups. The authors concluded that the Utrecht Work Engagement Scale needs further development before its use can be supported in further cross-cultural research.     

The effects of classroom and school environments on student engagement: the case of high school students in Abu Dhabi public schools

Drawing on data from a merged data set from a student survey and a parent survey that were conducted in the Emirate of Abu Dhabi in 2013, this article uses a multilevel framework to investigate the effects of individual characteristics and the classroom and school environments on high school students’ school engagement in a modernising education system that is different from Western ones. The results of the three-level model revealed that while students’ attributes remained strong predictors of their school engagement, the social and organisational environment of classrooms and school also greatly shaped the extent to which students emotionally and cognitively engaged with their school and learning. This study provided evidence to support the interactive nature of the impact of multilevel environments on student engagement. The policy and research implications were also discussed in the empirical context of Abu Dhabi.     

Asking students their opinions of the learning environment: an empirical analysis of elementary classroom climate

There is currently a lack of research into classroom climate as perceived by the students themselves. This article presents a new classroom climate evaluation instrument which is designed to gauge student perceptions of their own level of engagement in academic activities, their relationships with peers, and the level of support they feel from their teachers. The Classroom Climate Index was pilot tested with a group of 105 elementary students aged 8–12. The pilot data were factor analysed, and the instrument was refined accordingly. Following the pilot test, the refined instrument was next administered to a larger sample of 2340 elementary students. Data from the comprehensive study were subjected to factor and reliability analyses, the results of which support the use of the Classroom Climate Index in measuring student engagement, supportive teacher behaviour, and collegial support from the student perspective.     

The Construct Validity of Student Engagement: A Confirmatory Factor Analysis Approach

As institutions seek to promote student engagement on campus, the National Survey of Student Engagement (NSSE) is increasingly being used to chart progress and compare results using the Five Benchmark Scores. While recent research has begun to decompose the five benchmarks in a variety of ways; few research studies have sought to explore the underlying structure of these five benchmarks, their interdependence, and the extent to which the items do reflect those five dimensions. This study begins to address the instrument’s construct validity by submitting a single, first-time freshman cohort’s NSSE responses to a confirmatory factor analysis, and proposes as an alternative, eight “dimensions” of student engagement that fit this set of data slightly better and in a more useful way. Results have practical implications for institutions utilizing NSSE, but also contain conceptual implications pertaining to the application of these benchmarks.     

Development and validation of two scales to measure elaboration and behaviors associated with stewardship in children

This investigation examines the development of two scales that measure elaboration and behaviors associated with stewardship in children. The scales were developed using confirmatory factor analysis to investigate their construct validity, reliability, and psychometric properties. Results suggest that a second-order factor model structure provides the best fit. This model produced: (1) a stewardship elaboration scale measuring interest and cognitive engagement in stewardship issues, and (2) a stewardship behavior scale measuring in-park, community, and home behaviors. These scales will be useful for evaluating environmental educational programs focused on environmental and park stewardship. The scales may also help researchers assess whether environmental education results in participants elaborating on persuasive messaging, thereby increasing the likelihood that behavioral intentions leading to behavior change will occur.     

Psychometry of the Teacher's Sense of Efficacy Scale in Spanish Teachers' Education

Our objective was to test the factorial validity of the different versions (TSES-24, TSES-12, and TSES-11) of the Teacher's Sense of Efficacy Scale in Spanish preservice teachers'. The participants were 358 preservice teachers. The confirmatory factor analysis showed inadequate fit indexes for both TSES-24 and TSES-12 but adequate fit indexes for TSES-11, which was invariant across gender. The reliability analysis showed appropriate values. The global factor for teachers' sense of efficacy and the three factors that constitute it positively predicted intrinsic motivation and life satisfaction. The global factor for sense of efficacy and the classroom management and student engagement factor negatively predicted amotivation. The results support the use of the TSES-11 three-factor model, rather than TSES-24 or TSES-12, for Spanish preservice teachers.     

Measuring characteristics of teacher professional development

The primary purpose of the study was to create and psychometrically test an instrument which measured teachers’ perceptions of characteristics of professional development. The sample consisted of elementary teachers from five school districts in Washington State participating in a district improvement initiative. Results of exploratory factor analysis resulted in a five-component solution which provided evidence of construct validity. All components/subscales had moderate to strong internal consistency as measured by Cronbach’s alpha. The secondary purpose was to determine if characteristics of professional development predicted teachers’ use of new knowledge and skills, and student learning outcomes, thus contributing to the predictive validity of the instrument. There was a slight, but significant, correlation between Active Learning in Classroom and teachers’ use of new knowledge and skills, as measured by classroom observation scores. A small negative correlation emerged between Collective Participation and student learning, as measured by student scores on the state’s mathematics assessment. Potential uses of the instrument, called Characteristics of Teacher Professional Development (CTPD), are discussed.     

Assessing Social Competence and Behavior Problems in a Sample of Italian Preschoolers Using the Social Competence and Behavior Evaluation Scale

Research Findings: The main goals of this study were to examine the factor validity of the Social Competence and Behavior Evaluation (SCBE-30) scale using exploratory factor analysis and confirmatory factor analysis and to test factor invariance across gender in a sample of Italian preschool-age children (241 boys, 252 girls). The concurrent validity of the SCBE scale was examined with measures of children's popularity and rejection. Our findings replicated a 3-factor model of the SCBE scale found in other studies with 3 correlated factors of social competence, anger–aggression, and anxiety–withdrawal. Multigroup confirmatory factor analyses provided evidence of configural, metric, and partial scalar invariance of the scale across gender. Popularity was positively related to children's social competence and negatively related to anxiety–withdrawal. Rejection was positively related to children's anger–aggression and anxiety–withdrawal and negatively related to social competence. Practice or Policy: The use of the SCBE scale in the Italian educational setting may help teachers understand children's emotional and social competencies and thereby improve social adjustment in the classroom.