Disentangling Shape from Level Effects in Person-Centered Analyses: An Illustration Based on University Teachers’ Multidimensional Profiles of Effectiveness

This study compares alternative ways of disentangling the effects of level (the tendency for a person to be high, medium, or low across all factors) and shape (the tendency for a person to have a distinct pattern of factors on which they are high, medium, or low) in profile analyses. This issue is particularly relevant to performance appraisals where it is often useful to identify specific strengths and weaknesses over and above a person global performance, but also to person-centered analyses more generally where the observation of qualitative (shape) differences between profiles is often used as justification for the added value of profiles. Substantively, this study illustrates these issues in the identification of profiles of teachers based on multidimensional students’ ratings of their effectiveness, using an archival data set of 31,951 class-average ratings based on the Students’ Evaluations of Educational Quality (SEEQ) instrument collected over a 13-year period. The results show the superiority of a factor mixture operationalization of teaching effectiveness in which a global effectiveness factor was used to control for unnecessary level effects in the profiles.     

Development of a Questionnaire Assessing Teacher Perceived Support for and Attitudes About Social and Emotional Learning

Research Findings: In the past 20 years school districts have increasingly adopted classroom-based social and emotional development programs. The dissemination of these programs, however, has surpassed our understanding of and ability to assess factors that influence program implementation. The present study responded to this gap by developing a questionnaire that focuses on teacher perceptions of implementation support and teacher attitudes about social–emotional learning and by assessing its psychometric properties. One hundred forty-five Baltimore City Head Start preschool teachers completed the questionnaire. Factor analyses suggested 6 underlying constructs, which we termed administrative support, training, competence, program effectiveness, time constraints, and academic priority. Several of these scales predicted teacher reports of program implementation. Practice or Policy: The questionnaire holds significant promise as a tool for assessing readiness and barriers to social and emotional program implementation.     

Antecedents and consequences of teachers’ goal profiles in Australia and Israel

Studies drawing on Butler’s (2012) Goal Orientations for Teaching framework show mastery and relational goals are beneficial, ability-avoidance maladaptive and inconsistent effects for ability-approach. We adopt a person-centered approach to discern the extent to which teachers pursue different goal combinations. In two studies of mid-career primary and secondary teachers (Ns = 476 Australian, 507 Israeli) we assessed via self-report surveys: teachers' achievement goals, initial motivations to teach, school contexts, professional engagement and instructional practices. Latent profile analyses distinguished four goal combinations per sample. In both, mastery and relational goals covaried as ‘task’ goals. Profiles with stronger ability than task goals were maladaptive for teachers and teaching, and in the Australian sample linked with poor initial ‘fallback’ career motivation. Profiles with stronger task than ability goals were most adaptive, and linked to more positive school climate. Implications are outlined for theory and practice.     

DIMENSIONALITY AND MEASUREMENT INVARIANCE OF A SCHOOL READINESS SCREENER BY ETHNICITY AND HOME LANGUAGE

This study examined the dimensionality and measurement invariance of the Kindergarten Student Entrance Profile (KSEP) when used to rate the school readiness of children from different ethnic backgrounds (Latino or White, non‐Latino), as well as from households where a different language was predominant (Spanish or English). Teachers rated the readiness of 9,335 children during the first month of kindergarten in four ethnically diverse, medium‐sized school districts in central California. From the total sample, two overlapping subsamples (S1 and S2) were identified. First, a series of confirmatory factor analyses were conducted with S1 (n = 7,787) to examine the dimensionality and measurement invariance of the KSEP with children who identified as either Latino or White at the time of kindergarten enrollment. Next, the same set of analyses were replicated with S2 (n = 9, 234) to examine whether results held for students from households where the primary language spoken was Spanish or English. Results yielded evidence supporting a two‐factor structure encompassing social‐emotional and cognitive dimensions of children's readiness. In addition, results showed the KSEP exhibited measurement invariance across student ethnicities (Latino/White) and home languages (Spanish/English). The results of this study provide psychometric evidence that is particularly important for a universal school readiness screener.     

Applicability of the Classroom Assessment Scoring System in Chinese Preschools Based on Psychometric Evidence

Research Findings: This study examined the applicability of the Classroom Assessment Scoring System (CLASS) Pre-K (Pianta, La Paro, & Hamre, 2008) and its underpinning framework of teaching through interactions in typical Chinese kindergarten classrooms. A sample of 180 kindergarten classrooms in China was selected, and the CLASS was used to rate the classroom teaching interactions. Multiple analytical approaches were applied to examine how the CLASS functioned in the Chinese cultural and social context, including score pattern comparisons with samples from the United States and some international contexts, measurement reliability analysis for different measurement error sources (e.g., interrater reliability and stability across time), and confirmatory factor analysis for several competing models. The findings from these different analyses indicated that the CLASS exhibited the same or very similar structures as in the U.S. sample, and the ratings of the CLASS showed other desirable psychometric characteristics. Practice or Policy: The evidence from the multiple analytical approaches provide strong support for the applicability of the CLASS in the Chinese context. The findings also reveal a critical need for Chinese teachers to improve the quality of instructional support in order to promote children’s optimal development. Limitations are noted, and future research directions are discussed.     

Mapping the classroom emotional environment

Harvey and Evans (2003) have proposed that teachers’ emotional skills, as required in the classroom, can be organized into a five-dimensional model. Further research is necessary to validate this model and evaluate the importance of each dimension of teacher emotion competence for educational practice. Using a statistical method for mapping psychological constructs, teachers’ emotional practices were transformed into a ‘map’ with three spatial dimensions, which was supported by comparative validity checks. The nature of emotional practice was further investigated by administering a Q-Sort of emotional practice items to 100 teachers. Ten highly applicable clusters of emotional behavior across six profile patterns with teachers were identified. Authors’ model and the applicability of emotions were generally supported.     

Making Sense of What Students Know: Examining the Referents, Relationships and Modes Students Displayed in Response to a Decimal Task

Written tasks that request an explanation can offer detailed information concerning students' knowledge to the classroom teacher. The examination of students' written responses for the mathematical systems that are employed (referents), the connections that are displayed (relationships) and the communication methods that are utilized (modes) provides evidence that suggests how the student is making sense of a problem. The purpose of this article is to describe the constructs of referents, relationships and modes and to illustrate how these constructs may be reflected in students' written responses to a decimal task that requests an explanation. This article will also examine sets of responses from two classrooms using the proposed framework to illustrate the type of information that teachers may acquire through the application of this framework. This revised version was published online in July 2006 with corrections to the Cover Date.     

Construct validity of the universal nonverbal intelligence test (UNIT)

Data from 104 public school children were analyzed to determine the psychometric and conceptual soundness of a new model-based measure of intellectual functioning: the Universal Nonverbal Intelligence Test (UNIT; Bracken & McCallum, in press). The UNIT was designed to assess the construct of intelligence for at-risk children who are limited English language proficient. The UNIT and subtests from the Woodcock-Johnson-Revised Tests of Cognitive Ability (WJ-R; Woodcock & Johnson, 1989) were administered in counterbalanced order using standardized procedures. Spearman-Brown reliability coefficients provided evidence of the UNIT'S reliability: Global score coefficients ranged from .86 (Reasoning Quotient) to .93 (Symbolic Quotient). Confirmatory factor analyses of the UNIT provided evidence for a two-factor structure. Exploratory factor analyses of the UNIT with the WJ-R yielded support for the UNIT'S measurement of memory and reasoning constructs, although the many subconstructs assessed by the WJ-R and the UNIT preclude a simple interpretation. Correlations among UNIT global scores with age and with composite scores of the WJ-R provide additional evidence of the UNIT'S construct validity.     

Development and initial validation of the institutional child abuse processes and coping inventory among a sample of Irish adult survivors of institutional abuse

ObjectiveTo develop a psychometric instrument to evaluate psychological processes associated with institutional abuse and coping strategies used to deal with such abuse.MethodsAs part of a comprehensive assessment protocol, an inventory containing theoretically derived multi-item rational scales which assessed institutional abuse-related psychological processes and coping strategies were administered to 247 Irish adult survivors of institutional child abuse. Exploratory and confirmatory factor analyses were used to derive 6-factor scales, the reliability and validity of which were assessed.ResultsFactor scales to assess the following constructs were developed (1) traumatization, (2) re-enactment, (3) spiritual disengagement, (4) positive coping, (5) coping by complying, and (6) avoidant coping. There were varying degrees of support for the validity of the scales with most support for the traumatization and re-enactment scales.ConclusionsThe Institutional Child Abuse Processes and Coping Inventory (ICAPCI), particularly its traumatization and re-enactment scales, may be used in future research on adult survivors of institutional child abuse because they are currently the only scales that have been developed with this population to provide reliable and valid assessments of these constructs.Practice implicationsThe ICAPCI may be used, cautiously, to assess adult survivors of institutional child abuse.     

Deconstructing CHC

ABSTRACT

Twenty-five years after the introduction of Carroll’s (1993) Three Stratum (3S) theory of intelligence and McGrew’s (1997) subsequent synthesis of 3S with the extended Gf-Gc / Horn-Cattell theory, the Cattell-Horn-Carroll (CHC) theory represents the prevailing framework by which the structure of human cognitive and intellectual abilities is understood. This commentary reviews, discusses, and supplements the critical examinations of CHC theory appearing in this special issue of Applied Measurement in Education. After recounting the historical foundations and structural framework of CHC, a series of challenges to this theory are compiled including its weak explanatory capacity, its failure to make testable predictions, its enmeshment with a single measurement instrument (i.e., the Woodcock–Johnson), the inconsistent independent replication of its factor structure, and most importantly, its undemonstrated evidential value for identification and intervention with students with specific learning disorders (SLDs). The inherent limitations of any psychological theory developed or tested primarily from factor analyses are enumerated. Solutions are proposed to advance psychological science in areas related to CHC theory.     

Disentangling school leadership and its ties to instructional practices – an empirical comparison of various leadership styles

This paper investigates the direct and indirect ties between various leadership styles, namely, instructional, transformational, transactional, and laissez-faire leadership, and the instructional practices of teachers by applying a structural equation model. For this purpose, we analyzed survey data of n = 3,746 teachers from 126 schools collected by the Hamburg school inspection in Germany between 2012 and 2015. The underlying model is based on Leithwood’s framework for guiding research on leader effects on learning and instruction. First, the results show that a bi-factor model seems to be the best measurement model. Next, it is shown that mediating variables are influenced by a leadership core as well as by different leadership facets. Third, we found that for influencing complex instructional practices like cognitive activation with challenging content, a combination of leadership styles is most promising, while for classroom management instructional leadership is the only and, thus, the primary determinant.     

Tests,Test Scores,and Constructs

In the service of educational accountability, student achievement tests are being used to measure constructs quite unlike those envisioned by test developers. Scores are compared to cut points to create classifications like “proficient”; scores are combined over time to measure growth; student scores are aggregated to measure the effectiveness of teachers, schools, and school districts; indices are created to measure college and career readiness. These and other new uses rely on derived scores created to measure new constructs. The field of educational and psychological measurement has largely ignored these significant, consequential measurement applications. The conceptual frameworks and analytical tools of educational and psychological measurement should be used to study such derived scores and the validity of their uses and interpretations.     

The psychological dimension of transformation in teacher learning

Against a background which recognises pedagogical content knowledge as the distinctive element of teacher competence/expertise, this theoretical essay argues for its central construct – that of transformation – to be understood by teachers and teacher educators in psychological terms (as was originally proposed by Dewey). Transformation requires teachers to fashion disciplinary knowledge such that it is accessible to the learner. It is argued that for transformation to happen, teacher thinking must include a sophisticated grasp of cognition and metacognition if teachers are to be characterised as competent, let alone expert. This article is written within a context of considerable social and academic scrutiny in the UK of the form and content of professional teacher preparation and development. In recent years, the contribution of psychological knowledge to teacher education has been filtered through procedural lenses of how best to ‘manage classrooms’, ‘assess learning’, ‘build confidence’ or whatever without a matched concern for psychological constructs through which such issues might be interpreted; thus, leaving teachers vulnerable in their professional understandings of learning and its complexities. That society now requires high-level cognitive engagement amongst its participants places cognitive and metacognitive demands on teachers which can only be met if they themselves are conceptually equipped.     

Developing the next generation of engaged youth: Inspire Aspire – Global Citizens in the Making

School-based character education (CE) programs provide an opportunity to increase the moral fortitude of adolescents. This study is a preliminary evaluation of Inspire Aspire, a CE program that was implemented with 13- to 14-year-olds in Scotland. A relational developmental systems meta-theoretical approach and person-centered analyses were employed to understand whether teacher implementation variability is associated with student outcomes. The study aimed to: assess variation in program implementation across teachers; assess student poster quality, which served as a youth outcome measure; and, assess the relationship between variations in program implementation and poster quality. Teachers who fully integrated Inspire Aspire with the broader curriculum (as opposed to using it as a standalone program) as well as teachers who made more modifications to Inspire Aspire tended to have students with higher quality posters. This finding stands in contrast to the common narrative regarding evidence-based programs that requires teachers to strictly adhere to program guidelines in order to maintain implementation fidelity.     

Developing a Multi-Dimensional Early Elementary Mathematics Screener and Diagnostic Tool: The Primary Mathematics Assessment

There is a critical need to identify primary level students experiencing difficulties in mathematics to provide immediate and targeted instruction that remediates their deficits. However, most early math screening instruments focus only on the concept of number, resulting in inadequate and incomplete information for teachers to design intervention efforts. We propose a mathematics assessment that screens and provides diagnostic information in six domains that are important to building a strong foundation in mathematics. This article describes the conceptual framework and psychometric qualities of a web-based assessment tool, the Primary Math Assessment (PMA). The PMA includes a screener to identify students at risk for poor math outcomes and a diagnostic tool to provide a more in-depth profile of children’s specific strengths and weaknesses in mathematics. The PMA allows teachers and school personnel to make better instructional decisions by providing more targeted analyses.     

The Rosenberg Self-Esteem scale and Harter's Self-Perception profile for adolescents: a concurrent validity study

The Rosenberg Self-Esteem Scale (RSE) is a widely used measure of global self-esteem. Although its psychometric properties have found considerable support, its relationship to a multidimensional scale of self-concept has yet to be investigated. The sample for this study consisted of 150 adolescents randomly drawn in equal numbers and equated by gender from grades 8 to 12. Along with the RSE, Harter's Self-Perception Profile for Adolescents was administered to assess the adolescents' self-concept in nine separate domains. Correlational and cross-validation multiple regression analyses found that the RSE total score and both its factor scores were strongly related to Global Self-Worth, supporting Rosenberg's conclusions that his scale is a measure of global self-esteem and that its two identified factors are essentially measuring one rather than two different constructs. Other findings include a gender difference, with females reporting significantly lower RSE scores, and modest correlational support for a grade level rise found in the literature.     

A systematic review on empathy measurement tools for care professionals

ABSTRACT

Empathic understanding of older adults is a critical attribute required for care professionals to provide quality care and relationship-based practice. To assess care professionals’ empathy, identifying an appropriate empathy measurement tool with good psychometric properties is needed. This systematic review aimed to identify empathy measures that can be used for care professionals and evaluate the rigor of empathy measures. Eligible studies published between 1950 and 2018 were extracted from five databases. A five-criteria appraisal framework was used for quality appraisal of empathy measures. A total of 11 empathy measurement tools were included. Based on the scores of the appraisal framework, CARE, JSE and TES were the highest quality empathy measures, and the lowest quality measure was EQ. None of the measures was specifically developed to measure empathy of geriatric care professionals. This review addresses the limitations of the existing empathy measures and suggests future directions for research.