Measurement invariance across information and communication technology development index and gender: The case of the Pearson Test of English Academic reading

The computerization of reading assessments has presented a set of new challenges to test designers. From the vantage point of measurement invariance, test designers must investigate whether the traditionally recognized causes for violating invariance are still a concern in computer-mediated assessments. In addition, it is necessary to understand the technology-related causes of measurement invariance among test-taking populations. In this study, we used the available data (n = 800) from the previous administrations of the Pearson Test of English Academic (PTE Academic) reading, an international test of English comprising 10 test items, to investigate measurement invariance across gender and the Information and Communication Technology Development index (IDI). We conducted a multi-group confirmatory factor analysis (CFA) to assess invariance at four levels: configural, metric, scalar, and structural. Overall, we were able to confirm structural invariance for the PTE Academic, which is a necessary condition for conducting fair assessments. Implications for computer-based education and the assessment of reading are discussed.     

Generalizability of achievement goal profiles across five cultural groups: More similarities than differences

Previous results have shown possible cultural differences in students’ achievement goals endorsement and in their relations with various predictors and outcomes. In this person-centered study, we sought to identify achievement goal profiles and to assess the extent to which these configurations and their associations with predictors and outcomes generalize across cultures. We used a new statistical approach to assess latent profile similarities across adolescents from five cultural backgrounds (N = 2643, including Non-Indigenous Australians, Indigenous Australians, Indigenous American, Middle Easterners, and Asians). Our results supported the cross-cultural generalizability of the profiles, their predictors, and their outcomes. Five similar profiles were identified in each cultural group, but their relative frequency differed across cultures. The results revealed advantages of exploring multidimensional goal profiles.     

Assessing construct of DIBELS: Measurement invariance of DIBELS subscales,from kindergarten to the first grade

A challenge using the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) in studying reading growth is that reading skills children exhibit change by age. In order to study growth using changing subscales, it is necessary to examine measurement invariance and measurement structure underlying the different subscales. The purpose of this paper is to examine the measurement structure of the DIBELS subscales, particular measurement invariance. The results indicate that the DIBELS subscales do not seem to have metric invariance but they do share a common factor over time, suggesting that the same construct of reading skills were measured but they manifested in the different fashion over time.     

Measurement invariance of information,communication and technology (ICT) engagement and its relationship with student academic literacy: Evidence from PISA 2018

The present study aimed to examine the measurement invariance of the information, communication and technology (ICT) engagement questionnaire in the Programme for International Student Assessment (PISA) across countries, and further to explore the effects of ICT engagement on student literacy. The sample included 98,758 15-year-old students from 16 countries in PISA 2018. The multi-group confirmatory factor analyses results indicated that the ICT engagement questionnaire was invariant at the residual level so that meaningful cross-country comparisons can be made at the observed level. Furthermore, the multi-level model results indicated that perceived ICT autonomy was a positive predictor, while use of social media was a negative predictor of student literacy across the countries. However, the relations of interest in ICT and perceived ICT competence to student literacy were inconsistent, with some countries showing positive and linear relationships, while others negative and linear relationships.     

The structure of teaching practices across countries: A combination of factor analysis and network analysis

Teaching practices are pivotal for student learning. Due to pedagogical traditions and national cultures, the structure of teaching practices may differ across countries. This study investigates the structure of teaching practices across 12 countries grouped into four major linguistic/cultural clusters. First, factor analysis is applied to investigate if the theoretical distinction between teacher-directed and student-centred practices is generalizable across countries. Then, network analysis is used to explore how individual classroom assessment practices relate to either teacher-directed or student-centred practices. Main findings include that: (1) teacher-directed and student-centred practices are two distinct factors across countries; (2) the overall structure and connectivity of teaching practices differs across countries, with smaller differences within linguistic/cultural clusters; and (3) assessment practices with the aim to structure and guide learning strongly relate to teacher-directed practices, whereas assessment practices with the aim to individualize instruction more relate to student-centred practices. We discuss the global patterning and implications.     

Assessing task values in five subjects during secondary school: Measurement structure and mean level differences across grade level,gender, and academic subject

Students’ value beliefs tend to decrease across secondary school (Wigfield et al., 2015). However, previous studies did not differentiate between all the dimensions of task values defined by expectancy-value theory (Eccles et al., 1983). Therefore, this study evaluated an instrument for assessing multiple value dimensions across grade level and academic subjects and tested for differences between grade levels in these subjects. A total of 830 students from Grades 5 to 12 completed a questionnaire assessing their value beliefs in German, English, math, biology, and physics with 37 items each. The factor structure was shown to be invariant across academic subjects, grade levels, and gender. Generally, students in higher grades showed lower means on positive value facets and higher means on cost facets. However, the results varied substantially by facet and subject. Furthermore, stereotypical gender differences in value beliefs were found, and some of these differences increased with students’ grade level. The findings indicate that examining multiple dimensions and subjects is crucial for developing a complete understanding of the development of students’ value beliefs.     

“Bugs” built into the system:: How privileged representations influence mathematical reasoning across the lifespan

The initial foundations of human mathematical reasoning appear to be based on “naïve mathematics”—specific and persistent privileged mental representations that develop as a normal part of the human evolved phenotype. Based on the proposed existence of privileged representations in the conceptual domain of mathematics, this paper incorporates findings from early development, childhood mathematical reasoning, and adult statistical decision-making research. The utility of such a framework is demonstrated by analyzing how common errors in fraction and decimal use are explicable in terms of these systematic and reliably developing aspects of human mathematical reasoning. Additionally, the idea that privileged representations continue to exert some influence beyond early childhood holds implications for both research and practice in mathematics education.     

Changing approaches to classroom assessment: An empirical study across teacher career stages

Research indicates that there is considerable variability in teachers' approaches to assessment resulting in different learning cultures for students. The primary purpose of the study is to examine the relationship between teachers' approaches to assessment across a set of dimensions (including their conceptions of assessment purposes, processes, fairness, and measurement theory) and career stage. The results of this paper illustrate nuanced impacts of career stage on teachers' approaches to multiple dimensions of assessment and enable the generation of assessment profiles that provide empirical support for differences in teachers’ approaches to assessment both within and between career stages.     

Double standards: when an undergraduate dissertation becomes the object of two different assessment approaches

Within a transnational educational programme, students residing in the Czech Republic obtain baccalaureate degrees from an accredited American college. The college has a distinctive approach towards learning, co‐creation of knowledge and the use of mentors. Part of the degree assessment is an undergraduate dissertation, which serves as a capstone experience. The Czech partner instituted a policy whereby students could obtain a second local degree by, among other things, using the same dissertation. This study examines the situation confronting mentors from the American college. It considers assessment as an integral component of a constellation of educational assumptions and pedagogic values: a paradigm. It analyses the competing/conflicting, paradigms involved and discusses ways in which educational practice was reconsidered and changed to allow a single work product to be authentically and meaningfully assessed under both approaches.     

Classroom assessment in the eyes of Chinese primary mathematics teachers: A review of teacher-written papers

In this paper we report on a review of papers written by teachers aimed at knowing more about teachers’ perceptions of the current situation of classroom assessment in primary mathematics education in China. The review is based on 266 papers included in the China National Knowledge Infrastructure database. We found that the teacher-authors reflected various aspects of their classroom assessment practice, including the purpose of assessment, the content of it, the person who is the assessor, the assessment methods that are used, and the feedback that is provided. Most attention was paid to feedback; it seems many teacher-authors considered classroom assessment to be equivalent to feedback. In general, the conceived classroom assessment practice as described in the papers echoed well nearly all aspects that are advocated in the Chinese mathematics curriculum standards. The only aspect that was scarcely discussed in the papers was the use of assessment results to adapt and improve instruction.