Cross-cultural comparability of noncognitive constructs in TIMSS and PISA

Abstract

Noncognitive assessments in Programme for International Student Assessment (PISA) and Trends in International Mathematics and Science Study share certain similarities and provide complementary information, yet their comparability is seldom checked and convergence not sought. We made use of student self-report data of Instrumental Motivation, Enjoyment of Science and Sense of Belonging to School targeted in both surveys in 29 overlapping countries to (1) demonstrate levels of measurement comparability, (2) check convergence of different scaling methods within survey and (3) check convergence of these constructs with student achievement across surveys. We found that the three scales in either survey (except Sense of Belonging to School in PISA) reached at least metric invariance. The scale scores from the multigroup confirmatory factor analysis and the item response theory analysis were highly correlated, pointing to robustness of scaling methods. The correlations between each construct and achievement was generally positive within each culture in each survey, and the correlational pattern was similar across surveys (except for Sense of Belonging), indicating certain convergence in the cross-survey validation. We stress the importance of checking measurement invariance before making comparative inferences, and we discuss implications on the quality and relevance of these constructs in understating learning.     

Teacher education variables as correlates of primary science ratings in thirteen TIMSS systems

This paper presents a comparative overview of teacher education variables associated with primary science in 13 TIMSS educational systems. While using TIMSS mean cohort performances at the Year 4 level to rank the sample systems, the study went beyond TIMSS in that it was at the whole-system level and took into account developments since those tests. The study reinforced the view that primary teacher training ideally occurs in a university, and involves a 4-year degree programme that preferably adheres to common standards across institutions. Teachers’ attainment at high school emerged as a principal correlate with TIMSS rankings. Better rankings were also associated with the existence of mandatory science ‘content’ studies as part of teacher training. These observations are consistent with the axiom that teachers’ competence in primary science arises largely from their own mastery of scientific concepts. The authors propose that candidates for primary teacher training programmes should have been awarded passes in science, including physical science, at minimally the middle secondary level, and urge primary teacher training institutions to include compulsory science ‘content’ as well as science pedagogy courses in their programmes.     

Gender Differences in Mathematics Self-concept Across the World: an Exploration of Student and Parent Data of TIMSS 2015

International Journal of Science and Mathematics Education - Using data from the Trends in International Mathematics and Science Study (TIMSS) 2015 assessment of fourth-grade students in 32...     

清华大学和美国大学在学习过程指标上的比较:一种高等教育质量观

本研究对目前各种高等教育质量评估取向加以了分析和批判,指出学习过程性指标对于高等教育质量评估和保障的重要意义.在引进、修订美国当前最有影响的一套大学学习过程性指标NSSE(National Survey of Student Engagement)的基础上,本研究对清华大学和美国大学的本科教学进行了比较.研究结果显示,在整体水平上来说清华大学的本科教学和美国同类院校乃至美国总体大学水平之间不存在根本差异,但和美国最优异的本科教育相比清华还存在一定差距.     

Metaphorical images of an organization: the power of symbolic constructs in reading change in higher education organizations

Based on interviews with 24 faculty members at a large, public university, this article reports the use of metaphors as a new conceptual strategy to analyze change in higher education organizations. Results of the study indicate that strategic choices guiding the behavior of the organization under study and the metaphorical images held by the faculty members about their organization show a high degree of congruence. Implications for change and maintenance of enacted realities in higher education organizations are discussed.     

Policy making in adult education: A comparative approach across 21 European regions

International Review of Education -     

Understanding change in higher education: an archetypal approach

During the past three decades, higher education institutions have been changing, moving away from the traditional bureaucratic archetype towards a more managerialist one. Empirical research already demonstrated that organisations tend to be in a hybrid area of archetypal change. Considering the specific case of a government-imposed reform in Portugal, and using a case study approach of six public universities, this study aims to explore archetypal hybridism through the lens of two main dimensions: systems and structures and interpretive scheme. The theoretical background lies on academic literature on organisational change in higher education and specifically on archetype theory. The findings drawn from document analysis and interviews outline the main characteristics of the hybrid archetype that we chose to name efficient-collegiality.     

PISA,TIMSS and Finnish mathematics teaching: an enigma in search of an explanation

Finnish students’ success on all three content domains of each of the four cycles of the OECD’s Programme for International Student Assessment (PISA) has created much international interest. It has also prompted Finnish academics to offer systemic explanations typically linked to the structural qualities of Finnish schooling and teacher education. Less well-known has been the modest mathematics performance of Finnish grade 8 students on the two Trends in International Mathematics and Science Study (TIMSS) in which Finland has participated, which, when compared with its PISA successes, has created something of an enigma. In this paper, we attempt to shed light on this enigma through analyses of Finnish mathematics classroom practice that draw on two extant data sets—interviews with Finnish teacher educators and video-recordings of sequences of lessons taught on standard topics. Due to the international interest in Finnish PISA success, the analyses focus primarily on the resonance between classroom practice and the mathematical literacy component of the PISA assessment framework. The analyses indicate that Finnish mathematics didactics are more likely to explain the modest TIMSS achievements than PISA successes and allude to several factors thought to be unique to the Finns, which, unrelated to mathematics teaching practices, may be contributory to the repeated Finnish PISA successes. Some implications for policy-borrowing are discussed.     

Stimulating tensions in special education teachers' figured world: an approach toward inclusive education

I examine special educators’ professional identity emergence and tensions within a researcher-facilitated teacher learning community. I introduced tools to evoke and challenge inequities in educational systems and via which participants examined and planned general education instruction for students with dis/abilities. Initially, professional identity, or figured world, emerged as performance of pathologising and relatedly, remediating students. Over time, participants expressed tensions as they engaged tools to examine structural limitations and design more universally accessible instruction; figured worlds shifted to critical sense-making about their positioning by general education colleagues and school structural barriers, and procedural identity performance tied to investigating student assets. Findings suggest potential for purposefully designed artefacts to mediate special educators’ development as (more) inclusive educators.     

Identifying national cultures of mathematics education: Analysis of cognitive demands and differential item functioning in TIMSS

Large-scale assessments of student competencies address rather broad constructs and use parsimonious, unidimensional measurement models. Differential item functioning (DIF) in certain subpopulations usually has been interpreted as error or bias. Recent work in educational measurement, however, assumes that DIF reflects the multidimensionality that is inherent in broad competency constructs and leads to differential achievement profiles. Thus, DIF parameters can be used to identify the relative strengths and weaknesses of certain student subpopulations. The present paper explores profiles of mathematical competencies in upper secondary students from six countries (Austria, France, Germany, Sweden, Switzerland, the US). DIF analyses are combined with analyses of the cognitive demands of test items based on psychological conceptualisations of mathematical problem solving. Experts judged the cognitive demands of TIMSS test items, and these demand ratings were correlated with DIF parameters. We expected that cultural framings and instructional traditions would lead to specific aspects of mathematical problem solving being fostered in classroom instruction, which should be reflected in differential item functioning in international comparative assessments. Results for the TIMSS mathematics test were in line with expectations about cultural and instructional traditions in mathematics education of the six countries.     

Reconceptualise a dynamic framework of the learning constructs in higher education

This paper reconceptualised the interrelated learning constructs in higher education based on the Dynamic Systems Theory (DST). The university students' learning experience before, during and post the Emergency Online Learning (EOL) was investigated to explore the dynamic changes among the learning constructs in higher education. A case study of a Chinese university was conducted, and one hundred and ninety-three university students participated in the questionnaire. The data collected from this empirical research identify different hierarchical constructs of the conceptualised learning environment and reconceptualise the period of system reformation influenced by the EOL. The key findings include the identifications of the attractors and repellors framed by the DST and the impact on the changes in the learning constructs. The results of this paper contribute to further understanding of the university constructs' changes to better plan and support students' active learning in higher education.     

Student-generated content: an approach to harnessing the power of diversity in higher education

Internationally, classes in higher education institutions are becoming larger and more diverse. Support for ‘non-traditional’ students has often taken the form of additional remedial classes offered outside the main curriculum, which has met with limited success. Sociocultural theories of learning argue that the potential clash between the sociocultural context of disciplinary knowledge and the very different home contexts of many non-traditional students needs to be acknowledged. One way to achieve this is to use student-generated content, which allows teachers to bring student experiences and voices into the community of practice and acknowledges the importance of their prior experiences in knowledge production. Building on such sociocultural approaches to teaching and learning, this paper focuses on the use of student-generated podcasts as a means to harness the diversity of student experiences in a large (nearly 600 students) first-year Economics class at a South African University.     

Assessing conceptual change of teachers involved in STES education and curriculum devleopment - the STEMS project approach

Science-Technology-Environment-Society (STES) orientation in science education is currently being implemented in Israeli high schools within the framework of 'science for all' reform worldwide. This paper focuses on assessing the conceptual change of teachers who have been involved in the development, implementation, field-testing and evaluation of several modules. These modules constitute a grade 10-11 high school national curriculum titled STEMS - 'Science, Technology, Environment in Modern Society'. STEMS is aimed at developing an autonomous learner, capable of system thinking, decision making and problem solving within the real life STES context. We sensed that the intrinsic nature of STEMS curriculum requires that the teachers, who will teach it, will also be the developers of its modules. Involvement of this kind makes the teachers responsible for their own conceptual change, explanations and interpretations. Our formative evaluation indicates that the conceptual change of STEMS teachers was gradual. Participants differed with respect to what sort of 'treatment' or experience within the project actually affected who and when. It was apparent that the change occurred with respect to both their content knowledge and pedagogical views. A positive response towards teaching beyond the discipline boundaries was followed by teachers' active involvement and participation in the development process and team discussions. Thus, the STEMS project affected their teaching/learning perception towards interdisciplinarity. These findings are in accord with teachers' support of a life cycle approach for curriculum development as being suitable for achieving the STEMS objectives. The teachers emphasized the need to practice together with their students scientific inquiry and experiment design skills which, foster an autonomous learner. At the end of the first year of the curriculum development process, STEMS was finally conceptualized by the project teachers as a novel way of learning, rather than another sophisticated teaching technique. The major conceptual change was, the switch teachers made from the role of knowledge providers into that of learners. The interplay among action, participation and conceptualization turned out to be instrumental in our life cycle approach for developing the STEMS curriculum.     

试论网络思想政治教育环境的优化

在网络思想政治教育过程中,一个良好的网络环境起着至关重要的作用。信息时代如何更好地把握信息,网络思想政治教育环境的优化不可忽视。这就需要思想政治教育主客体共同努力,既有个人的道德约束,又有社会的强制规范,同时加大对网络资源的把握,只有这样,一个洁净的网络思想政治教育环境才可能出现。唯其如此,网络思想政治教育才会更富有成效。     

Collaborating with children in exploring marginalisation: an approach to inclusive education

This article focuses on the importance of engaging with children's voices in school settings in order to understand and deal with marginalisation. Engaging with the views of children and young people is an essential part of the process of developing inclusion. It can be viewed as an approach to inclusive education, which predominantly places emphasis on the views of the learners rather than on other organisational aspects within a school context. The study reported here is from a collaborative piece of research in one primary school classroom where practitioners, children and the researcher worked together in order to identify areas of concern within the particular context, in order to address marginalisation. The benefits, as well as the complexities of engaging in such processes are discussed. Furthermore, the study has led to the development of a framework for promoting inclusion that is presented here. The framework can be used to guide a systematic way of collecting evidence that will facilitate the process of understanding and dealing with marginalisation, and promoting inclusion, that can be incorporated into the daily work of practitioners.     

Achievement emotions in higher education: A diary study exploring emotions across an assessment event

An increasing number of emotions have been found to affect the way students learn and their academic achievement. However, little is known about how dynamic these achievement emotions (AEs) are, the extent to which they vary with the assessment process, and how they relate to prior academic ability and student achievement outcomes. Our intensive longitudinal diary study with tertiary students (N = 166) examined their AEs across a three week assessment period (study, test and feedback week). Overall, the results indicated that emotions during the study and test week were unrelated to both GPA and test score, but the starting level of emotions during the feedback week were related to GPA and test score. The changeability of emotions were not related to either GPA or test score. Overall, AEs seem to have a meaningful relationship to achievement only once results are known. These findings expand our knowledge about the relationship between AEs, prior academic ability and achievement and how emotions change across an assessment event.     

Creating conversations: an inclusive approach to the international networking of knowledge about education

Networking has been central to human communication and social engagement for centuries, but has changed dramatically with the advent of new technology. However, the practice of networking has received little academic attention and tends to be undertheorised. This paper considers the impact of the global digital and communication divide, and the contested nature of inclusive education, on the task of promoting information sharing internationally. A summative case study is presented of a network, established to support the documentation of promising practice on inclusive education in countries with limited access to information and material resources. The case study uses autoethnographic methods to identify key principles for the development of an inclusive network. The significance of this paper is in its proposition of a working model of inclusive networking, which has potential to be developed into a theory.     

The effects of motivational constructs and engagements on mathematics achievements: a comparative study using TIMSS 2011 data of Chinese Taipei,Singapore, and the USA

Motivational constructs and students’ engagements have great impacts on students’ mathematics achievements, yet they have not been theoretically investigated using international large-scale assessment data. This study utilized the mathematics data of the Trends in International Mathematics and Science Study 2011 to conduct a comparative and empirical study on exploring: (1) the changes of motivational constructs from 4th grade to 8th grade; and (2) the effects of motivational constructs from the expectancy-value model and students’ engagements on mathematics achievements. The countries investigated include Chinese Taipei, Singapore, and the USA. The results showed that: (1) students’ motivations deteriorate over school years; (2) self-confidence in mathematics (SCM) has the strongest relationship with mathematics achievements. Furthermore, it is evident that Singapore has the most effective schools in students’ mathematics education. More findings of this comparative study are subsequently discussed.