Psychometric quality of a student evaluation of teaching survey in higher education

Student evaluations of teaching (SET) are used globally by higher education institutions for performance assessment of academic staff and evaluation of course quality. Higher education institutions commonly develop their own SETs to measure variables deemed relevant to them. However, ‘home-grown’ SETs are rarely assessed psychometrically. One potential consequence of this limitation is that an invalidated instrument may not provide accurate information for the intended purposes. Moreover, in the absence of psychometric assessment, the students’ voices collected by the SETs often fail to provide insight relative to their intended purpose. The present study evaluates a ‘home-grown’ SET using a Rasch model and confirmatory factor analysis. Our results identified weaknesses in two areas: the rating categories and the number of items used to measure the intended constructs. Suggestions are provided to address these weaknesses. This work provides an additional tool set for critical analysis of SET that is generally applicable for a variety of institutions, including those in Asia.     

Science Process Skills and Cognitive Preference Styles

The study examined the differences in cognitive styles between two comparable groups of students at the Grade 9 (Secondary 3) level, namely the LSS (Lower Secondary Science) group who had been exposed to the practical-based, inquiry-oriented type of science and the non-LSS group of students who had studied the more traditional type of science in the “old” science curriculum. Their differences in science achievement are measured by the common IEA Science Paper-Pencil, Multiple Choice Criterion Test and also, by the Science Process or Practical Test (which measured three levels of process skills, such as the observation/manipulation, reasoning and investigation skills). Variance in science achievement thus measured is examined against the 4 cognitive preference styles of the students, (measured by the Combined Cognitive Preference Inventory) namely the “recall”, “principles”, “applications” and “questioning” modes of thinking. The findings indicated that (a) the attainment of the science process or practical skills was characterised by the type of science curriculum (LSS or non-LSS) and it was significantly associated with the achievement level of students, (b) the cognitive preference pattern covaried according to the students' level of science achievement and the type of curriculum and (c) the weak but significant relationship between performance in the science practical skills and the students' modes of cognitive style have important implications for teachers who are concerned about the intended effects of changes in the science curriculum on the consequent learning behaviour or cognitive outcome of their students.     

Perceptions of low ability students on group project work and cooperative learning

The benefits of cooperative learning strategies, such as group project work, have been advocated in a wide range of educational contexts. There is however, scant information on the outcome of such programs on students of low academic ability. In this study, interviews were conducted with four groups of low-ability students, with the aim of investigating their perceptions on the effectiveness of group project work in promoting their social and cognitive skills. Our findings show that although the students recognized that the program improved their competence in interacting with others, there are problematic organizational and instructional issues that have to be ironed out before students can profit fully from cooperative learning programs.     

Rasch analysis for psychometric improvement of science attitude rating scales

ABSTRACT

Students’ attitude towards science (SAS) is often a subject of investigation in science education research. Survey of rating scale is commonly used in the study of SAS. The present study illustrates how Rasch analysis can be used to provide psychometric information of SAS rating scales. The analyses were conducted on a 20-item SAS scale used in an existing dataset of The Trends in International Mathematics and Science Study (TIMSS) (2011). Data of all the eight-grade participants from Hong Kong and Singapore (N?=?9942) were retrieved for analyses. Additional insights from Rasch analysis that are not commonly available from conventional test and item analyses were discussed, such as invariance measurement of SAS, unidimensionality of SAS construct, optimum utilization of SAS rating categories, and item difficulty hierarchy in the SAS scale. Recommendations on how TIMSS items on the measurement of SAS can be better designed were discussed. The study also highlights the importance of using Rasch estimates for statistical parametric tests (e.g. ANOVA, t-test) that are common in science education research for group comparisons.     

On the Declining Interest in Physics among Students—From the perspective of teachers

This study examines physics teachers’ views on factors that contribute to the declining interest in physics among school students. A total of 190 physics teachers from 91 secondary schools and nine junior colleges in Singapore participated in this study, which used a survey instrument to seek their views. Rasch analysis was used to analyze the data. The findings showed that teachers are uncertain about the career prospects of physics graduates and are not very optimistic about the market demand for physics graduates. They feel that physics seems to be difficult and abstract to students. The perception that physics is for boys seems to be rooted in society. The teachers emphasized the importance of co‐curricular activities and laboratory work to enhance interest of students in learning physics. We provide several recommendations to arrest the declining interest in physics at the school level.