Screening for Chinese Children with Dyslexia in Hong Kong: The use of the Teachers' Behaviour Checklist

Primary school teachers rated the frequency of occurrence of 65 reading‐related behavioural characteristics in a sample of 251 Grade 1 to Grade 6 Chinese school children in Hong Kong. These behavioural characteristics were in the areas of general performance, reading, dictation, writing, mathematics, language, memory, concentration, sequential ability, motor co‐ordination, spatial orientation, and social/emotional adjustment. Of these 12 areas, 10 yielded scale scores that could distinguish children with dyslexia from those without dyslexia, identified on the basis of their performance in five domains of literacy and cognitive skills. Using a summary score derived from the 10 relevant scales, an optimal cut‐off score was suggested to arrive at a balance between high sensitivity and an acceptable rate of false positives in screening for children with dyslexia. The need for cross‐replication in screening children with dyslexia using the behaviour checklist with different samples of school students is emphasised.     

Annual Index Volume 8

The objective of this paper is to report the development and use of an observation checklist to identify typically developing children with behavioural manifestations associated with inattention, hyperactivity and impulsivity. This measure is termed the Scope Classroom Observation Checklist (SCOC). The SCOC was developed, assessed for reliability using an independent observer and teacher ratings, and administered to a sample of 157 mainstream schoolchildren. The SCOC revealed a normal distribution of scores on the checklist, and the top and bottom quartiles of this distribution were found, as would be expected, to differ significantly on SCOC score. The SCOC had good inter‐observer reliability and a high rate of concordance with teacher ratings of attentional difficulties. The SCOC has emerged as a reliable measure that could prove to be a useful tool in a battery of screening measures to identify mainstream schoolchildren with difficulties characterised by inattention and hyperactivity‐impulsivity.     

Factors associated with an increased impact of child sexual abuse

Data is presented identifying factors associated with the impact of sexual abuse on children. A group of 369 sexually abused children and a comparison group of 318 children recruited from the community were compared on a parent-completed behavior rating scale. Data describing the abused children were also available from a 38-item symptom checklist completed by the child's social worker. Using a score based on the symptom checklist as the measure of the impact of sexual abuse, 15 variables were in the final regression equation explaining 42% of the variance in impact. Using a score based on parent-generated data, 5 variables were in the final equation explaining 20% of the variance. The significance of the variables in identifying factors associated with an increased impact of abuse is discussed.     

Screening for multiple genes influencing dyslexia

Genetic linkage analysis is a means of localizing genes to specific chromosomal regions. Localization of genes influencing specific reading disability (dyslexia) can lead to characterization of the phenotypic effects of each gene and to early diagnosis of children at risk. Previous studies using the family study LOD score method of linkage analysis have identified two chromosomal regions that may contain genes influencing dyslexia. The present study examines the sib pair method of linkage analysis, which has several advantages over the LOD score method. In particular, the mode of inheritance does not need to be specified and diagnosis of parents is not required, but it is a less powerful technique. Using the same population as the previous studies (with less than 200 sib pairs) and two different means of diagnosis of dyslexia, the sib pair analysis was able to detect the same suggested linkages as the LOD score method, plus a possible third region. This confirms that the sib pair method is an effective means of screening for linkage with reasonable sample sizes.     

Hong Kong parents and their children’s music training: measurement properties of the Parental Involvement in Music Training Questionnaire

Many Hong Kong-Chinese parents are active in their support for their children’s music training. To better understand this support, the Parental Involvement in Music Training Questionnaire (PIMTQ) is designed to measure the variability in parental involvement in their children’s music training. This study begins by exploring the factor structure of the PIMTQ and then establishes its measurement properties using Rasch modelling. Two hundred and ninety-five Hong Kong-Chinese parents completed a Chinese version of the 42-item instrument with principal components analysis of the responses showing seven factors. However, Rasch modelling showed that two of the five factors (Family Music Background and Family Music Interest) are unable to reliably predict variability in parent responses. We conclude, however, that the remaining five factors (Parental Support Toward Music Training, Parental Expectations, Home Music Environment, Music Programme Support and Attitude Toward Music) of the PIMTQ can be used as subscales to measure the involvement of Hong Kong-Chinese parents in their children’s music training.     

Psychometric Characteristics of the Revised Procrastination Inventory

The Procrastination Inventory developed for use with doctoral students in clinical psychology was modified for use with ABD students and doctoral graduates in a College of Education. The original Procrastination Inventory contained 43 items with 11 subscales. The structure of the revised measure was analyzed both through factor and Rasch analyses and three subscales that were more generalized were found instead of the eleven originally posited. The three subscales were: (1) procrastination, 20 items, alpha = .88, (2) perfectionism, 9 items, alpha = .64, and (3) graduate school comfort, 6 items, alpha = .59. Eight items were deleted after Rasch and factor analyses, resulting in a 35-item scale. Validity was demonstrated by the measure's ability to predict dissertation completion and through correlations with related measures. The Procrastination Inventory is useful in the study of attrition from doctoral programs, particularly at the dissertation stage.     

Assessing the efficacy of the Measure of Understanding of Macroevolution as a valid tool for undergraduate non-science majors

Efficacy of the Measure of Understanding of Macroevolution (MUM) as a measurement tool has been a point of contention among scholars needing a valid measure for knowledge of macroevolution. We explored the structure and construct validity of the MUM using Rasch methodologies in the context of a general education biology course designed with an emphasis on macroevolution content. The Rasch model was utilized to quantify item- and test-level characteristics, including dimensionality, reliability, and fit with the Rasch model. Contrary to previous work, we found that the MUM provides a valid, reliable, and unidimensional scale for measuring knowledge of macroevolution in introductory non-science majors, and that its psychometric behavior does not exhibit large changes across time. While we found that all items provide productive measurement information, several depart substantially from ideal behavior, warranting a collective effort to improve these items. Suggestions for improving the measurement characteristics of the MUM at the item and test levels are put forward and discussed.     

Physics Metacognition Inventory Part II: Confirmatory factor analysis and Rasch analysis

The Physics Metacognition Inventory was developed to measure physics students’ metacognition for problem solving. In one of our earlier studies, an exploratory factor analysis provided evidence of preliminary construct validity, revealing six components of students’ metacognition when solving physics problems including knowledge of cognition, planning, monitoring, evaluation, debugging, and information management. The college students’ scores on the inventory were found to be reliable and related to students’ physics motivation and physics grade. However, the results of the exploratory factor analysis indicated that the questionnaire could be revised to improve its construct validity. The goal of this study was to revise the questionnaire and establish its construct validity through a confirmatory factor analysis. In addition, a Rasch analysis was applied to the data to better understand the psychometric properties of the inventory and to further evaluate the construct validity. Results indicated that the final, revised inventory is a valid, reliable, and efficient tool for assessing student metacognition for physics problem solving.     

Rasch analysis for psychometric improvement of science attitude rating scales

ABSTRACT

Students’ attitude towards science (SAS) is often a subject of investigation in science education research. Survey of rating scale is commonly used in the study of SAS. The present study illustrates how Rasch analysis can be used to provide psychometric information of SAS rating scales. The analyses were conducted on a 20-item SAS scale used in an existing dataset of The Trends in International Mathematics and Science Study (TIMSS) (2011). Data of all the eight-grade participants from Hong Kong and Singapore (N?=?9942) were retrieved for analyses. Additional insights from Rasch analysis that are not commonly available from conventional test and item analyses were discussed, such as invariance measurement of SAS, unidimensionality of SAS construct, optimum utilization of SAS rating categories, and item difficulty hierarchy in the SAS scale. Recommendations on how TIMSS items on the measurement of SAS can be better designed were discussed. The study also highlights the importance of using Rasch estimates for statistical parametric tests (e.g. ANOVA, t-test) that are common in science education research for group comparisons.     

Investigating young children’s human figure drawings using Rasch analysis

The Goodenough–Harris Drawing Test (GHDT) is a non-verbal assessment designed to infer young children’s levels of intellectual development and understanding via the collection of three human figure drawings (HFDs) – one each of a man, a woman and a self-portrait. This paper presents findings from a research project that applied the Rasch model for measurement to HFDs collected from 246 children aged between 4 and 10 years to assess the psychometric properties of the GHDT assessment, and young children’s HFDs in general, as the GHDT had not yet been examined from a modern test theory perspective in full. Results indicated that: (1) the GHDT and children’s HFDs were apt for Rasch analysis and deemed to be generally psychometrically sound; and (2) that children performed almost identically (within error) on all three HFDs collected for the assessment, suggesting that the collection of three HFDs – as well as many of the 217 items which comprise the GHDT – was potentially redundant. Consequently, a prototype human figure drawing continuum (HFDC) was constructed from those data and examined. Rasch analysis revealed that the researcher-constructed 45-item HFDC scale – requiring just one self-selected HFD of a man, woman or child – was just as effective as the 217-item GHDT. The HFDC, when normed, should be seen as a parsimonious and child-friendly HDF assessment.     

Developing a measure of wealth for primary student families in a developing country: Comparison of two methods of psychometric calibration

This article compares the invariance properties of two methods of psychometric instrument calibration for the development of a measure of wealth among families of Grade 5 pupils in five provinces in Vietnam. The measure is based on self-reported lists of possessions in the home. Its stability has been measured over two time periods. The concept of fundamental measurement, and the properties of construct and measurement invariance have been outlined. Item response modelling (IRM) and confirmatory factor modelling (CFM) as comparative methodologies, and the processes used for evaluating these, have been discussed. Each procedure was used to calibrate a 23-item instrument with data collected from a probability sample of Grade 5 pupils in a total of 60 schools. The two procedures were compared on the basis of their capacity to provide evidence of construct and measurement invariance, stability of parameter estimates, bias for or against sub samples, and the simplicity of the procedures and their interpretive powers. Both provided convincing evidence of construct invariance, but only the Rasch procedure was able to provide firm evidence of measurement invariance, parameter stability and a lack of bias across samples.     

The effects of features of examination questions on the performance of students with dyslexia

This research investigated whether features of examination questions influence students with dyslexia differently to others, potentially affecting whether they have a fair opportunity to show their knowledge, understanding and skills. A number of science examination questions were chosen. For some questions two slightly different versions were created. A total of 54 students considered by their teachers to have dyslexia and a matched control group of 51 students took the test under exam conditions. A dyslexia screening assessment was administered where possible and some students were interviewed. Facility values and Rasch analysis were used to compare performance between the versions of the same question and between those with and without dyslexia. Chi‐square statistics found no statistically significant differences in performance between groups or between question versions. However, some tentative implications for good practice can be inferred (e.g. avoiding ambiguous pronouns, using bullet points).     

Finding two dimensions in MMPI‐2 depression

This study examined the underlying structure of the Depression scale of the revised Minnesota Multiphasic Personality Inventory using dichotomous Rasch model and factor analysis. Rasch methodology was used to identify and restructure the Depression scale, and factor analysis was used to confirm the structure established by the Rasch model. The item calibration and factor analysis were carried out on the full sample of 2,600 normative subjects. The results revealed that the Depression scale did not consist of one homogeneous set of items, even though the scale was developed to measure one dimension of depression. Rasch analysis, as well as factor analysis, recognized two distinct content‐homogeneous subscales, here labeled mental depression and physical depression. The Rasch methodology provided a basis for a better understanding of the underlying structure and furnished a useful solution to the scale refinement.     

Psychometric characteristics of integrated multi-specialty examinations: Ebel ratings and unidimensionality

Over recent years, UK medical schools have moved to more integrated summative examinations. This paper analyses data from the written assessment of undergraduate medical students to investigate two key psychometric aspects of this type of high-stakes assessment. Firstly, the strength of the relationship between examiner predictions of item performance (as required under the Ebel standard setting method employed) and actual item performance (‘facility’) in the examination is explored. It is found that there is a systematic pattern of difference between these two measures, with examiners tending to underestimate the difficulty of items classified as relatively easy, and overestimating that of items classified harder. The implications of these differences for standard setting are considered. Secondly, the integration of the assessment raises the question as to whether the student total score in the exam can provide a single meaningful measure of student performance across a broad range of medical specialties. Therefore, Rasch measurement theory is employed to evaluate psychometric characteristics of the examination, including its dimensionality. Once adjustment is made for item interdependency, the examination is shown to be unidimensional with fit to the Rasch model implying that a single underlying trait, clinical knowledge, is being measured.     

Prevalence,gender ratio and gender differences in reading‐related cognitive abilities among Chinese children with dyslexia in Hong Kong

Based on the data of the normative study of the Hong Kong test of specific learning difficulties in reading and writing, and the Test of visual‐perceptual skills (non‐motor)—Revised, 99 children aged between 6 and 10½ years were identified as children with dyslexia out of the normative sample of 690 children. By excluding 12 children known to score below average in IQ, 87 children, including 20 children not tested for IQ, could be regarded as children with dyslexia, yielding a prevalence rate of 12.6% and a boy:girl gender ratio of 1.6 to 1. The figures would become 9.7% and 2.0 to 1 if the 20 children were omitted from computation. However, gender imbalance could not be readily explained by gender differences in reading‐related cognitive abilities, as there were only minor and subtle differences. Regression analyses to evaluate the relative contribution of different cognitive abilities to reading and writing suggested that orthographic knowledge and naming speed were most important among children with dyslexia. Implications of the findings and the need for early intervention are discussed.     

Digital Module 10: Rasch Measurement Theory https://ncme.elevate.commpartners.com

In this digital ITEMS module, Dr. Jue Wang and Dr. George Engelhard Jr. describe the Rasch measurement framework for the construction and evaluation of new measures and scales. From a theoretical perspective, they discuss the historical and philosophical perspectives on measurement with a focus on Rasch's concept of specific objectivity and invariant measurement. Specifically, they introduce the origins of Rasch measurement theory, the development of model‐data fit indices, as well as commonly used Rasch measurement models. From an applied perspective, they discuss best practices in constructing, estimating, evaluating, and interpreting a Rasch scale using empirical examples. They provide an overview of a specialized Rasch software program (Winsteps) and an R program embedded within Shiny (Shiny_ERMA) for conducting the Rasch model analyses. The module is designed to be relevant for students, researchers, and data scientists in various disciplines such as psychology, sociology, education, business, health, and other social sciences. It contains audio‐narrated slides, sample data, syntax files, access to Shiny_ERMA program, diagnostic quiz questions, data‐based activities, curated resources, and a glossary.     

A Framework for Measuring the Amount of Adaptation of Rasch‐based Computerized Adaptive Tests

A key consideration when giving any computerized adaptive test (CAT) is how much adaptation is present when the test is used in practice. This study introduces a new framework to measure the amount of adaptation of Rasch‐based CATs based on looking at the differences between the selected item locations (Rasch item difficulty parameters) of the administered items and target item locations determined from provisional ability estimates at the start of each item. Several new indices based on this framework are introduced and compared to previously suggested measures of adaptation using simulated and real test data. Results from the simulation indicate that some previously suggested indices are not as sensitive to changes in item pool size and the use of constraints as the new indices and may not work as well under different item selection rules. The simulation study and real data example also illustrate the utility of using the new indices to measure adaptation at both a group and individual level. Discussion is provided on how one may use several of the indices to measure adaptation of Rasch‐based CATs in practice.     

Measuring ocean literacy in pre-service teachers: psychometric properties of the Greek version of the Survey of Ocean Literacy and Experience (SOLE)

The aim of the present study was to respond to the increasing demand for comprehensive tools for the measurement of ocean literacy, by investigating the psychometric characteristics of a Greek version of the Survey of Ocean Literacy and Experience (SOLE), an instrument that assesses conceptual understanding of general ocean sciences content, focusing on the knowledge component. Four hundred twenty-one pre-service primary school teachers participated in a cross-sectional study. The dichotomous Rasch model was used to examine the measurement properties of the SOLE, namely, person-item targeting and separation, reliability, dimensionality and differential item functioning (DIF). Steps were taken to improve the instrument, where any of these attributes were outside acceptable ranges. Results suggested that a modified SOLE showed an adequate fit to the Rasch model, is unidimensional, free of DIF, and is particularly well-suited to the population under study. Our findings suggest that the SOLE constitutes a valuable tool which can be applied to a different cultural context and population. The proposed use of the instrument could contribute to the assessment of the quality of marine education in school-based and non-formal education contexts and to the cross-cultural comparison of ocean literacy, which are prerequisites for the improvement of ocean literacy.     

Validation of the perceived school bullying severity scale

Research on school bullying has tended to focus on its prevalence or frequency while ignoring its perceived severity. This study attempted to construct a perceived School Bullying Severity Scale (SBSS). The original 24-item instrument, revised from the Victim Scale of the School Bullying Scales, covered the four categories of physical, verbal, relational and cyber bullying. The partial credit model was used to conduct Rasch analysis with ConQuest software on data derived from two samples of Taiwanese secondary school students. Sample 1 and sample 2 consisted of 605 and 869 students, respectively. Three items were deleted after examining the quality of the data from sample 1. The reliability and validity of the 21 items on the final scale were verified using data from sample 2. Results demonstrated the reliability and validity of information collected by the SBSS. This study also found that secondary school students rated relational and cyber bullying as more severe than physical and verbal bullying. Differences between teachers’ and students’ perspectives on the perceived severity of various bullying behaviours as well as implications for preventing and intervening in bullying are discussed.     

Development of an instrument to measure perceived cognitive,affective, and psychomotor learning in traditional and virtual classroom higher education settings

The purpose of this study was to develop and validate a self-report instrument that can be used to measure learning in the cognitive, affective, and psychomotor domains. The study underwent three phases, each with its own data collection and analysis. Phase I featured the development, testing, and factor analysis of an 80-item instrument that addressed cognitive, affective, and psychomotor learning that was administered to a sample of 142 online and face-to-face learners. Based on the results, the instrument was reduced to 21 items for Phase II and tested with a new sample of 171 online and face-to-face students. The results of confirmatory factor analysis suggested a better data fit with an even smaller 9-item instrument, which was then administered to a new sample of 221 online and face-to-face students in Phase III. The results of this final phase are presented along with the resulting CAP Perceived Learning Scale, a 9-item self-report measure of perceived cognitive, affective, and psychomotor learning. Implications and usage of the CAP Perceived Learning Scale for research and practice are also discussed.