Detecting Measurement Disturbances in Rater‐Mediated Assessments

The term measurement disturbance has been used to describe systematic conditions that affect a measurement process, resulting in a compromised interpretation of person or item estimates. Measurement disturbances have been discussed in relation to systematic response patterns associated with items and persons, such as start‐up, plodding, boredom, or fatigue. An understanding of the different types of measurement disturbances can lead to a more complete understanding of persons or items in terms of the construct being measured. Although measurement disturbances have been explored in several contexts, they have not been explicitly considered in the context of performance assessments. The purpose of this study is to illustrate the use of graphical methods to explore measurement disturbances related to raters within the context of a writing assessment. Graphical displays that illustrate the alignment between expected and empirical rater response functions are considered as they relate to indicators of rating quality based on the Rasch model. Results suggest that graphical displays can be used to identify measurement disturbances for raters related to specific ranges of student achievement that suggest potential rater bias. Further, results highlight the added diagnostic value of graphical displays for detecting measurement disturbances that are not captured using Rasch model–data fit statistics.     

Parental Age and Offspring Childhood Mental Health: A Multi-Cohort,Population-Based Investigation

To examine the contributions of maternal and paternal age on offspring externalizing and internalizing problems, this study analyzed problem behaviors at age 10–12 years from four Dutch population-based cohorts (N = 32,892) by a multiple informant design. Bayesian evidence synthesis was used to combine results across cohorts with 50% of the data analyzed for discovery and 50% for confirmation. There was evidence of a robust negative linear relation between parental age and externalizing problems as reported by parents. In teacher-reports, this relation was largely explained by parental socio-economic status. Parental age had limited to no association with internalizing problems. Thus, in this large population-based study, either a beneficial or no effect of advanced parenthood on child problem behavior was observed.     

An Instructional Module on Mokken Scale Analysis

Mokken scale analysis (MSA) is a probabilistic‐nonparametric approach to item response theory (IRT) that can be used to evaluate fundamental measurement properties with less strict assumptions than parametric IRT models. This instructional module provides an introduction to MSA as a probabilistic‐nonparametric framework in which to explore measurement quality, with an emphasis on its application in the context of educational assessment. The module describes both dichotomous and polytomous formulations of the MSA model. Examples of the application of MSA to educational assessment are provided using data from a multiple‐choice physical science assessment and a rater‐mediated writing assessment.     

Does the Cognitive Interview Promote the Coherence of Narrative Accounts in Children With and Without an Intellectual Disability?

We examined whether the cognitive interview (CI) procedure enhanced the coherence of narrative accounts provided by children with and without intellectual disabilities (ID), matched on chronological age. Children watched a videotaped magic show; one day later, they were interviewed using the CI or a structured interview (SI). Children interviewed using the CI reported more correct details than those interviewed using the SI. Additionally, children interviewed using the CI reported more contextual background details, more logically ordered sequences, more temporal markers, and fewer inconsistencies in their stories than those interviewed using the SI. However, the CI did not increase the number of story grammar elements compared with the SI. Overall children interviewed with the CI told better stories than those interviewed with the SI. This finding provided further support for the effectiveness of the CI with vulnerable witnesses, particularly children with ID.     

Learning by Doing: Performance Assessment of Information Literacy across the First-Year Curriculum

In the 2011–2012 school year, the instruction librarians at Augustana College, Illinois, changed their assessment approach in the college's required first-year sequence to focus on higher-level information literacy concepts. The librarians replaced a quantitative assessment instrument with performance assessments, which they integrated into their first-year library sessions. Although the sequence is taught by many faculty with diverse assignments, these new assessments could be applied organically across sections yet provide generalizable results. This case study describes that assessment project and its initial findings, analyzes the project's implications, and suggests how other college libraries might adopt similar qualitative assessments.     

Enhancing employability through industrial training in the Malaysian context

This study discusses the industrial training programme at the University of Malaya in Malaysia, specifically the issues that need to be addressed in order to enhance the employability skills of graduates. Findings from the feedback obtained from trainees and organizations in the 2008/2009 academic session were examined in terms of the extent to which trainees felt they were prepared for their training, and the extent to which the tasks given to them during their training were appropriate. Further, trainees’ self-rating of particular skills and industry’s rating of the trainees were also examined. The feedback is discussed in the context of the need to equip graduates with employability skills, an issue facing Higher Education Providers worldwide. The findings indicate that most of the students were prepared to face the world of work. However, there were several issues which needed attention. These included the need to address the possible mismatch between the tasks assigned to trainees and their areas of study, and the need to enhance English language competency and particular soft skills throughout their degree programme. Continuous input from industry is also necessary to ensure that the training benefits all parties and contributes to the employability skills of trainees.     

When Parents’ Praise Inflates,Children's Self‐Esteem Deflates

Western parents often give children overly positive, inflated praise. One perspective holds that inflated praise sets unattainable standards for children, eventually lowering children's self‐esteem (self‐deflation hypothesis). Another perspective holds that children internalize inflated praise to form narcissistic self‐views (self‐inflation hypothesis). These perspectives were tested in an observational‐longitudinal study (120 parent–child dyads from the Netherlands) in late childhood (ages 7–11), when narcissism and self‐esteem first emerge. Supporting the self‐deflation hypothesis, parents’ inflated praise predicted lower self‐esteem in children. Partly supporting the self‐inflation hypothesis, parents’ inflated praise predicted higher narcissism—but only in children with high self‐esteem. Noninflated praise predicted neither self‐esteem nor narcissism. Thus, inflated praise may foster the self‐views it seeks to prevent.     

The Effects of Incomplete Rating Designs in Combination With Rater Effects

Researchers have explored a variety of topics related to identifying and distinguishing among specific types of rater effects, as well as the implications of different types of incomplete data collection designs for rater‐mediated assessments. In this study, we used simulated data to examine the sensitivity of latent trait model indicators of three rater effects (leniency, central tendency, and severity) in combination with different types of incomplete rating designs (systematic links, anchor performances, and spiral). We used the rating scale model and the partial credit model to calculate rater location estimates, standard errors of rater estimates, model–data fit statistics, and the standard deviation of rating scale category thresholds as indicators of rater effects and we explored the sensitivity of these indicators to rater effects under different conditions. Our results suggest that it is possible to detect rater effects when each of the three types of rating designs is used. However, there are differences in the sensitivity of each indicator related to type of rater effect, type of rating design, and the overall proportion of effect raters. We discuss implications for research and practice related to rater‐mediated assessments.     

An Analysis of Anatomy Education Before and During Covid-19: May–August 2020

Coronavirus disease 2019 (Covid-19) created unparalleled challenges to anatomy education. Gross anatomy education has been particularly impacted given the traditional in-person format of didactic instruction and/or laboratory component(s). To assess the changes in gross anatomy lecture and laboratory instruction, assessment, and teaching resources utilized as a result of Covid-19, a survey was distributed to gross anatomy educators through professional associations and listservs. Of the 67 survey responses received for the May–August 2020 academic period, 84% were from United States (US) institutions, while 16% were internationally based. Respondents indicated that in-person lecture decreased during Covid-19 (before: 76%, during: 8%, P < 0.001) and use of cadaver materials declined (before: 76 ± 33%, during: 34 ± 43%, P < 0.001). The use of cadaver materials in laboratories decreased during Covid-19 across academic programs, stand-alone and integrated anatomy courses, and private and public institutions (P ≤ 0.004). Before Covid-19, cadaveric materials used in laboratories were greater among professional health programs relative to medical and undergraduate programs (P ≤ 0.03) and among stand-alone relative to integrated anatomy courses (P ≤ 0.03). Furthermore, computer-based assessment increased (P < 0.001) and assessment materials changed from cadaveric material to images (P < 0.03) during Covid-19, even though assessment structure was not different (P > 0.05). The use of digital teaching resources increased during Covid-19 (P < 0.001), with reports of increased use of in-house created content, BlueLink, and Complete Anatomy software (P < 0.05). While primarily representing US institutions, this study provided evidence of how anatomy educators adapted their courses, largely through virtual mediums, and modified laboratory protocols during the initial emergence of the Covid-19 pandemic.     

Mexican-Origin Adolescent Mothers’ Economic Contexts,Educational Re-Engagement,and Their Children’s School Readiness

Drawing on data from a longitudinal study of 204 Mexican-origin adolescent mothers, their mother figures, and their children, the current investigation examined (a) adolescent mothers’ educational re-engagement and attainment beginning during their pregnancy and ending when their child was 5 years old; and (b) the influence of the family economic context on adolescent mothers’ educational re-engagement and attainment and their children’s academic and social-emotional outcomes. Findings detailed adolescent mothers’ re-engagement in school after the birth of their child and revealed that family income during adolescents’ pregnancies was directly associated with re-engagement and attainment, and also initiated cascade effects that shaped adolescents’ economic contexts, their subsequent re-engagement and attainment, and ultimately their children’s academic and social-emotional outcomes at age 5.