Sources of difficulty in assessment: example of PISA science items

ABSTRACT

The understanding of what makes a question difficult is a crucial concern in assessment. To study the difficulty of test questions, we focus on the case of PISA, which assesses to what degree 15-year-old students have acquired knowledge and skills essential for full participation in society. Our research question is to identify PISA science item characteristics that could influence the item’s proficiency level. It is based on an a-priori item analysis and a statistical analysis. Results show that only the cognitive complexity and the format out of the different characteristics of PISA science items determined in our a-priori analysis have an explanatory power on an item’s proficiency levels. The proficiency level cannot be explained by the dependence/independence of the information provided in the unit and/or item introduction and the competence. We conclude that in PISA, it appears possible to anticipate a high proficiency level, that is, students’ low scores for items displaying a high cognitive complexity. In the case of a middle or low cognitive complexity level item, the cognitive complexity level is not sufficient to predict item difficulty. Other characteristics play a crucial role in item difficulty. We discuss anticipating the difficulties in assessment in a broader perspective.     

Nothing about us without us! Youth-led solutions to improve high school completion rates

This Youth Participatory Action Research (YPAR) study represents a collaboration with six students from alternative education to inquire about the experiences of vulnerable youth – students in alternative education and youth who have dropped out of school. Utilizing the Enhanced Critical Incident Technique, youth researchers asked their peers what helped and hindered their retention and success in mainstream and alternative education. Youth researchers engaged in authentic participation and took part in the iterative phases of YPAR – critical reflection and social action. Their involvement empowered them to advocate for their peers by disseminating the results and recommendations to key stakeholders within the community. Youth researchers conducted semi-structured interviews with 18 participants. Overall, the findings show that relationships with staff and peers, flexibility, psychosocial and academic supports, and personal circumstances are vital in helping vulnerable students succeed in school. Engagement in YPAR provided insight into working with vulnerable youth in a manner that promotes agency and social change within educational institutions.     

Acceleration: A coat of many colours

The word “acceleration” continues to generate controversy. Although the literature makes it clear that acceleration in its various guises is a positive intervention for many gifted students, educational practitioners in Australia continue to resist the evidence and campaign actively against allowing students to take this option. This article synthesizes 3 research projects conducted in New South Wales, Australia, exploring different forms of acceleration. The first project involved an investigation of the Early Entry policy for gifted children in one region of the state; the second reports on the experiences of students who skipped at least 1 grade; and the third examines a vertical programming system that allows students to accelerate within subjects at an academically selective high school. Based on these research studies, this article explores the issues that are at the center of the acceleration debate.     

Behind the Findings: Yes,the Science Explorations Program Worked,but Why?

Abstract The Science Explorations program was developed by the Milwaukee Public Museum (MPM) out of a desire to use the unique resources of MPM to advance informal science education and to address a community need of local and national concern: improving science education and accessibility for underserved audiences. In 2002, with support from the National Science Foundation (NSF) and private donors, MPM launched this after‐school program for a target group of urban, mostly minority, middle school girls, a group at risk for underachievement in science and technology. The museum staff built a combined program with five middle schools and also sought to reach out to family members of the participating girls in order to increase support for the young women's science endeavors. A three‐year evaluation of the Science Explorations program demonstrated positive findings from primarily quantitative data. An aim of this article is to present findings from the qualitative data to shed light on the reasons this program met nearly all of its targets. Findings from case studies and qualitative interviews suggest that the museum staff's efforts to demystify science—a process that provided ongoing access to real scientific endeavors and invited personal contact with scientists—influenced the program's success. Findings also suggest that strong school liaisons may help increase family support for young women's scientific pursuits, which can in turn play a role in their success in this program.     

Development of Gender Typicality and Felt Pressure in European French and North African French Adolescents

Trajectories of gender identity were examined from Grade 6 (M_age = 11.9 years) to Grade 9 in European French (n = 570) and North African French (n = 534) adolescents, and gender and ethnic group differences were assessed in these trajectories. In Grade 6, boys of both ethnic groups reported higher levels of gender typicality and felt pressure for gender conformity than girls. European French girls and boys and North African French girls reported decreasing gender typicality from Grade 6 to Grade 9, whereas North African French boys did not change. Felt pressure decreased among girls, did not change in European French boys, and increased in North African French boys. Ethnic and gender differences in gender identity development are discussed.