Equivalence of Free-Response and Multiple-Choice Items

This study examined the relationship of multiple-choice and free-response items contained on the College Board's Advanced Placement Computer Science (APCS) examination. Confirmatory factor analysis was used to test the fit of a two-factor model where each item format marked its own factor. Results showed a single-factor solution to provide the most parsimonious fit in each of two random-half samples. This finding might be accounted for by several mechanisms, including overlap in the specific processes assessed by the multiple-choice and free-response items and the limited opportunity for skill differentiation afforded by the year-long APCS course.     

Childhood acquired brain injury: the knowledge and training needs of special educational needs coordinators

The aim of this study was to further explore Special Educational Need Co-ordinators' (SENCos) knowledge of childhood acquired brain injury (ABI) and if they have received training on how to effectively support children and young people (CYP) with an ABI in school. SENCos from Nottinghamshire were asked to complete a survey face-to-face or online. Data reported by Howe and Ball (Support for Learning, 32, 1, 85–100), was also used to allow comparisons between different counties in the UK for knowledge of childhood ABI. Results indicated that SENCos from Nottinghamshire hold numerous uncertainties about childhood ABI, although less uncertainties than SENCos from the West Midlands. A majority SENCos from Nottinghamshire had not received training about childhood ABI. Additional challenges in supporting CYP with an ABI were also identified. The findings show a clear need for more training on childhood ABI across UK schools. It is also apparent that obtaining funding for CYP with an ABI can be a challenge for SENCos. However, further research is needed to determine what these barriers to funding are.     

Determining Gain of Selected Learnings in Science Using a Specific Biological Unit

The purpose of this study was to investigate the factor structure of an inventory designed to sample preservice teachers’ opinions about children’s nature and needs, classroom discipline, subject matter, and appropriate teacher behavior. Image component analysis yielded three interpretable factors. The first two factors, accounting for most of the variance, tend to confirm earlier findings that “progressivism” and “traditionalism” are independent attitudinal dimensions, rather than opposite poles of a single continuum.     

Leg Muscle Metabolism in Trained and Untrained Men

The purpose of this study was to describe substrate changes in blood and muscles during submaximal exercise and to look at the relationship of these changes to some enzyme activities important in lipid and carbohydrate metabolism. A group of six trained and six untrained men were exercised for 1 hour at 50% [Vdot]O₂ max; three from each group cycled and the rest ran on a treadmill. During the exercise, blood-free fatty acid (FFA) levels increased twofold, glycerol by 3.6 times, triglyceride (TG) by 11/100 ml, and glucose by a mean 11 mg/100 ml in the trained men. In the untrained subjects, however, FFA increased 1.8 fold, glycerols 1.6 fold, and TG by a mean 24/100 ml. Glucose fell in every subject by an average 5 mg%. Muscle glycogen depletion averaged 34 mmoles/kg in the untrained and 23 mmoles/kg in the trained group. The activities of succinic acid dehydrogenase (SDH), malic acid dehydrogenase (MDH), and carnitine palmityl transferase (CPT) were greater by 177%, 93%, and 64%, respectively, in the trained group than in the untrained subjects. The results indicated that at the same percent [Vdot]O₂ max, the trained men seemed to “preferentially” use FFA as an energy source, possibly due to a well developed Krebs cycle and increased CPT activities. This preference would tend to inhibit the major rate-limiting enzymes of glycolysis, thus slowing even further the use of carbohydrates as an energy source. The differences in response to exercise were independent of fiber type and indicate that these catabolic responses were primarily due to aerobic training.     

Children's Understanding of Knowledge Acquisition: The Tendency for Children to Report That They Have Always Known What They Have Just Learned

Children's attention to knowledge-acquisition events was examined in 4 experiments in which children were taught novel facts and subsequently asked how long they had known the new information. In Experiment 1, 4- and 5-year-olds tended to claim they had known novel animal facts for a long time and also reported that other children would know the novel facts. This finding was replicated in Experiment 2, using facts associated with chemistry demonstrations. In Experiments 3 and 4, children were taught new color words. 5-year-olds, but not 4-year-olds, distinguished between novel and familiar color words, reporting they had not known the novel words before the test session, but they had always known the familiar words. 4-year-olds in Experiment 4 were better able to distinguish novel and familiar color words when the teaching of the novel words was an explicit and salient part of the procedure.     

Determinants of Impact: Towards a Better Understanding of Encounters with the Arts

The article argues that current methods for assessing the impact of the arts are largely based on a fragmented and incomplete understanding of the cognitive, psychological and socio-cultural dynamics that govern the aesthetic experience. It postulates that a better grasp of the interaction between the individual and the work of art is the necessary foundation for a genuine understanding of how the arts can affect people. Through a critique of philosophical and empirical attempts to capture the main features of the aesthetic encounter, the article draws attention to the gaps in our current understanding of the responses to art. It proposes a classification and exploration of the factors—social, cultural and psychological—that contribute to shaping the aesthetic experience, thus determining the possibility of impact. The ‘determinants of impact’ identified are distinguished into three groups: those that are inherent to the individual who interacts with the artwork; those that are inherent to the artwork; and ‘environmental factors’, which are extrinsic to both the individual and the artwork. The article concludes that any meaningful attempt to assess the impact of the arts would need to take these ‘determinants of impact’ into account, in order to capture the multidimensional and subjective nature of the aesthetic experience.     

Cookies, web bugs, webcams and cue cats: Patterns of surveillance on the world wide web

This article addresses the question of whetherpersonal surveillance on the world wide web isdifferent in nature and intensity from that inthe offline world. The article presents aprofile of the ways in which privacy problemswere framed and addressed in the 1970s and1990s. Based on an analysis of privacy newsstories from 1999–2000, it then presents atypology of the kinds of surveillance practicesthat have emerged as a result of Internetcommunications. Five practices are discussedand illustrated: surveillance by glitch,surveillance by default, surveillance bydesign, surveillance by possession, andsurveillance by subject. The article offerssome tentative conclusions about theprogressive latency of tracking devices, aboutthe complexity created by multi-sourcing, aboutthe robustness of clickstream data, and aboutthe erosion of the distinction between themonitor and the monitored. These trendsemphasize the need to reject analysis thatframes our understanding of Internetsurveillance in terms of its impact onsociety. Rather the Internet should beregarded as a form of life whose evolvingstructure becomes embedded in humanconsciousness and social practice, and whosearchitecture embodies an inherent valence thatis gradually shifting away from the assumptionsof anonymity upon which the Internet wasoriginally designed.     

In practice ‐ Venturing into scary places: The minority experience

Students who have lived as part of a majority for their entire life may find it frightening to be asked to visit a place where they will be in the minority. Craig Bennett uses this assignment to demonstrate to his students how some people feel every day.     

A systematic review of empirical studies using log data from open-ended learning environments to measure science and engineering practices

Technology-based, open-ended learning environments (OELEs) can capture detailed information of students' interactions as they work through a task or solve a problem embedded in the environment. This information, in the form of log data, has the potential to provide important insights about the practices adopted by students for scientific inquiry and problem solving. How to parse and analyse the log data to reveal evidence of multifaceted constructs like inquiry and problem solving holds the key to making interactive learning environments useful for assessing students' higher-order competencies. In this paper, we present a systematic review of studies that used log data generated in OELEs to describe, model and assess scientific inquiry and problem solving. We identify and analyse 70 conference proceedings and journal papers published between 2012 and 2021. Our results reveal large variations in OELE and task characteristics, approaches used to extract features from log data and interpretation models used to link features to target constructs. While the educational data mining and learning analytics communities have made progress in leveraging log data to model inquiry and problem solving, multiple barriers still exist to hamper the production of representative, reproducible and generalizable results. Based on the trends identified, we lay out a set of recommendations pertaining to key aspects of the workflow that we believe will help the field develop more systematic approaches to designing and using OELEs for studying how students engage in inquiry and problem-solving practices.

Practitioner notes

What is already known about this topic

• Research has shown that technology-based, open-ended learning environments (OELEs) that collect users' interaction data are potentially useful tools for engaging students in practice-based STEM learning.
• More work is needed to identify generalizable principles of how to design OELE tasks to support student learning and how to analyse the log data to assess student performance.

What this paper adds

• We identified multiple barriers to the production of sufficiently generalizable and robust results to inform practice, with respect to: (1) the design characteristics of the OELE-based tasks, (2) the target competencies measured, (3) the approaches and techniques used to extract features from log files and (4) the models used to link features to the competencies.
• Based on this analysis, we can provide a series of specific recommendations to inform future research and facilitate the generalizability and interpretability of results:
  • Making the data available in open-access repositories, similar to the PISA tasks, for easy access and sharing.
  • Defining target practices more precisely to better align task design with target practices and to facilitate between-study comparisons.
  • More systematic evaluation of OELE and task designs to improve the psychometric properties of OELE-based measurement tasks and analysis processes.
  • Focusing more on internal and external validation of both feature generation processes and statistical models, for example with data from different samples or by systematically varying the analysis methods.

Implications for practice and/or policy

• Using the framework of evidence-centered assessment design, we have identified relevant criteria for organizing and evaluating the diverse body of empirical studies on the topic and that policy makers and practitioners can use for their own further examinations.
• This paper identifies promising research and development areas on the measurement and assessment of higher-order constructs with process data from OELE-based tasks that government agencies and foundations can support.
• Researchers, technologists and assessment designers might find useful the insights and recommendations for how OELEs can enhance science assessment through thoughtful integration of learning theories, task design and data mining techniques.