Differential Item Functioning on the SAT-M Braille Edition

This study attempted to pinpoint the causes of differential item difficulty for blind students taking the braille edition of the Scholastic Aptitude Test's Mathematical section (SAT-M). The study method involved reviewing the literature to identify factors that might cause differential item functioning for these examinees, forming item categories based on these factors, identifying categories that functioned differentially, and assessing the functioning o f the items comprising deviant categories to determine if the differential effect was pervasive. Results showed an association between selected item categories and differential functioning, particularly for items that included figures in the stimulus, items for which spatial estimation was helpful in eliminating at least two of the options, and items that presented figures that were small or medium in size. The precise meaning of this association was unclear, however, because some items from the suspected categories functioned normally, factors other than the hypothesized ones might have caused the observed aberrant item behavior, and the differential difficulty might reflect real population differences in relevant content knowledge     

SAT Differential Item Performance for Nine Handicapped Groups

The purpose of this study was to identify broad classes of items that behave differentially for handicapped examinees taking special, extended-time administrations of the Scholastic Aptitude Test (SA T). To identify these item classes, the performance of nine handicapped groups and one nonhandicapped group on each of two forms of the SAT was investigated through a two-stage procedure. The first stage centered on the performance of item clusters. Individual items composing clusters showing questionable performance were then examined. This two-stage procedure revealed little indication of differentially functioning item classes. However, some notable instances of differential performance at the item level were detected, the most serious of which affected visually impaired students taking the braille edition of the test.     

Exploring the Co‐Development of Reading Fluency and Reading Comprehension: A Twin Study

This study explores the co‐development of two related but separate reading skills, reading fluency and reading comprehension, across Grades 1–4. A bivariate biometric dual change score model was applied to longitudinal data collected from 1,784 twin pairs between the ages of 6 and 10 years. Grade 1 skills were influenced by highly overlapping genetic and environmental factors. Growth in both skills was influenced by highly overlapping shared environmental factors. Cross‐lagged parameters indicated bidirectional effects, with stronger effects from fluency to comprehension change than from comprehension to fluency change.     

Exploring Differences in Student Learning and Behavior Between Real-life and Virtual Reality Chemistry Laboratories

Recent global events and educational trends have led schools to heavily rely on digital media to educate their students. Science classes, in particular, stand to lose substantial learning opportunities without the ability to provide physical laboratory experiences. Virtual reality (VR) technology has the potential to resolve this issue, but little is known if VR environments can produce similar results to real-life (RL) science learning environments. This 2?×?1, between-subjects study compares students’ learning results and safety behaviors in VR and RL chemistry laboratories. The study attempts to identify differences in learning experience (i.e., general chemistry content, experiment comprehension, laboratory safety knowledge) and laboratory safety behavior. Results indicate learning general content knowledge, laboratory skills, and procedure-related safety behaviors were comparable between RL and VR conditions, but clean-up behaviors were less frequent in VR. Also, the exploratory, risk-free nature of VR environments may have allowed the learners to elaborate and reflect more on general chemistry content and laboratory safety knowledge than in the RL environment.

     

Psychometric and Cognitive Functioning of an Under-Determined Computer-Based Response Type for Quantitative Reasoning

We evaluated a computer-delivered response type for measuring quantitative skill. "Generating Examples" (GE) presents under-determined problems that can have many right answers. We administered two GE tests that differed in the manipulation of specific item features hypothesized to affect difficulty. Analyses related to internal consistency reliability, external relations, and features contributing to item difficulty, adverse impact, and examinee perceptions. Results showed that GE scores were reasonably reliable but only moderately related to the GRE quantitative section, suggesting the two tests might be tapping somewhat different skills. Item features that increased difficulty included asking examinees to supply more than one correct answer and to identify whether an item was solvable. Gender differences were similar to those found on the GRE quantitative and analytical test sections. Finally, examinees were divided on whether GE items were a fairer indicator of ability than multiple-choice items, but still overwhelmingly preferred to take the more conventional questions.     

INSTRUCTOR AND STUDENT ATTITUDES TOWARD DISTANCE LEARNING

Eleven community college instructors and the 334 students in their distance learning classes were surveyed. Data showed instructors had conflicting attitudes about distance education. They were willing to teach a distance learning class again, but they rated the quality of the courses as equal or lower quality than other classes taught on campus. Their students, on the other hand, were highly satisfied with these instructors and the courses . But the critical factor in much of traditional classroom instruction, direct interaction with instructors, played no role in determining students' satisfaction in these courses. This difference in the nature of student-teacher interaction in distance learning classes may explain instructors' conflicting attitudes.     

A Computer-Based Task for Measuring the Representational Component of Quantitative Proficiency

In this study, we created a computer-delivered problem-solving task based on the cognitive research literature and investigated its validity for graduate admissions assessment. The task asked examinees to sort mathematical word problem stems according to prototypes. Data analyses focused on the meaning of sorting scores and examinee perceptions of the task. Results showed that those who sorted well tended to have higher GRE General Test scores and college grades than did examinees who sorted less proficiently. Examinees generally preferred this task to multiple-choice items like those found on the General Test's Quantitative section and felt the task was a fairer measure of their ability to succeed in graduate school. Adaptations of the task might be used in admissions tests, as well as for instructional assessments to help lower- scoring examinees localize and remediate problem-solving difficulties.     

Automated scoring in context: Rapid assessment for placed students

This study investigated the use of automated essay scoring (AES) to identify at-risk students enrolled in a first-year university writing course. An application of AES, the Criterion^® Online Writing Evaluation Service was evaluated through a methodology focusing on construct modelling, response processes, disaggregation, extrapolation, generalization, and consequence. Based on the results of our two-year study with students (N = 1,482) at a public technological research university in the United States, we found that Criterion offered a defined writing construct congruent with established models, achieved acceptance among students and instructors, showed no statistically significant differences between ethnicity groups of sufficient sample size, correlated at acceptable levels with other writing measures, performed in a stable fashion, and enabled instructors to identify at-risk students to increase their course success.     

Scoring Constructed Responses Using Expert Systems

The use of constructed-response items in large scale standardized testing has been hampered by the costs and difficulties associated with obtaining reliable scores. The advent of expert systems may signal the eventual removal of this impediment. This study investigated the accuracy with which expert systems could score a new, nonmultiple-choice item type. The item type presents a faulty solution to a computer programming problem and asks the student to correct the solution. This item type was administered to a sample of high school seniors enrolled in an Advanced Placement course in Computer Science who also took the Advanced Placement Computer Science (APCS) examination. Results indicated that the expert systems were able to produce scores for between 82% and 95% of the solutions encountered and to display high agreement with a human reader on the correctness of the solutions. Diagnoses of the specific errors produced by students were less accurate. Correlations with scores on the objective and free-response sections of the APCS examination were moderate. Implications for additional research and for testing practice are offered.     

Children's Social Reasoning in Context: Reply to Gabennesch

Children's social reasoning is multifaceted. Moral judgments of justice, welfare, and rights are an important aspect of domains of social reasoning that are not merely the arbitrary and relative products of social formations. In defining all social formations as conventions that are either reified or accurately perceived as arbitrary and relative human inventions, Gabennesch relegates the moral concepts of many philosophers, moral leaders, and laypersons to ethnocentrism and reification. In the process, he fails to distinguish between metaethics relating to the source of knowledge and the form of knowledge. He also fails to account for distinctions between conventionality and moral concepts that do not constitute realism or reification. His review of the evidence of moral realism in children is selective in that isolated findings are taken out of the context of a particular study and of the entire body of evidence. Moreover, he has incorrectly interpreted many of the findings he cites in support of childhood realism. We present a summary of 48 studies demonstrating that children distinguish morality and convention. Gabennesch's perceptual metaphors hinge on an exaggerated role for conventionality in social formations at the expense of other complex social phenomena.