Untangling citation networks

The pattern of citation links between articles permits a placement in a space of points representing these articles. Using the rule that articles are placed at the centroid of the articles to which they are linked, pictures can be drawn revealing key papers and their descendants. This simple rule for placing papers by their citation pattern may also be used to scale other citation-based data. The example used in this paper is the placement of overlapping groups of articles linked by shared articles.     

A Computer-Based Task for Measuring the Representational Component of Quantitative Proficiency

In this study, we created a computer-delivered problem-solving task based on the cognitive research literature and investigated its validity for graduate admissions assessment. The task asked examinees to sort mathematical word problem stems according to prototypes. Data analyses focused on the meaning of sorting scores and examinee perceptions of the task. Results showed that those who sorted well tended to have higher GRE General Test scores and college grades than did examinees who sorted less proficiently. Examinees generally preferred this task to multiple-choice items like those found on the General Test's Quantitative section and felt the task was a fairer measure of their ability to succeed in graduate school. Adaptations of the task might be used in admissions tests, as well as for instructional assessments to help lower- scoring examinees localize and remediate problem-solving difficulties.     

The 3 × 2 Achievement Goal Questionnaire for Teachers

Within the past decade, the achievement goal approach has begun to be used to study teacher achievement motivation. In recent research with students, a 3 × 2 model of achievement goals has been proffered that separates mastery-based goals in terms of a task/self distinction. The purpose of the present study was to extend this 3 × 2 model to teachers in order to provide important insights into teachers’ individual differences. We devised items for the questionnaire and showed that data obtained with the 3 × 2 Achievement Goal Questionnaire for Teachers fit the 3 × 2 model (in absolute terms and relative to 10 alternative models) and that each goal of the model exhibited good internal consistency. In addition, we linked the 3 × 2 goals to other key constructs relevant to achievement goals, namely, implicit theories of ability, instructional practices and intrinsic interest.     

Exploring the Co‐Development of Reading Fluency and Reading Comprehension: A Twin Study

This study explores the co‐development of two related but separate reading skills, reading fluency and reading comprehension, across Grades 1–4. A bivariate biometric dual change score model was applied to longitudinal data collected from 1,784 twin pairs between the ages of 6 and 10 years. Grade 1 skills were influenced by highly overlapping genetic and environmental factors. Growth in both skills was influenced by highly overlapping shared environmental factors. Cross‐lagged parameters indicated bidirectional effects, with stronger effects from fluency to comprehension change than from comprehension to fluency change.     

Differential Item Functioning on the SAT-M Braille Edition

This study attempted to pinpoint the causes of differential item difficulty for blind students taking the braille edition of the Scholastic Aptitude Test's Mathematical section (SAT-M). The study method involved reviewing the literature to identify factors that might cause differential item functioning for these examinees, forming item categories based on these factors, identifying categories that functioned differentially, and assessing the functioning o f the items comprising deviant categories to determine if the differential effect was pervasive. Results showed an association between selected item categories and differential functioning, particularly for items that included figures in the stimulus, items for which spatial estimation was helpful in eliminating at least two of the options, and items that presented figures that were small or medium in size. The precise meaning of this association was unclear, however, because some items from the suspected categories functioned normally, factors other than the hypothesized ones might have caused the observed aberrant item behavior, and the differential difficulty might reflect real population differences in relevant content knowledge     

Scoring Constructed Responses Using Expert Systems

The use of constructed-response items in large scale standardized testing has been hampered by the costs and difficulties associated with obtaining reliable scores. The advent of expert systems may signal the eventual removal of this impediment. This study investigated the accuracy with which expert systems could score a new, nonmultiple-choice item type. The item type presents a faulty solution to a computer programming problem and asks the student to correct the solution. This item type was administered to a sample of high school seniors enrolled in an Advanced Placement course in Computer Science who also took the Advanced Placement Computer Science (APCS) examination. Results indicated that the expert systems were able to produce scores for between 82% and 95% of the solutions encountered and to display high agreement with a human reader on the correctness of the solutions. Diagnoses of the specific errors produced by students were less accurate. Correlations with scores on the objective and free-response sections of the APCS examination were moderate. Implications for additional research and for testing practice are offered.     

SAT Differential Item Performance for Nine Handicapped Groups

The purpose of this study was to identify broad classes of items that behave differentially for handicapped examinees taking special, extended-time administrations of the Scholastic Aptitude Test (SA T). To identify these item classes, the performance of nine handicapped groups and one nonhandicapped group on each of two forms of the SAT was investigated through a two-stage procedure. The first stage centered on the performance of item clusters. Individual items composing clusters showing questionable performance were then examined. This two-stage procedure revealed little indication of differentially functioning item classes. However, some notable instances of differential performance at the item level were detected, the most serious of which affected visually impaired students taking the braille edition of the test.     

Exploring Differences in Student Learning and Behavior Between Real-life and Virtual Reality Chemistry Laboratories

Recent global events and educational trends have led schools to heavily rely on digital media to educate their students. Science classes, in particular, stand to lose substantial learning opportunities without the ability to provide physical laboratory experiences. Virtual reality (VR) technology has the potential to resolve this issue, but little is known if VR environments can produce similar results to real-life (RL) science learning environments. This 2?×?1, between-subjects study compares students’ learning results and safety behaviors in VR and RL chemistry laboratories. The study attempts to identify differences in learning experience (i.e., general chemistry content, experiment comprehension, laboratory safety knowledge) and laboratory safety behavior. Results indicate learning general content knowledge, laboratory skills, and procedure-related safety behaviors were comparable between RL and VR conditions, but clean-up behaviors were less frequent in VR. Also, the exploratory, risk-free nature of VR environments may have allowed the learners to elaborate and reflect more on general chemistry content and laboratory safety knowledge than in the RL environment.

     

Psychometric and Cognitive Functioning of an Under-Determined Computer-Based Response Type for Quantitative Reasoning

We evaluated a computer-delivered response type for measuring quantitative skill. "Generating Examples" (GE) presents under-determined problems that can have many right answers. We administered two GE tests that differed in the manipulation of specific item features hypothesized to affect difficulty. Analyses related to internal consistency reliability, external relations, and features contributing to item difficulty, adverse impact, and examinee perceptions. Results showed that GE scores were reasonably reliable but only moderately related to the GRE quantitative section, suggesting the two tests might be tapping somewhat different skills. Item features that increased difficulty included asking examinees to supply more than one correct answer and to identify whether an item was solvable. Gender differences were similar to those found on the GRE quantitative and analytical test sections. Finally, examinees were divided on whether GE items were a fairer indicator of ability than multiple-choice items, but still overwhelmingly preferred to take the more conventional questions.