Gender DIF in Reading and Mathematics Tests With Mixed Item Formats

This was a study of differential item functioning (DIF) for grades 4, 7, and 10 reading and mathematics items from state criterion-referenced tests. The tests were composed of multiple-choice and constructed-response items. Gender DIF was investigated using POLYSIBTEST and a Rasch procedure. The Rasch procedure flagged more items for DIF than did the simultaneous item bias procedure—particularly multiple-choice items. For both reading and mathematics tests, multiple-choice items generally favored males while constructed-response items generally favored females. Content analyses showed that flagged reading items typically measured text interpretations or implied meanings; males tended to benefit from items that asked them to identify reasonable interpretations and analyses of informational text. Most items that favored females asked students to make their own interpretations and analyses, of both literary and informational text, supported by text-based evidence. Content analysis of mathematics items showed that items favoring males measured geometry, probability, and algebra. Mathematics items favoring females measured statistical interpretations, multistep problem solving, and mathematical reasoning.     

Gender differences in mathematics achievement: an investigation of gender differences by item difficulty interactions

This paper presents findings from research exploring gender by item difficulty interaction on mathematics test scores in Cyprus. Data steamed from 2 longitudinal studies with 4 different age groups of primary school students. The hypothesis that boys tended to outperform girls on the hardest items and girls tended to outperform boys on the easiest items was generally supported for each year group. The effect of social class was also examined. For each social class, there was a correlation between the item difficulty differences estimated on girls and boys separately and the difficulty of the item estimated on the whole sample. It is claimed that in understanding gender differences in mathematics, item difficulty should be treated as an independent variable. Suggestions for further studies are provided, and implications for the development of assessment policy in mathematics are drawn.     

Achievement as a Function of Test Item Complexity and Difficulty

The effects of training tests on subsequent achievement were studied using 2-test item characteristics: item difficulty and item complexity. Ninety Ss were randomly assigned to treatment conditions having easy or difficult items and calling for rote or complex skills. Each S was administered two training tests during the quarter containing only items defined by his treatment condition. The dependent measure was a sixty item final examination with fifteen items reflecting each of the four treatment condition item types. The results showed greater achievement for those trained with difficult items and with rote items. In addition, two interaction of treatment conditions with type of test items were found. The results are discussed as supporting a hierarchical model rather than a “similarity” transfer model of learning.     

The Relationship Between Item Parameters and Item Fit

The effect of item parameters (discrimination, difficulty, and level of guessing) on the item-fit statistic was investigated using simulated dichotomous data. Nine tests were simulated using 1,000 persons, 50 items, three levels of item discrimination, three levels of item difficulty, and three levels of guessing. The item fit was estimated using two fit statistics: the likelihood ratio statistic (X²_B), and the standardized residuals (SRs). All the item parameters were simulated to be normally distributed. Results showed that the levels of item discrimination and guessing affected the item-fit values. As the level of item discrimination or guessing increased, item-fit values increased and more items misfit the model. The level of item difficulty did not affect the item-fit statistic.     

Is the Item-Position Effect in Achievement Measures Induced by Increasing Item Difficulty?

This work examines the hypothesis that the arrangement of items according to increasing difficulty is the real source of what is considered the item-position effect. A confusion of the 2 effects is possible because in achievement measures the items are arranged according to their difficulty. Two item subsets of Raven’s Advanced Progressive Matrices (APM), one following the original item order, and the other one including randomly ordered items, were applied to a sample of 266 students. Confirmatory factor analysis models including representations of both the item-position effect and a possible effect due to increasing item difficulty were compared. The results provided evidence for both effects. Furthermore, they indicated a substantial relation between the item-position effects of the 2 APM subsets, whereas no relation was found for item difficulty. This indicates that the item-position effect stands on its own and is not due to increasing item difficulty.     

Italian students’ results in the PISA mathematics test: does reading competence matter?

In Italy, from the 2003 reports to the present, the National Institute for the Educational Evaluation of Instruction and Training (INVALSI) has conducted research on Programme for International Student Assessment (PISA) results in order to understand Italian students’ low achievement in mathematics. In the present paper, data from a representative sample of 15-year-old Italian students who participated in PISA 2012 were analysed. This study’s primary aim is to verify how students’ linguistic competences are associated with their performance in mathematics. For the evaluation of the impact of item reading demand on students’ performance, we selected 24 mathematics items with a high reading demand and 31 mathematics items with a low reading demand, as classified by Italian language and methodology experts. Repeated measure variance analyses were conducted. The results showed differences in function of gender: females are advantaged in mathematics items with a high reading demand, independent of their level of reading literacy. In contrast, males are advantaged in mathematics items with a low reading demand, independent of their level of reading literacy. Possible policy implications are discussed.     

The effect of simulation games and worksheets on learning of varying ability groups in a high school biology classroom

High school biology students who were taught by the simulation game method had comparable achievement gains to the students who were taught using worksheets. The three simulation games used in this study were able to teach factual information as well as the worksheet activities. This effect was constant across ability and sex. There were no significant differences in the retention scores of high ability students utilizing gaming. Also, there was no significant difference in the scores of low ability students utilizing worksheets and low ability students using games. Students' sex was significantly related with retention in the three-way interaction of treatment by ability by sex. Low ability females using simulation games scored higher on retention than low ability females utilizing worksheets. Low ability males utilizing worksheets scored higher on retention than low ability males using simulation games.     

Are effective or meritorious schools meretricious?

Data on student achievement gain in a large urban school district during the 1984–85 school year were analyzed to identify schools with achievement consistently higher or lower than schools with similar student and school characteristics. Before calculating gain scores, students were grouped with others of the same sex, ethnicity, initial grade level, and initial achievement level. After residuals for students who were above or below similar students were summed and averaged for reading and math at each grade, school-level regression analysis was used to further control for student and school characteristics. Results indicated that schools with reading gains one standard deviation above the mean had average raw scores less than two items higher than the district average on the sixty-item reading test, while schools with math gains this large had raw scores little more than one item higher than the average on the 44-item math test. In addition, examination of data on schools with residual (combined) reading and math gain scores 1.5 or more standard deviations above the average showed that after taking account of student background information and salient school characteristics, only one elementary school among 173 and no secondary school among 71 had unusually high achievement scores. Implications are discussed regarding efforts to identify and reward meritorious schools which allegedly have improved achievement more than other similar schools.     

命题者：影响阅读理解测试效度的一个因素

本研究应用项目反应理论,从被试的阅读能力值和题目的难度值这两个方面,分析阅读理解测试中多项选择题命题者对考试效度的影响。实验设计中,将两组被试同时施测于一项“阅读水平测试”,根据测试结果估计出的两组被试能力值之间无显著性差异。再次将这两组被试分别施测于两位不同命题者所命制的题目,尽管这些题目均产生于相同的阅读材料,且题目的难度值之间并没有显著性差异,被试的表现却显著不同。Rasch模型认为,被试表现由被试能力和试题难度共同决定。因此,可以推测,这是由于不同命题者所命制的题目影响了被试的表现,并进而影响了使用多项选择题进行阅读理解测试的效度。     

Achievement Attributions of Sixth Grade Children and Their Parents

The present study examines the achievement attributions of Greek Cypriot students and their parents. Its aim was to investigate the role of parental and child achievement attributions as parameters of the child's actual school achievement and to examine the existing differences between attributions made by children and their parents. A total of 477 Sixth Grade Greek Cypriot students and their parents participated in the study. A structural equation model was constructed and its ability to fit the data was tested. It was found that child attributions of achievement to effort, ability and other internal factors are positively related to actual achievement, while attributions to luck and external factors are negatively related to achievement. This is in line with earlier findings. Parental and child attributions are not strongly and reliably related. Thus, claims that children develop their own attributions on the basis of their parents’ attributions were not supported. Gender differences were found, with females attributing their achievement to effort more than males did. Finally, underachievers tended to attribute their school performance to external factors (luck, role of others such as parents and teachers), while highly achieving students tended to attribute their performance to their own effort and other internal factors.     

Relationships between psychometric dimensions of item quality and student ratings of item relevancy and ambiguity

Students rated the quality of the items on a classroom test that had been taken previously. On the same test, psychometric item indices were calculated. The results showed that the student ratings were related to the item difficulty, but not to the item-test correlation. In addition, the better-achieving students tended to rate the items as less ambiguous. Finally, the ambiguity ratings were more highly related to the item-test correlations for the better achieving students. These findings support opinions held by many instructors of students' judgments of item quality.     

Gender Differences on the Math Subtest of the Scholastic Aptitude Test may be Culture-Specific

Cross-cultural studies can shed new light on theories of gender differences in cognition. In the present study, Chinese students were given items from the math subtest of the Scholastic Aptitude Test (SAT) that have been found to produce the largest gender differences in American students. The authors describe how four different explanations of gender differences make different predictions regarding the possible size of the gender difference in Chinese students. Consistent with the Differential Coursework view but contrary to the predictions of several other views, the results revealed no difference in performance on the SAT items between Chinese males and females.     

SEX DIFFERENCES IN THE TAYLOR MANIFEST ANXIETY SCALE

A study comparing the performance of males and females on the Taylor Manifest Anxiety Scale extended the previous findings of a differential response pattern for males and females to a Canadian high school population. An item analysis identified 12 of the 50 items as being responsible for the higher female scores. The scores for Canadian high school students in this study were found to fall midway between those of university students and those of psychiatric patients tested in previous studies.     

Advanced Math Course Taking: Effects on Math Achievement and College Enrollment

Using data from the Educational Longitudinal Study of 2002–2006, the authors investigated the effects of advanced math course taking on math achievement and college enrollment and how such effects varied by socioeconomic status and race/ethnicity. Results from propensity score matching and sensitivity analyses showed that advanced math course taking had positive effects on math achievement and college enrollment. Results also demonstrated that the effect of advanced math course taking on math achievement was greater for low socioeconomic status students than for high socioeconomic status students, but smaller for Black students than for White students. No interaction effects were found for college enrollment. Limitations, policy implications, and future research directions are discussed.     

Item Function Characteristics and Dimensionality for Alternative Response Formats in Mathematics

The purpose of this study was to investigate the technical properties of stem-equivalent mathematics items differing only with respect to response format. Using socio- economic factors to define the strata, a proportional stratified random sample of 1,366 Connecticut sixth-grade students were administered one of three forms. Classical item analysis, dimensionality assessment, item response theory goodness-of-fit, and an item bias analysis were conducted. Analysis of variance and confirmatory factor analysis were used to examine the functioning of the items presented in the three different formats. It was found that, after equating forms, the constructed-response formats were somewhat more difficult than the multiple-choice format. However, there was no significant difference across formats with respect to item discrimination. A differential item functioning (DIF) analysis was conducted using both the Mantel-Haenszel procedure and the comparison of the item characteristic curves. The DIF analysis indicated that the presence of bias was not greatly affected by item format; that is, items biased in one format tended to be biased in a similar manner when presented in a different format, and unbiased items tended to remain so regardless of format.     

Gender peer effects in university: Evidence from a randomized experiment

Recent studies for primary and secondary education find positive effects of the share of females in the classroom on achievement of males and females. This study examines whether these results can be extrapolated to higher education. We conduct an experiment in which the shares of females in workgroups for first year students in economics and business are manipulated and students are randomly assigned to these groups. Males tend to postpone, but not abandon, their dropout decision when surrounded by more females and perform worse on courses with high math content. There is also a modest reduction in absenteeism early in the year. Overall, however, we find no substantial gender peer effects on achievement. This in spite of the fact that according to students’ perceptions, both their own, and their peers’ behavior are influenced by the share of females.     

Associations between Fine Motor and Mathematics Instruction and Kindergarten Mathematics Achievement

Research Findings: This study investigates the role of fine motor and mathematics instruction in mathematics achievement in an international sample of kindergarteners from the United States and China. Multilevel modeling was used to assess the interaction between students’ entering skills and classroom time spent on basic math, higher-order math and fine-motor instruction. For American children, the effect of basic math and higher-order math instruction on student achievement depended on entering skills; however, fine motor instruction had negative average effects on student achievement and did not depend on students’ entering skills. Instruction time was not a significant predictor of achievement for Chinese students. Practice or Policy: Though fine motor skills have a robust correlation with mathematics achievement, a causal link has not been established. Our study indicates that time spent in fine motor instruction does not advance mathematics achievement in kindergarten and in fact may weaken mathematics achievement, given the limited time in the instructional day. American teachers in our sample who spent more time in fine-motor instruction tended to spend less time on basic math and higher-order mathematics instruction. Educators should weigh instructional trade-offs carefully and work to tailor instruction to students’ skill levels.     

Mobile Gaming and Student Interactions in a Science Center: the Future of Gaming in Science Education

This article explores the impact of an augmented reality iPad-based mobile game, called The Great STEM Caper, on students’ interaction at a science center. An open-source, location-based game platform called ARIS (i.e. Augmented Reality and Interactive Storytelling) was used to create an iPad-based mobile game. The game used QR scan codes and a challenge-based game structure to encourage engagement in science. Participants wore head-mounted GoPro cameras to record interactions within the physical and social environment. The purpose of this research was to study the impact of a mobile game on student interactions in a science center. Gender differences in gameplay behaviors and perceptions were compared. The study included a quasi-experimental design with two groups: one group that played the iPad mobile game during their science center visit, and one group that explored the science center in a traditional free exploration manner. Video data from the GoPro cameras provided examination of the different ways students interacted with the science center. The female students outperformed the male students on every measure of the iPad game achievement. Lazzaro’s (2004) four types of fun was used to describe the gender differences in game perceptions and interactions. Females tended to enjoy hard fun and collaborative people fun, while males enjoyed easy fun and competitive people fun. The females tended to be more goal-oriented, persistent in the face of difficulty, and appreciative of hard fun. The results of this study have implications for the design of mobile-based gaming technology to encourage interaction in an informal science center setting.     

Can Item Response Times Provide Insight Into Students’ Motivation and Self‐Efficacy in Math? An Initial Application of Test Metadata to Understand Students’ Social–Emotional Needs

As computer‐based tests become more common, there is a growing wealth of metadata related to examinees’ response processes, which include solution strategies, concentration, and operating speed. One common type of metadata is item response time. While response times have been used extensively to improve estimates of achievement, little work considers whether these metadata may provide useful information on social–emotional constructs. This study uses an analytic example to explore whether metadata might help illuminate such constructs. Specifically, analyses examine whether the amount of time students spend on test items (after accounting for item difficulty and estimates of true achievement), and difficult items in particular, tell us anything about the student's academic motivation and self‐efficacy. While results do not indicate a strong relationship between mean item durations and these constructs in general, the amount of time students spend on very difficult items is highly correlated with motivation and self‐efficacy. The implications of these findings for using response process metadata to gain information on social–emotional constructs are discussed.     

Mathematics achievement and gender: A longitudinal study of selected cognitive and affective variables [grades 6–12]

This study examined the relationship of selected cognitive and affective variables to mathematics achievement for a random sample of 60 students as they progressed through 6th, 8th,10th and 12th grades. A consistent gender difference was found for stereotyping mathematics as a male domain. No consistent significant gender difference between means was found for spatial skills, verbal skill or mathematics achievement. Confidence, verbal skill and spatial visualization were each consistently positively correlated with mathematics achievement for both males and females. No gender difference was found for these correlations. However, spatial skills alone were found to be consistent significant predictors of mathematics achievement for females each year of the study, but not for males. Verbal skill was a consistent significant predictor of mathematics achievement for males, but not for females. The results of this study could lead to a reevaluation of the hypothesis that spatial skills help boys achieve in mathematics.