Gender‐Based Differential Prediction by Curriculum Samples for College Admissions

A longstanding concern about admissions to higher education is the underprediction of female academic performance by admission test scores. One explanation for these findings is selection system bias, that is, not all relevant KSAOs that are related to academic performance and gender are included in the prediction model. One solution to this problem is to include these omitted KSAOs in the prediction model, many of these KSAOs are 'noncognitive' and “hard‐to‐measure” skills in a high‐stakes context. An alternative approach to capture relevant KSAOs is using representative performance samples. We examined differential prediction of first year‐ and third year academic performance by gender based on a curriculum‐sampling test that was designed as a small‐scale simulation of later college performance. In addition, we examined differential prediction using both frequentist and Bayesian analyses. Our results showed no differential prediction or small female underprediction when using the curriculum‐sampling tests to predict first year GPA, and no differential prediction for predicting third year GPA. In addition, our results suggest that more comprehensive curriculum samples may show less differential prediction. We conclude that curriculum sampling may offer a practically feasible method that yields minimal differential prediction by gender in high‐stakes operational selection settings.     

Examination of Gender Bias in University Admissions

In this study we examined whether the measures used in the admission of students to universities in Israel are gender biased. The criterion used to measure bias was performance in the first year of university study; the predictors consisted of an admission score, a high school matriculation score, and a standardized test score as well as its component subtest scores. Statistically, bias was defined according to the boundary conditions given in Linn (1984). No gender bias was detected when using the admission score (which is used for selection) as a predictor of first-year performance in the university. Bias in favor of women was found predominantly using school grades as predictor whereas bias against women was found predominantly in using the standardized test scores. It was concluded that the admission score is a valid and unbiased predictor of first-year university performance for the two genders.     

Selection for special education services: the role of gender and socio-economic status

Children from some demographic groups disproportionately receive special education (SE) services. Due to methodological shortcoming in existing work, it remains unclear whether this is due to real differences in academic needs or cultural selection/bias. Hence, in a community sample of 1250 Norwegian children, we examined the role of third grade SE services, academic test scores, behavioural problems, and teacher’s level of helplessness in mediating the effect of family socio-economic status (SES) and students’ gender on fifth-grade SE services. Results revealed no direct effects of either gender or SES on fifth-grade SE, but four mediated pathways were identified: (1) Boys had a greater likelihood of receiving fifth-grade SE services when (a) they previously received SE and (b) they experienced more ADHD symptoms, both in third grade. (2) Students from low-SES families were more likely to receive SE services in fifth grade when (a) they performed poorly in math in third grade and (b) their teacher reported greater feelings of helplessness when teaching these students. The findings are discussed with respect to the differential-needs hypothesis, mechanisms of cultural selection and the possibility of gendered selection for SE at younger ages.     

CHOICE OF VARIABLES AND GENDER DIFFERENTIATED PREDICTION WITHIN SELECTED ACADEMIC PROGRAMS

The existence of differential prediction of 2-year grade point average is reported for gender groups within programs of study at a large state university. Johnson-Neyman analyses indicate regions on the predictor score scales where differential prediction has practical impact within each program. A significant portion of students is generally affected. It is shown that careful selection of a subset of variables reduces differential prediction while maintaining predictive potency.     

美国ACT公平性析论

ACT在美国大学招生过程中发挥着重要的作用。作为大规模入学考试,ACT也是一个公平的考试。ACT在考试内容的选取上注重全面覆盖知识点并保证知识点与高中学科教学的相关性,这就使得所有考生拥有了公平的竞争机会;在试题命制过程中,ACT把公平性作为试题取舍的必要条件以保证试题不偏袒任何群体,这是考试公平的关键所在;分数报告系统的全面细致,使得考生之间的差异性在大学招生过程中能够得到恰当的对待,这是考试公平必不可少的组成部分。     

Who Should Go to University? Justice in University Admissions

Current debates regarding justice in university admissions most often approach the question of access to university from a technical, policy‐focussed perspective. Despite the attention that access to university receives in the press and policy literature, ethical discussion tends to focus on technical matters such as who should pay for university or which schemes of selection are allowable, not the question of who should go to university in the first place. We address the question of university admissions—the question of who should go to university—from an ethical perspective. We find that most discussions draw on a generic conception of what the university is good for that is too thin to provide deliberative guidance and hold that a full account of the ethics of admissions needs to take into account the distinctive good that the university provides—knowledge and understanding. This view, we hold, does not imply that measures should not be taken to widen access to university; however, the basis for such measures should be grounded in that distinctive good in the first instance.     

Examining high-school students’ overconfidence bias in biology exam: a focus on the effects of country and gender

Accurate, rational, and scientific decision making is now considered to be the most important skill in science education. Many studies have found that overconfidence bias is one of the cognitive biases hindering people from achieving such decision making. Gender and country play crucial roles in overconfidence bias. For instance, some particular cultures and genders tend to be more overconfident than others. However, whether or not the two variables interact to influence overconfidence bias also indirectly influences decision making, especially in the context of science education. The purpose of this study is to identify the effects of country and gender on performance, confidence, and overconfidence bias in the samples of Indonesian and Korean high-school students while doing on a biology exam. The twenty-one American Association for the Advancement of Science (AAAS) questions on the topics of genetics and evolution were administered to 297 Indonesian and 235 Korean high-school students, in their first and second years. Every question was featured with a question asking students how confident they are in answering the question correctly. The two-way Analysis of Variances (2-way ANOVA) test was used to answer the research questions. Based on the analyses, we found no significance interactional effects of gender and country in test scores. In contrast, we found a significant interactional effects in both confidence in genetics and evolution. Regarding overconfidence bias, for which that we merged both concepts, we found that country had a higher influence on students’ overconfidence bias than did gender. Additionally, we found the hard-easy effect phenomenon followed overconfidence bias phenomenon. The relationships between country, gender, science education, cognitive bias, and overconfidence bias are discussed. Suggestions for reducing overconfidence bias are also provided.     

The role of student academic beliefs in perceptions of instructor ideological bias

A common perception exists in the media that university instructors are a liberal elite who use their positions to instill a biased perspective in their students. Exploring student attitudes in the United States and Australia, the present study examined how students’ academic beliefs are related to their inferences of instructor ideological bias and the degree to which they identify as reacting to these inferences in the college classroom. Results of regression analysis using previously validated survey instruments revealed that students’ academic entitlement and grade orientation (GO) predicted perceptions of instructor bias. In addition, students’ GO predicted students’ reactions to such inferred bias. Institutional differences, pedagogical implications, and areas for future research are discussed.     

WICS: A Model for College and University Admissions

This article outlines shortcomings of currently used university admissions tests and discusses ways in which they could potentially be improved, summarizing two projects designed to enhance college and university admissions. The projects were inspired by the augmented theory of successful intelligence, according to which successful intelligence involves creative, analytical, practical, and wisdom-based skills. In the Rainbow Project, we found that, through our measures, prediction of 1st-year students’ university grade point average was substantially increased over that provided by SAT and high school grade point average; ethnic-group differences, relative to SAT, were simultaneously reduced. In the Kaleidoscope Project, we found that students admitted for the set of expanded skills performed as well as or better than did other students, without the ethnic-group differences typically obtained in such measures. Enhanced prediction of active-citizenship and leadership activities was also demonstrated through these measures.     

An investigation of the gender differential performance on a high-stakes language proficiency test in Iran

There has been a growing consensus among the educational measurement experts and psychometricians that test taker characteristics may unduly affect the performance on tests. This may lead to construct-irrelevant variance in the scores and thus render the test biased. Hence, it is incumbent on test developers and users alike to provide evidence that their tests are free of such bias. The present study exploited generalizability theory to examine the presence of gender differential performance on a high-stakes language proficiency test, the University of Tehran English Proficiency Test. An analysis of the performance of 2,343 examinees who had taken the test in 2009 indicated that the relative contributions of different facets to score variance were almost uniform across the gender groups. Further, there is no significant interaction between items and persons, indicating that the relative standings of the persons were uniform across all items. The lambda reliability coefficients were also uniformly high. All in all, the study provides evidence that the test is free of gender bias and enjoys a high level of dependability.     

Teacher classroom interactions and behaviours: Indications of bias

In some recent studies, researchers have measured teacher implicit bias and some have shown that teacher bias predicts student academic outcomes. Currently, however, how bias is portrayed to individual students is largely unknown. In this exploratory qualitative study, observations totalling 4 h per participant were undertaken with ten secondary, six intermediate, and four primary teachers in New Zealand. We utilised a thematic analysis approach to identify three themes: “differential nonverbal behaviours”, “differential academic interactions” and “differential reactions under the same conditions” that captured teachers' classroom interactions. Our findings suggested that teachers’ differential behaviours were mediating teacher bias. This was because teacher bias was recorded in every classroom; however, teachers displayed biased behaviours to varying degrees. Bias was displayed through various nonverbal and academic interactions, and some teachers portrayed their bias by treating some students differently when the situation for all students was the same.     

Language Background and Assessment in the Physical Sciences

A project was initiated at the University of Technology, Sydney to look at the role of language and gender in assessment of first‐year university students in Physics. Research on student learning has shown that assessment, to a large extent, determines the curriculum for students and their approaches to learning. Assessment plays a crucial role in students’ interest and motivation in a subject and the quality of their learning outcomes. In this paper, we report on a three‐part study. The first involved an analysis of assessment items. The second involved the use of paired questions to investigate the effect of the format of the question. The third involved investigating the effects of including a range of assessment types. The effects of language background were important and were found to interact with gender. Strategies will be discussed for changing assessments in the physical sciences to make them more equitable.     

A sense of belonging among Australian university students

ABSTRACT

This study was designed to explore what students perceive as essential factors influencing their sense of belonging. Gaining insight into what students perceive as influencing their sense of belonging enables design of interventions to foster that sense of belonging. Two hundred and nine students from a university in Sydney responded to the question: ‘What would help you to feel that you belong at Macquarie University?’ Results showed students most frequently identified people as instrumental in feeling that they belonged at university. Students also reported that places where there is respect, and where others are accepted, contributed to their feelings of belonging. Lastly, opportunities to interact with other students, such as within clubs, societies and at events, were reported by students to contribute to their feelings of belonging. Findings of the current study are discussed in relation to existing literature, to develop approaches that may enhance university students’ sense of belonging.     

Distractor Similarity and Item-Stem Structure: Effects on Item Difficulty

Methods for detecting copying of multiple-choice test responses developed over a 60-year period are reviewed and compared with respect to their effectiveness and the practicality of their application for groups of varying size. Although effective detection methods are routinely used in large admissions and licensing testing programs, they are seldom applied in the areas of standardized testing or classroom testing in schools or colleges. Circumstances associated with this discrepancy are discussed.     

Item Function Characteristics and Dimensionality for Alternative Response Formats in Mathematics

The purpose of this study was to investigate the technical properties of stem-equivalent mathematics items differing only with respect to response format. Using socio- economic factors to define the strata, a proportional stratified random sample of 1,366 Connecticut sixth-grade students were administered one of three forms. Classical item analysis, dimensionality assessment, item response theory goodness-of-fit, and an item bias analysis were conducted. Analysis of variance and confirmatory factor analysis were used to examine the functioning of the items presented in the three different formats. It was found that, after equating forms, the constructed-response formats were somewhat more difficult than the multiple-choice format. However, there was no significant difference across formats with respect to item discrimination. A differential item functioning (DIF) analysis was conducted using both the Mantel-Haenszel procedure and the comparison of the item characteristic curves. The DIF analysis indicated that the presence of bias was not greatly affected by item format; that is, items biased in one format tended to be biased in a similar manner when presented in a different format, and unbiased items tended to remain so regardless of format.     

Ongoing issues in test fairness

In the attempt to identify or prevent unfair tests, both quantitative analyses and logical evaluation are often used. For the most part, fairness evaluation is a pragmatic attempt at determining whether procedural or substantive due process has been accorded to either a group of test takers or an individual. In both the individual and comparative approaches to test fairness, counterfactual reasoning is useful to clarify a potential charge of unfairness: Is it plausible to believe that with an alternative assessment (test or item) or under different test conditions an individual or groups of individuals may have fared better? Beyond comparative questions, fairness can also be framed by moral and ethical choices. A number of ongoing issues are evaluated with respect to these topics including accommodations, differential item functioning (DIF), differential prediction and selection, employment testing, test validation, and classroom assessment.     

Identification and Evaluation of Local Item Dependencies in the Medical College Admissions Test

Measurement specialists routinely assume examinee responses to test items are independent of one another. However, previous research has shown that many contemporary tests contain item dependencies and not accounting for these dependencies leads to misleading estimates of item, test, and ability parameters. The goals of the study were (a) to review methods for detecting local item dependence (LID), (b) to discuss the use of testlets to account for LID in context-dependent item sets, (c) to apply LID detection methods and testlet-based item calibrations to data from a large-scale, high-stakes admissions test, and (d) to evaluate the results with respect to test score reliability and examinee proficiency estimation. Item dependencies were found in the test and these were due to test speededness or context dependence (related to passage structure). Also, the results highlight that steps taken to correct for the presence of LID and obtain less biased reliability estimates may impact on the estimation of examinee proficiency. The practical effects of the presence of LID on passage-based tests are discussed, as are issues regarding how to calibrate context-dependent item sets using item response theory.     

Is meritocracy fair? A qualitative case study of admissions at the University of Oxford

This paper applies an original framework to disentangle the concept of meritocracy, and the relationship between meritocracy and fairness, in elite university admissions. In Britain, elite universities are regularly criticised for being unmeritocratic and by implication unfair, but stakeholders often lack a shared understanding of meritocracy and confound it with outcomes-based measures of fairness. Interview data from admissions tutors at the University of Oxford, viewed through the prism of Joseph Soares’s ideological categories of organic conservatism, democratic elitism and social democratism, indicate that Oxford admissions are highly meritocratic. Thus the question arises: in a deeply unequal society, where academic achievement correlates with family income and social class, is meritocracy a satisfactory admissions framework? The conclusion outlines an alternative path of ‘outcomes-based meritocracy’, as exhibited by the University of California, that makes significant inroads towards integrating social inclusion and academic excellence.     

High school grades,admissions policies,and the gender gap in college enrollment

The female advantage in college enrollment and completion has generated concern among university officials and sparked debate about gender-conscious college admissions. There are a number of explanations for this increasing gender imbalance on college campuses. This paper focuses on the role played by admissions policies that base decisions solely on applicants’ high school grades. Given that females earn higher grades than males, such policies can contribute to growing female shares in admissions. To exemplify this trend, I use publicly-available data from Texas to show that the Texas Top 10% plan, which guarantees public university admission to students who graduate in the top decile of their high school class, led to an increase in the female share of accepted students. The increase was particularly large among black students, where the female share of admitted students was already highest among the major racial/ethnic groups.