Gender Differences in College Preparedness: A Statewide Study

In this study, the authors investigated the extent to which boys and girls differed in their college-ready performance in reading, in math, and in both subjects for the 2006–2007 school year for the state of Texas (n = 1,099 high schools). Also examined were the extent to which boys and girls differed in their performance on the Scholastic Assessment Test (SAT) and the American College Test (ACT) for the 2005–2006 and 2006–2007 school years. Differences in criterion scores, as well as how many boys and girls actually took these exams, were examined in addition to their average scores. Statistically significant differences were present between the college-ready scores of boys and of girls in reading, math, and in both subject areas. Of special note is that less than one-third of all students were deemed to be college-ready in both subject areas. Statistically significant differences also were revealed between boys and girls in the percentage taking the SAT or ACT, in the percentage meeting the SAT or ACT criterion, and in their SAT average scores, but not in their ACT average scores. Implications of these findings are discussed.     

Selection bias in college admissions test scores

Data from college admissions tests can provide a valuable measure of student achievement, but the non-representativeness of test-takers is an important concern. We examine selectivity bias in both state-level and school-level SAT and ACT averages. The degree of selectivity may differ importantly across and within schools, and across and within states. To identify within-state selectivity, we use a control function approach that conditions on scores from a representative test. Estimates indicate strong selectivity of test-takers in “ACT states,” where most college-bound students take the ACT, and much less selectivity in SAT states. To identify within- and between-school selectivity, we take advantage of a policy reform in Illinois that made taking the ACT a graduation requirement. Estimates based on this policy change indicate substantial positive selection into test participation both across and within schools. Despite this, school-level averages of observed scores are extremely highly correlated with average latent scores, as across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between-school variation in achievement but is otherwise unlikely to lead to misleading conclusions.     

The Primary Classification Test

ABSTRACT

The authors sought to better understand the relationship between students participating in the Advanced Placement (AP) program and subsequent performance on the Scholastic Aptitude Test (SAT). Focusing on students graduating from U.S. public high schools in 2010, the authors used propensity scores to match junior year AP examinees in 3 subjects to similar students who did not take any AP exams in high school. Multilevel regression models with these matched samples demonstrate a mostly positive relationship between AP exam participation and senior year SAT performance, particularly for students who score a 3 or higher. Students who enter into the AP year with relatively lower initial achievement are predicted to perform slightly better on later SAT tests than students with similar initial achievement who do not participate in AP.     

Contextual Factors Associated With the Validity of SAT Scores and High School GPA for Predicting First-Year College Grades

Prior research has shown that there is substantial variability in the degree to which the SAT and high school grade point average (HSGPA) predict 1st-year college performance at different institutions. This article demonstrates the usefulness of multilevel modeling as a tool to uncover institutional characteristics that are associated with this variability. The results revealed that the predictive validity of HSGPA decreased as mean total SAT (i.e., sum of the three SAT sections) score at an institution increased and as the proportion of White freshmen increased. The predictive validity of the three SAT sections (critical reading, mathematics, and writing) varied differently as a function of different institution-level variables. These results suggest that the estimates of validity obtained and aggregated from multiple institutions may not accurately reflect the unique contextual factors that influence the predictive validity of HSGPA and SAT scores at a particular institution.     

美国大学入学考试SAT与ACT对我国高考的启示——对我国高考的近期与远期改革建议

美国大学入学考试主要有两种:SAT与ACT。美国大学在选拔新生时强调素质的全面考查,但SAT、ACT等大学入学考试的成绩也至关重要。美国大学入学考试SAT或ACT每年均开展多次,大学入学考试与高校录取工作相对独立地开展,实行"招考分离"。学生亦可报名参加多次SAT或ACT考试,高校以最高成绩为选拔依据。SAT、ACT考试编制的科学化程度较高,试题难度稳定,得到了多数大学的信任。变"一次高考机会"为"多次高考机会",实行"招考分离",是我国高考改革的大势所趋。但具体改革的操作细节应深入研究,谨慎实施,以免发生执行上的偏差。     

Selectivity bias and the determinants of SAT scores

The determinants of state-wide average SAT scores are estimated for 1982 in a regression analysis which corrects for the proportion of students taking the test. The selectivity correction has a large impact on the estimates of the effects of other variables. Little effect of schooling variables (teachers' salaries, teachers per pupil, other expenditures) is found in the selectivity-adjusted estimates, except that large schools seem to depress SAT scores and private schools enhance scores. State-wide high school graduation standards also do not explain SAT score variation. In contrast, several demographic variables are quite important, including family size, the college-educated fraction of the population, and female-headed households. We can explain some of the SAT score decline in the 1970s with these cross-section estimates, suggesting that the decline over time is not due to changing resources for schooling but, in part, to changing demographics. In particular, a large part of the recent SAT score decline was caused by the large families of the post-war baby boom.     

One-year experimental study of the ACT objective test with non-traditional students

The Curriculum Committee of the General College (GC) at the University of Minnesota selected the ACT Objective Test to assess general educational outcomes, educational gains, and to provide an individual assessment of academic achievement for graduating Associate of Arts degree candidates.Data analyses to assess general educational outcomes revealed that GC Associate of Arts degree candidates scored significantly lower than an ACT national sample on total score and four out of six subscores.Data analyses to assess educational gains revealed no significant differences between three comparison groups of GC students: Associate of Arts candidates, GC sophomores, and GC freshman; although there was a significant difference on grade-point average (GPA).Finally, data analyses to assess ethnic group performance revealed significant differences between four ethnic groups, although there were no significant differences on GPA. In addition, it was discovered that the relationship between total score and GPA was strongly positive for the Black and Caucasian groups but was negative for the Asian and Other groups.Thus, since the ACT Objective Test did not provide an adequate measure of educational gains, and it appears that the ACT Objective Test might possibly be a culturally biased instrument, it was decided to discontinue use of the ACT Objective Test in the General College after the one-year experimental period.Gloria B. Wood received her Ph.D. in Counseling Psychology from the University of Minnesota. She teaches psychology and career planning to a predominantly nontraditional and under-prepared college population. Her research interests include: cognitive development and learning styles; the psychological and motivational variables related to academic performance; and variables related to career assessment and development.     

All Validities are Not Created Equal: Determinants of Variation in SAT Validity Across Schools

Previous research has demonstrated that cognitive test validities are generalizable and predictive of academic performance across situations. However, even after accounting for statistical artifacts (e.g., sampling error, range restriction, criterion reliability), substantial variability often remains around estimates of cognitive test–performance relationships suggesting the presence of additional moderators. In the present study, we examine the sources of institutional variation in Scholastic Aptitude Test (SAT) validity across a sample of 110 institutions. Institutional characteristics moderated the size of SAT validities, such that more selective schools and schools that emphasize traditional assessment techniques (i.e., school records, standardized tests) showed higher SAT validities while schools that were larger and where students demonstrated more financial need, schools that emphasized the usage of alternative assessment techniques (i.e., essays, letters of recommendations, extracurricular activities), and schools that enrolled higher percentages of historically disadvantaged minority students generally exhibited lower SAT validities. Future directions in the understanding of situational influences on SAT–grade point average validities are discussed.     

Examination of Indices of High School Performance Based on the Graded Response Model

We examined summary indices of high school performance (coursework, grades, and test scores) based on the graded response model (GRM). The indices varied by inclusion of ACT test scores and whether high school courses were constrained to have the same difficulty and discrimination across groups of schools. The indices were examined with respect to skewness, incremental prediction of college degree attainment, and differences across racial/ethnic and socioeconomic subgroups. The most difficult high school courses to earn an “A” grade included calculus, chemistry, trigonometry, other advanced math, physics, algebra 2, and geometry. The GRM‐based indices were less skewed than simple high school grade point average (HSGPA) and had higher correlations with ACT Composite score. The index that included ACT test scores and allowed item parameters to vary by school group was most predictive of college degree attainment, but had larger subgroup differences. Implications for implementing multiple measure models for college readiness are discussed.     

A Study of the Effectiveness of Certain Techniques of Reading Instruction in Developing the Ability of Junior High-School Students to Conceptualize Social Studies Content

Abstract

Declines in college entrance examination test scores have been documented both on the Scholastic Aptitude Test (SAT) and the American College Testing Program (ACT). Sources of such declines seem to be functions of three main contexts: (a) school-related factors, (b) student-related factors, and (c) family structure-related factors. It was found that academic course taking had the greatest partial regression coefficient. Next in predictive power was the level of educational aspiration of the student. Size of school was negatively associated with composite ACT scores. Student-related factors comprised the most significant context of association. More modest support was demonstrated for school-related factors. Family structure factors did not appear significantly related to composite ACT scores.     

A multi-level modeling approach to investigating students’ critical thinking at higher education institutions

Twenty-first century and higher-learning skills such as critical thinking are highly desired for new college graduates entering the workplace, especially with rapid changing technology and a more globalized economy. As a result, the importance of critical thinking development at higher education institutions has increased. The purpose of this study was to examine the effect of student and institutional variables on the difference in critical thinking scores between first-year and third/fourth-year students at higher education institutions. We examined the extent to which institutional variables moderate the relationship between student variables and the critical thinking score difference. We used a multi-level modeling approach to account for the clustering of students in institutions. Results of this study showed that: (a) institutional variability accounted for 15% of the variance in estimated critical thinking scores, (b) third/fourth-year students had higher critical thinking scores as compared to their first-year peers even when controlling for college admissions score, and (c) in the final model, the main effects of college admission scores, student-faculty ratio, and percent Black/African American were statistically significant; however, retention was the only significant moderator of the relationship between critical thinking scores and class level. Implications and future research are discussed.     

Answering reading Comprehension Items Without Passages on the SAT-I,the ACT,and the GRE

Performance on the reading comprehension (RC) tasks of the Scholastic Assessment Test-I (SAT-I or the "new" SAT), the Enhanced American College Testing Assessment (ACT), and the Graduate Record Examination (GRE) when passages were missing was examined. For the SAT-I and ACT, scores were well above chance and correlated substantially with verbal score on the earlier version of the SAT, indicating that examinees perform similarly with or without passages. Comparable but weaker results were found for the GRE. The findings raise doubts about the construct validity of the RC task. We argue that performance is influenced by the plausibility of item choices with or without the passages and that this, in turn, is the result of the construction of test items with little knowledge of the underlying reading process.     

Identifying the Best Buys in U.S. Higher Education

Which U.S. institutions of higher education offer the best value to consumers? To answer this question, we evaluate U.S. institutions relative to a data envelopment analysis (DEA) multi-factor frontier based on 2000?C2001 data for 1,179 4-year institutions. The resulting DEA ??best buy?? scores allow the ranking of institutions by a weighted sum of institutional characteristics per dollar of average net price. The net price is calculated as tuition, fees, room, and board less per student financial aid. Institutional characteristics include SAT score, athletic expenditures, instructional expenditures, value of buildings, dorm capacity, and student body characteristics. The DEA scores indicate the distance of each institution from the ??best buy?? frontier for the chosen characteristics, providing an objective means of ranking institutions as the best values in higher education.     

The Second Mathematics Standard Assessment Tasks at Key Stage 1: issues of reliability raised by a five school study

This paper assesses the value that can be put on the mathematical Standard Assessment Task (SAT) score as an indicator of how children are doing compared to others. The results of a study of a sample of 171 Year 2 children from five randomly selected primary schools within one Local Education Authority (LEA) are presented. Pupils’ scores on Mathematics 7, a standardised mathematics test for 7 year‐olds are compared to the mathematics SAT score elicited by them in the previous half‐term. Results show that children with the same Mathematics 7 score may be designated Level 1, 2 or 3 on the mathematics SAT. Conclusions, based on such a small study, are tentative. There is a need to be aware that finding out how a child is doing may be more complicated than simply looking at the SAT results. It may be hypothesised that the 10 point assessment scale is too crude a summative device to measure children's mathematical standards accurately.     

Can High School Achievement Tests Serve to Select College Students?

Postsecondary schools have traditionally relied on admissions tests such as the SAT and ACT to select students. With high school achievement assessments in place in many states, it is important to ascertain whether scores from those exams can either supplement or supplant conventional admissions tests. In this study we examined whether the Arizona Instrument to Measure Standards (AIMS) high school tests could serve as a useful predictor of college performance. Stepwise regression analyses with a predetermined order of variable entry revealed that AIMS generally did not account for additional performance variation when added to high school grade-point average (HSGPA) and SAT. However, in a cohort of students that took the test for graduation purposes, AIMS did account for about the same proportion of variance as SAT when added to a model that included HSGPA. The predictive value of both SAT and AIMS was generally the same for Caucasian, Hispanic, and Asian American students. The ramifications of universities using high school achievement exams as predictors of college success, in addition to or in lieu of traditional measures, are discussed.     

Expenditures and postsecondary graduation: An investigation using individual-level data from the state of Ohio

Using detailed individual-level data from public universities in the state of Ohio, I estimate the effect of various institutional expenditures on the probability of graduating from college. Using a competing risks regression framework, I find differential impacts of expenditure categories across student characteristics. I estimate that student service expenditures have a larger impact on students with low SAT/ACT scores, while instructional expenditures are more important for high test score students and those majoring in scientific/quantitative fields. The individual-level nature of these data allows me to address measurement error and endogeneity concerns the previous literature has been unable to deal with.     

Measuring value‐added in higher education: conditions and caveats – results from using the Measure of Academic Proficiency and Progress (MAPP™)

The Commission on the Future of Higher Education in the USA emphasises accountability in higher education as one of its key areas of interest. A programme, called the Voluntary System of Accountability, was developed to evaluate the effectiveness of general public college education. This study examines how students progress in college, indicated by the performance difference between freshmen and seniors after controlling for admission scores, can be measured using the Measure of Academic Proficiency and Progress? (MAPP?) test. A total of 6196 students from 23 institutions was included in this study. Results indicated that MAPP was able to differentiate between the performance of freshmen and seniors after controlling for SAT^®/ACT scores. The institutions were classified into 10 groups on the basis of the difference in the actual versus expected MAPP performance. The assumptions and implications of linking student learning growth to institutional effectiveness are discussed. Methodological issues on value‐added calculation and student motivation in taking standardised tests are also noted.     

Assessing Differential Prediction of College Grades by Race/Ethnicity with a Multilevel Model

Previous research notes the importance of understanding racial/ethnic differential prediction of college grades across multiple institutions. Institutional variation in selection indices is especially important given some states' laws governing public institutions' admissions decisions. This paper employed multilevel moderated multiple regression to study the variation of selection indices across 30 institutions and the accuracy of selection indices in predicting college grades for students of different racial/ethnic backgrounds. Several benefits of multilevel models for cross-institutional differential prediction studies were described and include: controlling for institutional differences in range restriction, providing reliability estimates of least squares estimates, and adjusting criterion scores for differences in coursework difficulty. The findings from this study provide evidence of institutional variation in selection indices, which challenges current laws aimed at standardizing them. Specifically, there was evidence that the predictor slope coefficients varied across institutions, in addition to the estimates that measured intercept differences for African and Asian American students. Across universities, the results mirrored previous findings: high school grade point average (GPA) differentially predicted grades for African Americans, SAT verbal scores differentially predict grades for Asian Americans, and SAT math scores were better predictors of Asian Americans' grades.     

MEASUREMENT OF CHANGE IN THE MEANING OF WRITING IN FRESHMAN ENGLISH

This study had a two‐fold purpose: first, to determine whether a measurable change in the meaning of the word writing could occur during a writing course; and second, to measure meaning rather than merely student attitudes toward writing. To satisfy these purposes, a semantic differential was administered to students in both regular and developmental classes in a two‐year campus at the beginning and the end of the 10‐week term. The scale measured the meaning of the term writing according to three dimensions: evaluation, activity, and potency. Scores were compared for each student and for each class of students. A significant increase occurred in the activity dimension for students in the regular classes by the end of the term; however, their evaluative rating decreased. Developmental students showed no change. Correlations with other measures of student performance were performed: final grades in the writing course, high school G.P.A., and verbal SAT scores. Regular students’ final grades correlated positively with the evaluation dimension of writing, and their verbal SAT scores correlated positively with the activity and potency dimensions. An inverse correlation was obtained between the developmental students’ verbal SAT scores and their evaluative rating. All correlations were stronger by the end of the term. Implications for writing teachers are discussed. Use of this instrument for measuring change in students’ perceptions of writing is recommended.     

Constructing a Universal Scale of High School Course Difficulty

This study examined the usefulness of applying the Rasch rating scale model (Andrich, 1978) to high school grade data. ACT Assessment test scores (English, Mathematics, Reading, and Science Reasoning) were used as "common items" to adjust for different grading standards in individual high school courses both within and across schools. This scaling approach yielded an ACT Assessment-adjusted high school grade point average (AA-HSGPA) on a common scale across high schools and cohorts within a large public university. AA-HSGPA was a better predictor of first-year college grade point average (CGPA) than the regular high school grade point average. The best model for predicting CGPA included both the ACT composite score and AA-HSGPA.