Selection bias in college admissions test scores

Data from college admissions tests can provide a valuable measure of student achievement, but the non-representativeness of test-takers is an important concern. We examine selectivity bias in both state-level and school-level SAT and ACT averages. The degree of selectivity may differ importantly across and within schools, and across and within states. To identify within-state selectivity, we use a control function approach that conditions on scores from a representative test. Estimates indicate strong selectivity of test-takers in “ACT states,” where most college-bound students take the ACT, and much less selectivity in SAT states. To identify within- and between-school selectivity, we take advantage of a policy reform in Illinois that made taking the ACT a graduation requirement. Estimates based on this policy change indicate substantial positive selection into test participation both across and within schools. Despite this, school-level averages of observed scores are extremely highly correlated with average latent scores, as across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between-school variation in achievement but is otherwise unlikely to lead to misleading conclusions.     

Traversing the Challenges of Conducting Research with Young, Immigrant Children: The Case of Korean Children

Based on a qualitative study on young Korean immigrant children’s interpretation of American popular culture, this paper reflects on the researchers’ role by examining the research challenges faced. Prior to discussing the research obstacles in conducting the study, we begin with briefly describing its methodology and design. Next, we discuss the various obstacles we encountered when conducting our study and howwe attempted to overcome these dilemmas by discussing two major struggles during the research process: a) the relationships between the researchers and our young participants, and b) the cultural sensitivity that was needed in initially gaining permission from and working with the parents of our informants. It then illuminates how the researchers attempted to cope with such difficulties by rethinking a researchers’s role emphasizing the importance of cultural context in research. Finally, the paper provides some discussions and implications based on our research experiences.     

Pacem Portantes Advenerint: Ambivalent Images of Muslims in the Chronicles of Norman Italy

The goal of this essay is to call attention to some of the more positive and ambivalent depictions of Muslims in a set of historical texts associated with the Norman takeover of Sicily in the fifth/eleventh and sixth/twelfth centuries. To achieve that aim, it considers social vocabulary applied to Muslims in five sources written by Amatus of Montecassino, Geoffrey Malaterra, William of Apulia, Alexander of Telese, and Hugo Falcandus. Although recent scholarship has posited that medieval identity was often felt through a “self versus other” or “Christian versus non-Christian” dichotomy, this essay questions the notion that the actual language contained in these sources ever devolved into such simplistic, binary terms. On the contrary, though the perceptions and definitions applied to this group of people were, admittedly, sometimes based on uninformed stereotypes, they were more often deliberately constructed images that were highly dependent on the cultural milieu in which they were created.     

An Assessment of the Quantitative Literacy of Undergraduate Students

Quantitative literacy (QLT) represents an underlying higher-order construct that accounts for a person's willingness to engage in quantitative situations in everyday life. The purpose of this study is to retest the construct validity of a model of quantitative literacy (Wilkins, 2010). In this model, QLT represents a second-order factor that accounts for the interrelationship among three first-order factors: mathematical beliefs, mathematical disposition, and mathematical cognition. Using data from two samples of undergraduate students (n = 186 and n = 184), a series of confirmatory factor analyses were conducted. Results supported the hierarchical three-factor structure and confirmed its factorial invariance across multiple groups of students. Latent QLT scores from the model were used to compare and rank student QLT by gender and class standing.     

Team-based Introductory Research Experiences in Mathematics

Abstract

As part of Middle Tennessee State University’s (MTSU’s) initiative to improve retention of at-risk STEM majors, they recruit first-time, full-time freshman STEM majors with mathematics ACT scores of 19 to 23 to participate in MTSU’s “Mathematics as a FirstSTEP to Success in STEM” project (FirstSTEP). This article overviews MTSU’s FirstSTEP research approach and demonstrates that introductory, early research experiences can be successful for mathematics projects. At the end of their freshman year, mathematics students can develop a foundation to conduct mathematical research, increase depth of content knowledge, and build confidence in their technical abilities.     

Predicting Kindergarteners' Achievement and Motivation From Observational Measures of Teaching Effectiveness

We investigated the premise that observation measures of instruction are indicators of effective teaching, using the definition of effectiveness articulated by departments of education: teaching that boosts student achievement. We argued that student motivation is equally as important as achievement in the evaluation of teaching effectiveness (TE); therefore, we examined students' (N = 145) achievement and motivation outcomes. We scored 40 lessons (from 10 kindergarten teachers) with two TE observation measures: the content-independent Classroom Assessment Scoring System (CLASS) and the content-specific Reformed Teaching Observation Protocol (RTOP). We found that the two measures' scores were related differently to student outcomes. Instructionally supportive practices (CLASS and RTOP total) predicted achievement and motivation. Emotional support (CLASS) was positively related to motivation but not to achievement. Classroom organization (CLASS) was negatively related to both motivation and achievement. The CLASS total score did not predict student outcomes; its use masked differences across domains of teaching practices.     

Using first-order cognitive analysis to understand problem solving behavior: An example from accounting

We report a study of human problem solving in the accounting domain. This study has three characteristics. First, it is explicitly intended to aid in the development of instructional materials. Second, it uses a powerful task analyzing scheme as the basis for its conclusions. Third, we use a technique called First-Order Cognitive Analysis that combines the advantages of task analysis to overcome some of the traditional methodological problems posed to protocol analysis. Our representational scheme, KO, seems to be largely valid as a basis for reasoning about cognition, although only three subjects were studied.