Coaching for the SAT: A Summary of the Summaries and an Update

Why is it important to investigate the effectiveness of coaching for a test such as the SAT? How can you evaluate the effectiveness of coaching? What are some common misconceptions? What do we know and what is still unclear about the effects of coaching? What can we tell students and their families about the results of coaching studies ?     

Selectivity bias and the determinants of SAT scores

The determinants of state-wide average SAT scores are estimated for 1982 in a regression analysis which corrects for the proportion of students taking the test. The selectivity correction has a large impact on the estimates of the effects of other variables. Little effect of schooling variables (teachers' salaries, teachers per pupil, other expenditures) is found in the selectivity-adjusted estimates, except that large schools seem to depress SAT scores and private schools enhance scores. State-wide high school graduation standards also do not explain SAT score variation. In contrast, several demographic variables are quite important, including family size, the college-educated fraction of the population, and female-headed households. We can explain some of the SAT score decline in the 1970s with these cross-section estimates, suggesting that the decline over time is not due to changing resources for schooling but, in part, to changing demographics. In particular, a large part of the recent SAT score decline was caused by the large families of the post-war baby boom.     

The Relationship between Examinee Anxiety and preference for Self-Adapted Testing

In this study, we investigated the hypothesis that the previously found positive effects of self-adapted testing are attributable to examinees having an increased perception of control over a stressful testing situation. Examinees were randomly assigned to either (a) take a computerized-adaptive test (CAT), (b) take a self-adapted test (SAT), or (c) choose between taking a CAT or SAT. Results showed that the strongest preference for SAT was shown by examinees reporting high levels of math anxiety. Moreover, highly mathanxious examinees who were allowed to choose between the test types exhibited higher mean proficiency estimates than examinees who were assigned to test type.     

Computerized-Adaptive and Self-Adapted Music Listening Tests: Psychometric Features and Motivational Benefits

In this study, we compared the efficiency, reliability, validity, and motivational benefits of computerized-adaptive and self-adapted music-listening tests (referred to hereafter as CAT and SAT, respectively). Junior high school general music students completed a tonal memory CAT, a tonal memory SAT, standardized music aptitude and achievement tests; and questionnaires assessing test anxiety, demographics, and attitudes about the CAT and SAT. Standardized music test scores and music course grades served as criterion measures in the concurrent validity analysis. Results showed that the SAT elicited more favorable attitudes from examinees and yielded ability estimates that were higher and less correlated with test anxiety than did the CAT. The CAT, however, required fewer items and less administration time to match the reliability and concurrent validity of the SAT and yielded higher levels of reliability and concurrent validity than the SAT when test length was held constant. These results reaffirm important tradeoffs between the two administration procedures observed in prior studies of vocabulary and algebra skills, with the SAT providing greater potential motivational benefits and the CAT providing greater efficiency. Implications and questions for future research are discussed.     

Preparing students for college admissions tests

Attaining successful outcomes on the SAT can have profound educational and financial consequences for college-bound students. Using archival data from a private tutoring centre, we investigated variables we hypothesised to contribute to SAT score increases. Our analyses revealed significant effects of time on task and rate of SAT homework completion. Starting preparation earlier in the junior year and distributing tutoring sessions over larger intervals contributed to higher SAT score improvements. Testing effects were significant: more frequent practice tests significantly increased SAT scores. Official SAT administrations contributed more to score increases than did unofficial practice tests. Individual tutoring hours yielded greater increases than did group tutoring hours. Student characteristics such as gender, SES, or private or public school attendance were not significant. Implications for improving test preparation for students as well as directions for future research are discussed.     

企业教练技术对管理职能的影响初探

将企业教练技术应用于管理领域是当前的流行趋势,教练式管理模式以其独特的优势受到越来越多的大中型企业的青睐。企业教练技术给计划、组织、领导、控制和创新等管理职能带来了一定的影响。企业教练技术的引入和实施增强了管理实践的效率和效果。     

Training Test-Wiseness and Flawed Item Types

This study investigated several current coaching practices used in training test-wiseness for analogy items in standardized test batteries. A three-group design was used which included a general test-taking, "encouragement" condition in addition to a no-training control group condition. The specific techniques used in training are described. Scholastic Aptitude Test (SAT) scores were obtained from university admission files to verify that no overall aptitude differences existed in the three conditions. Differences were observed for the coached group relative to the two control groups in terms of overall number of correct responses for the coached item types (analogies). No differences were found for the non-coached item types. Item difficulties for the three groups are also reported which show that several items were indeed made easier for individuals in the coached group. A qualitative analysis of the items made easier by coaching in terms of the training techniques used is given along with an analysis of the items that did not respond to coaching. Finally, a discussion of potentially flawed item types and item characteristics and suggestions for dealing with such flaws are given.     

The Coaching of Teachers: Results of Five Training Studies

In this study, the results of five training studies evaluating the effects of a coaching program for use in Dutch primary and secondary schools are described. The coaches involved in these studies were: (1) school counsellors, (2) school principals, (3) mentors of beginning teachers, (4) mentors of teachers-in-training and (5) secondary school teachers. Coaching is a form of in-class support to provide (novice) teachers with feedback on their functioning and thereby stimulate the self-reflection and self-analysis needed to improve instructional effectiveness. Based on the pre- and post-training ratings of coaching conferences, a significant treatment effect was found for the coaching skills concerned with the development of autonomy (empowerment), feedback and business-like attitude. The pre- and post-training ratings from the coached teachers themselves showed a significant treatment effect for the coaching skills concerned with the planning of observational data collection. In addition, the teachers coached by trained coaches valued the coaching skills demonstrated significantly more than those teachers coached by untrained coaches. Inspection of the interactions for the adjusted post-test scores revealed different training effects for the five studies.     

All Validities are Not Created Equal: Determinants of Variation in SAT Validity Across Schools

Previous research has demonstrated that cognitive test validities are generalizable and predictive of academic performance across situations. However, even after accounting for statistical artifacts (e.g., sampling error, range restriction, criterion reliability), substantial variability often remains around estimates of cognitive test–performance relationships suggesting the presence of additional moderators. In the present study, we examine the sources of institutional variation in Scholastic Aptitude Test (SAT) validity across a sample of 110 institutions. Institutional characteristics moderated the size of SAT validities, such that more selective schools and schools that emphasize traditional assessment techniques (i.e., school records, standardized tests) showed higher SAT validities while schools that were larger and where students demonstrated more financial need, schools that emphasized the usage of alternative assessment techniques (i.e., essays, letters of recommendations, extracurricular activities), and schools that enrolled higher percentages of historically disadvantaged minority students generally exhibited lower SAT validities. Future directions in the understanding of situational influences on SAT–grade point average validities are discussed.     

Developing medical students as teachers: An anatomy‐based student‐as‐teacher program with emphasis on core teaching competencies

Teaching is an increasingly recognized responsibility of the resident physician. Residents, however, often assume teaching responsibilities without adequate preparation. Consequently, many medical schools have implemented student‐as‐teacher (SAT) programs that provide near‐peer teaching opportunities to senior medical students. Near‐peer teaching is widely regarded as an effective teaching modality; however, whether near‐peer teaching experiences in medical school prepare students for the teaching demands of residency is less understood. We explored whether the anatomy‐based SAT program through the Human Structure didactic block at Mayo Medical School addressed the core teaching competencies of a medical educator and prepared its participants for further teaching roles in their medical careers. A web‐based survey was sent to all teaching assistants in the anatomy‐based SAT program over the past five years (2007–2011). Survey questions were constructed based on previously published competencies in seven teaching domains – course development, course organization, teaching execution, student coaching, student assessment, teacher evaluation, and scholarship. Results of the survey indicate that participants in the anatomy‐based SAT program achieved core competencies of a medical educator and felt prepared for the teaching demands of residency. Anat Sci Educ 6: 385–392. © 2013 American Association of Anatomists.     

EFFECTS OF COACHING ON GRE APTITUDE TEST SCORES

Test preparation activities were determined for a large representative sample of Graduate Record Examination (GRE) Aptitude Test takers. About 3% of these examinees had attended formal coaching programs for one or more sections of the test.
After adjusting for differences in the background characteristics of coached and uncoached students, effects on test scores were related to the length and the type of programs offered. The effects on GRE verbal ability scores were not significantly related to the amount of coaching examinees received, and quantitative coaching effects increased slightly but not significantly with additional coaching. Effects on analytical ability scores, on the other hand, were related significantly to the length of coaching programs, through improved performance on two analytical item types, which have since been deleted from the test.
Overall, the data suggest that, when compared with the two highly susceptible item types that have been removed from the GRE Aptitude Test, the test item types in the current version of the test (now called the GRE General Test) appear to show relatively little susceptibility to formal coaching experiences of the kinds considered here.     

Contextual Factors Associated With the Validity of SAT Scores and High School GPA for Predicting First-Year College Grades

Prior research has shown that there is substantial variability in the degree to which the SAT and high school grade point average (HSGPA) predict 1st-year college performance at different institutions. This article demonstrates the usefulness of multilevel modeling as a tool to uncover institutional characteristics that are associated with this variability. The results revealed that the predictive validity of HSGPA decreased as mean total SAT (i.e., sum of the three SAT sections) score at an institution increased and as the proportion of White freshmen increased. The predictive validity of the three SAT sections (critical reading, mathematics, and writing) varied differently as a function of different institution-level variables. These results suggest that the estimates of validity obtained and aggregated from multiple institutions may not accurately reflect the unique contextual factors that influence the predictive validity of HSGPA and SAT scores at a particular institution.     

Substituting SAT II: Subject Tests for SAT I: Reasoning Tests: Impact on Admitted Class Composition and Quality

Using data from a sample of 10 colleges at which most students had taken both SAT I: Reasoning tests and SAT II: Subject tests, we simulated the effects of making selection decisions using SAT II scores in place of SAT I scores. Specifically, we treated the students in each college as forming the applicant pool for a more select college, and then selected the top two thirds (and top one third) of the students using high school grade point average combined with either SAT I scores or the average of SAT II scores. Success rates, in terms of first-year grade point averages, were virtually identical for students selected by the different models. The percentage of African American, Asian American, and White students selected varied only slightly across models. Appreciably more Mexican American and Other Latino students were selected with the model that used SAT II scores in place of SAT I scores because these students submitted subject test scores for the Spanish test on which they had high scores.     

Mothers' Social Coaching, Mother-Child Relationship Style, and Children's Peer Competence: Is the Medium the Message?

Contributions of mothers' social coaching and responsive style to preschoolers' peer competence were evaluated in 2 studies. In Study 1, 43 mother-child dyads participated in 3 laboratory tasks; videotapes were coded for responsive interaction style in play, advice regarding videotaped peer dilemmas (coaching), and nonsocial teaching in a puzzle task. Coaching and style were largely independent and were correlated with measures of social competence. In Study 2 ( n = 62), coaching and style uniquely predicted teacher ratings, but only style predicted peer acceptance. To investigate whether coaching mediated the effects of style and/or whether style moderated the effects of coaching, the samples were combined. No evidence was found for mediation, but coaching was a more powerful predictor of lower levels of boys' aggression when the mother-child relationship was less responsive. Discussion focuses on models of socialization that stress the interplay of general style and specific socialization practices in promoting social competence.     

Does intensive coaching reduce school dropout? Evidence from a randomized experiment

School dropout is an important social and economic problem. This paper investigates the effect of an intensive coaching program aimed at reducing school dropout rates among students aged 16–20. Students received support and guidance with their study activities, personal problems and internships in firms. The coaching program lasted one or two years. Students were randomly assigned to the coaching program. We find that one year of coaching reduced school dropout rates by more than 40% from 17 to 10 percentage points. The second year of coaching further reduced school dropout by 1 percentage point. The program is most effective for students with a high ex-ante probability of dropping out, such as students no longer obliged to be in formal education, male students, and students not living with both parents. Cost-benefit analysis suggests that one year of coaching is likely to yield a net social gain.     

Improving Instruction and Classroom Management Behaviour in Mixed‐age Classrooms: results of two improvement studies

This article describes the results of a second school improvement study directed at schools with mixed‐age classes, and compares the results of this second study with the first study. The second improvement study examines the effects of a staff development programme which introduced selected findings from teaching effectiveness research into ongoing school settings with mixed‐age classes. In addition to the staff development programme the effects of coaching are also evaluated. In contrast to the first improvement study the staff development programme was implemented by teacher educators and school counsellors in the regular educational support context (in‐service teacher education). A quasi‐experimental, treatment‐control group investigation was designed to test the effects of the staff development programme ‘Dealing with mixed‐age classes’, and the effects of coaching. Based on pre‐ and post‐training classroom observations, the second improvement study‐‐like the first one‐‐revealed a significant treatment effect for pupils’ time‐on‐task levels in mixed‐age classrooms, and for teachers’ instructional and classroom management skills. Only two coaching effects were found: for organising effective instruction and for dealing with disturbances. Time‐on‐task levels improved more strongly in classes of coached teachers. In the second improvement study the effects on teacher instructional and classroom management skills, and pupil behaviour are smaller than in the first study. The staff development programme as conducted in regular in‐service settings by teacher educators and school counsellors was less effective than the staff development programme as conducted in an evaluation setting by the designers of the programme.     

What Happens During Language and Literacy Coaching? Coaches’ Reports of Their Interactions With Educators

Research Findings: This study investigated coaches’ interactions with educators in the context of a large-scale, state-implemented literacy professional development (PD). We examined log data and open-comment reports to understand what coaches found salient about their interactions with educators as well as how those reports aligned with the initial design of the PD. Coaches reported spending a large proportion of their interactions with educators completing administrative tasks. Our findings also indicate that coaches disproportionally targeted instructional content from the PD while also adding unrelated instructional content to their coaching. Although coaches reported focusing on relationship building, they reported using less efficacious coaching strategies (e.g., observation and discussion) more frequently than coaching strategies demonstrated to be more efficacious (e.g., modeling and coteaching). Practice or Policy: Our findings suggest an explanation for the mixed evidence around coaching, as coaches in the study seemed to move beyond the specifications of the PD in their coaching interactions. This work has implications for the design of PD for both improving coach training and allowing some flexibility to meet educators’ learning needs that may be secondary to the content of the PD. Findings also support the need for more nuanced mechanisms for investing in coaching and coaching outcomes.     

The effects of college grade adjustments on the predictive validity and utility of SAT scores

Regressing adjusted grade-point averages on freshman SAT scores and high school grade-point averages results in large increases in the incremental predictive validity of the SAT. Even so, the SAT still changes no more than a small proportion of admissions decisions and does not result in substantively important increases in freshman grades. The test does, however, change the composition of the freshman class by altering acceptances to some major areas of study and by limiting the access of women and blacks.     

New Perspectives on the Correlation of SAT Scores, High School Grades, and Socioeconomic Factors

In studies of the SAT, correlations of SAT scores, high school grades, and socioeconomic factors (SES) are usually obtained using a university as the unit of analysis. This approach obscures an important structural aspect of the data: The high school grades received by a given institution come from a large number of high schools, all of which have potentially different grading standards. SAT scores, on the other hand, can be assumed to have the same meaning across high schools. Our analyses of a large national sample show that, when pooled within-high-school analyses are applied, high school grades and class rank have larger correlations with family income and education than is evident in the results of typical analyses, and SAT scores have smaller associations with socioeconomic factors. SAT scores and high school grades, therefore, have more similar associations with SES than they do when only the usual across-high-school correlations are considered .