A Bayesian Hierarchical Selection Model for Academic Growth With Missing Data

Using a sample of schools testing annually in grades 9–11 with a vertically linked series of assessments, a latent growth curve model is used to model test scores with student intercepts and slopes nested within school. Missed assessments can occur because of student mobility, student dropout, absenteeism, and other reasons. Missing data indicators are modeled using logistic regression, with grade 9 and potentially unobserved growth scores used as covariates. Under a hierarchical selection model, estimates of school effects on academic growth and missingness are obtained. The results from the selection model are compared to a model that ignores the missing data process.     

Decision Theory Models for Validating Course Placement Tests

Most American postsecondary institutions have course placement systems for their first-year students. Placement systems typically consist of an assessment component (to estimate students' probability of success in standard first-year courses) and an instructional component (in which academically underprepared students are taught the skills and knowledge they need to succeed in the standard courses). Validity issues related to these functions are discussed in the context of decision theory, and methods are proposed for determining appropriate cutoff scores on placement tests.     

Reading,Writing, and English Course Pathways when Developmental Education is Optional: Course Enrollment and Success for Underprepared First-time-in-College Students

Academic underpreparedness is an issue for many first-time-in-college students, particularly those entering community colleges. Whereas many underprepared students enroll in developmental education, research has indicated that traditional remediation may not increase students’ chances for success. Therefore, states and colleges have begun to implement new course placement strategies to increase the accuracy of initial course placement and new instructional approaches to better serve their developmental students. Specifically, in 2013, the state of Florida passed Senate Bill 1720 which redesigned developmental coursework and placement policies across the Florida College System. The reform lifted developmental education placement exam testing and course enrollment requirements for certain exempt students, irrespective of prior academic preparation or achievement. The current study focuses on these exempt students—those who had the option to bypass developmental education—who were also underprepared, and their initial course selection and subsequent success in their gateway (introductory college-level) English course. Using statewide student-level data and logistic regression techniques, the results indicated that level of preparation was related to students’ course enrollment and gateway English course success. Students slightly underprepared in reading or writing were more likely than severely underprepared students to enroll in the gateway English class, relative to a developmental reading or writing course. In reading and writing, slightly underprepared students were more likely to pass English, relative to severely underprepared students. The authors consider the findings in light of recent national changes to developmental education and offer recommendations for policy and practice.     

Validating a computerized scoring system for assessing writing and placing students in composition courses

How do scores from writing samples generated by computerized essay scorers compare to those generated by “untrained” human scorers and what combination of scores, if any, is more accurate at placing students in composition courses? This study endeavored to answer this two-part question by evaluating the correspondence between writing sample scores generated by the IntelliMetric™ automated scoring system and scores generated by University Preparation English faculty, as well as examining the predictive validity of both the automated and human scores. The results revealed significant correlations between the faculty scores and the IntelliMetric™ scores of the ACCUPLACER™ OnLine WritePlacer Plus test. Moreover, logistic regression models that utilized the IntelliMetric™ scores and average faculty scores were more accurate at placing students (77% overall correct placement rate) than were models incorporating only the average faculty score or the IntelliMetric™ scores.     

Reading research for students with LD: a meta-analysis of intervention outcomes

The present article provides a meta-analysis of instructional research with samples of children and adolescents with learning disabilities in the domains of word recognition and reading comprehension. The results of the synthesis showed that a prototypical intervention study has an effect size (ES) of .59 for word recognition and .72 for reading comprehension. Four important findings emerged from the synthesis: (a) Effect sizes for measures of comprehension were higher when studies included derivatives of both cognitive and direct instruction, whereas effect sizes were higher for word recognition when studies included direct instruction; (b) effect sizes related to reading comprehension were more susceptible to methodological variation than studies of word recognition; (c) the magnitude of ES for word recognition studies was significantly related to samples defined by cutoff scores (IQ > 85 and reading < 25th percentile), whereas the magnitude of ES for reading comprehension studies was sensitive to discrepancies between IQ and reading when compared to competing definitional criteria; and (d) instructional components related to word segmentation did not enter significantly into a weighted least square hierarchical regression analysis for predicting ES estimates of word recognition beyond an instructional core model, whereas small-group interactive instruction and strategy cuing contributed significant variance beyond a core model to ES estimates of reading comprehension. Implications related to definition and instructional components that optimize the magnitude of outcomes are discussed.     

Evaluating an individualized science program for community college students

Two classes of community college students having less than the usual minimal preparation required for admission were taught half of a basic science course using individualized instruction and half using traditional instruction. The course was divided into chemistry and physics segments and random halves of each class received the individualized treatment in one segment and the traditional treatment in the other. The individualized treatment was found to yield superior achievement gains and more positive attitudes toward science than the traditional treatment while both resulted in equivalent student satisfaction. No differences between segments were obtained. Of the four treatment-segment combinations, individualized physics appeared to yield the greatest achievement. Individualized instruction was concluded to be effective.     

Regression Discontinuity Designs With Multiple Rating-Score Variables

Abstract

In the absence of a randomized control trial, regression discontinuity (RD) designs can produce plausible estimates of the treatment effect on an outcome for individuals near a cutoff score. In the standard RD design, individuals with rating scores higher than some exogenously determined cutoff score are assigned to one treatment condition; those with rating scores below the cutoff score are assigned to an alternate treatment condition. Many education policies, however, assign treatment status on the basis of more than one rating-score dimension. We refer to this class of RD designs as “multiple rating score regression discontinuity” (MRSRD) designs. In this paper, we discuss five different approaches to estimating treatment effects using MRSRD designs (response surface RD; frontier RD; fuzzy frontier RD; distance-based RD; and binding-score RD). We discuss differences among them in terms of their estimands, applications, statistical power, and potential extensions for studying heterogeneity of treatment effects.     

THE ASSOCIATION AMONG STUDENT SUCCESS IN COURSES,PLACEMENT TEST SCORES,STUDENT BACKGROUND DATA,AND INSTRUCTOR GRADING PRACTICES

Growth in the use of testing to determine student eligibility for community college courses has prompted debate and litigation regarding over the equity, access, and legal implications of these practices. In California, this has resulted in state regulations requiring that community colleges provide predictive validity evidence of test-score?based inferences and course prerequisites. In addition, companion measures that supplement placement test scores must be used for placement purposes. However, for both theoretical and technical reasons the predictive validity coefficients between placement test scores and final grades or retention in a course generally demonstrate a weak relationship. The study discussed in this article examined the predictive validity of placement test scores with course grade and retention in English and mathematics courses. The investigation produced a model to explain variance in course outcomes using test scores, student background data, and instructor differences in grading practices. The model produced suggests that student dispositional characteristics explain the high proportion of variance in the dependent variables. Including instructor grading practices in the model adds significantly to the explanatory power and suggests that grading variations make accurate placement more problematic. This investigation underscores the importance of academic standards as something imposed on students by an institution and not something determined by the entering abilities of students.     

Offshore and onsite placement testing for English pathway programmes

English language programmes provide established pathways for international students seeking university admission in countries such as Australia and the United Kingdom. In order to refer international applicants to appropriate levels and durations of English language support prior to matriculation into their main course of study, pathway providers need effective and efficient language assessment tools. This report evaluates the effectiveness of an online vocabulary knowledge test as an index of English proficiency for university English pathway programme applicants (N = 177). The Timed Yes/No (TYN) test measures vocabulary recognition size and speed in a time- and resource-effective format. Test results were correlated with performance on a comprehensive placement test consisting of speaking, writing, reading and listening components. The predictive validity of word recognition accuracy (a proxy for size) and response time (a measure of efficiency) for placement test outcomes were examined independently and in combination. The TYN test scores’ sensitivity at predicting comprehensive placement test scores were assessed using a cut-score analysis resulting in an identification accuracy rate ranging from 76 to 86% for five critical band scores. The potential use of the online vocabulary-screening test for measuring international students’ English language proficiency is discussed in terms of reliability, validity, speed, usability and cost-effectiveness in onsite and offshore testing conditions.     

Birthday effects and preschool attendance

Young-for-grade students have been shown to receive lower grades and have a higher likelihood of retention compared to their oldest peers upon kindergarten entry. Our study of 1474 economically disadvantaged first-time kindergarteners investigates if preschool attendance may ameliorate some of the risks potentially associated with being young-for-grade. Using the state-mandated age cutoff date, we establish four groups of students based on age (oldest/youngest in the cohort) and preschool experience (attended preschool/did not attend preschool) and use multilevel linear and logistic regression models in analyzing early literacy scores as well as the likelihood of being retained. Our findings show that while preschool attendance is associated with higher emergent literacy performance, young students still experience higher retention risks compared to their older peers, regardless of preschool attendance and controlling for end-of-year literacy scores.     

The effects of passive and active learning on student preference and performance in an undergraduate basic science course

Active learning is based on self-directed and autonomous teaching methods, whereas passive learning is grounded in instructor taught lectures. An animal physiology course was studied over a two-year period (Year 1, n = 42 students; Year 2, n = 30 students) to determine the effects of student-led seminar (andragogical) and lecture (pedagogical) teaching methods on students' retention of information and performance. For each year of the study, the course was divided into two time periods. The first half was dedicated to instructor-led lectures, followed by a control survey in which the students rated the efficiency of pedagogical learning on a five-point Likert scale from one (strongly disagree) to five (strongly agree). During the second period, students engaged in andragogical learning via peer-led seminars. An experimental survey was then administered to students using the same scale as above to determine students' preferred teaching method. Raw examination scores and survey results from both halves of the course were statistically analyzed by ANOVA with Newman-Keuls multiple comparison test. By the end of the study, student preference for peer-led seminars increased [mean ± SD: (2.47 ± 0.94)/(4.03 ± 1.36), P < 0.04], and examination scores significantly increased [mean ± SD: (73.91% ± 13.18)/(85.77 ± 5.22), P < 0.001]. A majority of students (68.8%) preferred a method that contained peer-led seminars and instructor-led lectures. These results may indicate that integration of active and passive learning into undergraduate courses may have greater benefit in terms of student preference and performance than either method alone.     

Using checklists in a gross anatomy laboratory improves learning outcomes and dissection quality

Checklists have been widely used in the aviation industry ever since aircraft operations became more complex than any single pilot could reasonably remember. More recently, checklists have found their way into medicine, where cognitive function can be compromised by stress and fatigue. The use of checklists in medical education has rarely been reported, especially in the basic sciences. We explored whether the use of a checklist in the gross anatomy laboratory would improve learning outcomes, dissection quality, and students' satisfaction in the first-year Human Structure didactic block at Mayo Medical School. During the second half of a seven-week anatomy course, dissection teams were each day given a hardcopy checklist of the structures to be identified during that day's dissection. The first half of the course was considered the control, as students did not receive any checklists to utilize during dissection. The measured outcomes were scored on four practice practical examinations and four dissection quality assessments, two each from the first half (control) and second half of the course. A student satisfaction survey was distributed at the end of the course. Examination and dissection scores were analyzed for correlations between practice practical examination score and checklist use. Our data suggest that a daily hardcopy list of anatomical structures for active use in the gross anatomy laboratory increases practice practical examination scores and dissection quality. Students recommend the use of these checklists in future anatomy courses.     

Predictors of placement for children who initially remained in their homes after an investigation for abuse or neglect

Objective

To examine the frequency and predictors of out-of-home placement in a 30-month follow-up for a nationally representative sample of children investigated for a report of maltreatment who remained in their homes following the index child welfare report.

Methods

Data came from the National Survey of Child and Adolescent Well-being (NSCAW), a 3-year longitudinal study of 5,501 youth 0-14 years old referred to child welfare agencies for potential maltreatment between 10/1999 and 12/2000. These analyses focused on the children who had not been placed out-of-home at the baseline interview and examined child, family and case characteristics as predictors of subsequent out-of-home placement. Weighted logistic regression models were used to determine which baseline characteristics were related to out-of-home placement in the follow-up.

Results

For the total study sample, predictors of placement in the 30-month follow-up period included elevated Conflict Tactics Scale scores, prior history of child welfare involvement, high family risk scores and caseworkers’ assessment of likelihood of re-report without receipt of services. Higher family income was protective. For children without any prior child welfare history (incident cases), younger children, low family income and a high family risk score were strongly related to subsequent placement but receipt of services and case workers’ assessments were not.

Conclusions/practice implications

Family risk variables are strongly related to out-of-home placement in a 30-month follow-up, but receipt of child welfare services is not related to further placements. Considering family risk factors and income, 25% of the children who lived in poor families, with high family risk scores, were subsequently placed out-of-home, even among children in families who received child welfare services. Given that relevant evidence-based interventions are available for these families, more widespread tests of their use should be explored to understand whether their use could make a substantial difference in the lives of vulnerable children.     

Concurrent enrollment in lecture and laboratory enhances student performance and retention

Laboratories have been a cornerstone in teaching and learning across multiple scientific disciplines for more than 100 years. At the collegiate level, science laboratories and their corresponding lectures are often offered as separate courses, and students may not be required to concurrently enroll in both. In this study, we provide evidence that enrolling in an introductory laboratory concurrently with the corresponding lecture course enhances learning gains and retention in comparison to students who enroll in the lecture alone. We examined the impact of concurrent versus nonconcurrent enrollment on 9,438 students' withdrawal rates from and final grades in the general chemistry lecture at the University of Michigan at Ann Arbor using multiple linear and binary logistic regression analyses, respectively, at a significance level of 0.05. We found that concurrent enrollment in the lecture and laboratory positively impacts (1) the odds of retention in the lecture by 2.2 times on average and (2) final lecture grades by up to 0.19 grade points on a 4.0 scale for the lowest‐scoring students according to university‐level mathematics and chemistry placement exam scores. These data provide important results for consideration by curriculum advisors and course planners at universities that do not require concurrent enrollment in general chemistry as well as other science courses. In the face of current budget cuts that threaten to shorten or eliminate laboratory experiences altogether at multiple educational levels, this study demonstrates the value of laboratories in promoting science learning and retention. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 659–682, 2012     

The Relationship of Essay and Multiple-Choice Scores With Grades in College Courses

Essay and multiple-choice scores from Advanced Placement (AP) examinations in American History, European History, English Language and Composition, and Biology were matched with freshman grades in a sample of 32 colleges. Multiple-choice scores from the American History and Biology examinations were more highly correlated with freshman grade point averages than were essay scores from the same examinations, but essay scores were essentially equivalent to multiple-choice scores in correlations with course grades in history, English, and biology. In history courses, men and women received comparable grades and had nearly equal scores on the AP essays, but the multiple-choice scores of men were nearly one half of a standard deviation higher than the scores of women.     

Exploratory Analysis of a Blended-learning Course Redesign for Developmental Writers

This quasiexperimental investigation describes the relative effects of a technology-driven course redesign on retention and achievement in a developmental writing program at a large, urban community college in the southwest United States. The new program focused on student learning and included online as well as face-to-face components that allowed students to complete the program in half the usual time in most cases. On average, students enrolled in the new program had lower placement test scores compared to students in the old program, but the gap was closed in the subsequent test and English course. Details of the redesign and challenges to conducting educational research and data analysis in community colleges are provided for other 2-year colleges considering course redesigns intended to impact student success.     

Adaptation of an anatomy graduate course in ultrasound imaging from in-person to live,remote instruction during the Covid-19 pandemic

Health concerns during the Covid-19 pandemic required the adaptation of a lecture-laboratory course in ultrasound imaging for graduate students from an in-person to a live, remote learning format. The adaptation of in-person lectures to live, remote delivery was achieved by using videoconferencing. The adaptation of in-person laboratory sessions to live, remote instruction was achieved in the first half of the course by providing a hand-held ultrasound instrument to each student who performed self-scanning at their remote locations, while the instructor provided live instruction using videoconferencing. In the second half of the course, the students transitioned to using cart-based, hospital-type instruments and self-scanning in the ultrasound laboratory on campus. The aim of this study was to measure the success of this adaptation to the course by comparing assessment scores of students in the live, remote course with assessment scores of students in the in-person course offered in the previous year. There were no statistically significant differences in the assessment scores of students in the two courses. The adaptation of a course in ultrasound imaging from an in-person to a live, remote learning format during the Covid-19 pandemic described here suggests that contrary to the prevailing view, ultrasound imaging can be taught to students without in-person instruction. The adapted course can serve as a model for teaching ultrasound where instructors and learners are physically separated by constraints other than health concerns during a pandemic.     

Logistic Regression and Its Use in Detecting Differential Item Functioning in Polytomous Items

A computer simulation study was conducted to determine the feasibility of using logistic regression procedures to detect differential item functioning (DIF) in polytomous items. One item in a simulated test of 25 items contained DIF; parameters' for that item were varied to create three conditions of nonuniform DIF and one of uniform DIF. Item scores were generated using a generalized partial credit model, and the data were recoded into multiple dichotomies in order to use logistic regression procedures. Results indicate that logistic regression is powerful in detecting most forms of DIF; however, it required large amounts of data manipulation, and interpretation of the results was sometimes difficult. Some logistic regression procedures may be useful in the post hoc analysis of DlF for polytomous items.     

PROGRAMS OF STUDY AS A BASIS FOR SELECTION, PLACEMENT AND GUIDANCE OF COLLEGE STUDENTS

Studies of collegiate success and attrition are generally conducted at the all-college level. The definition of academic programs that are homogeneous in the abilities and interests of their students and the grading standards of their faculties may lead to more accurate prediction of success and more effective control of attrition.
Homogeneous curricular groups were defined via Ward's hierarchical grouping analysis applied to curricular means on high school percentile rank, four ACT subscores, first semester GPA, and 16 Kuder scores. Programs so defined differed on scientific-verbal and competitive level dimensions. Prediction of grades was more accurate within programs than colleges. Drop and transfer rates were correlated with discriminant scores.
The programs are discussed as promising units within which differential selection and placement strategies might reduce attrition.     

慕课学习者群体的聚类分析——以“STEM课程设计与案例分析”慕课为例

文章选取"学堂在线"平台上参与"STEM课程设计与案例分析"慕课学习的1068名学习者进行研究,以视频观看比例、平时作业得分率、期末考试得分率、课程得分率四个变量来表征学习者的学习行为特征,并运用凝聚层次聚类分析、K-Means聚类算法、相关性分析和逻辑回归分析来处理获取的课程数据。研究结果表明,案例慕课的学习者群体可分成积极学习者、一般学习者、单纯注册者三类;四个变量在3个学习者群体之间均存在差异,其中平时作业得分率、期末考试得分率、课程得分率对积极学习者有显著影响,视频观看比例对单纯注册者有显著影响;视频观看比例是影响慕课学习者学习效果的主要因素。