PERSISTENCE AND THE RELATIONSHIP TO ACADEMIC PERFORMANCE MEASURES: A LONGITUDINAL STUDY

The persistence rates of community college transfer students were contrasted to those of native university students over a six year period. A general trend was observed which suggested community college transfer students were less likely to persist when returning to school for their second or third semesters of coursework. However, those transfer students remaining for the third semester of coursework were just as likely to persist as the native students for the next semester. Questions were raised about the validity of the notion of “transfer shock”, which may actually be a reflection of the change in the student population being observed.     

THE CONTRIBUTION OF ACADEMIC INTEREST MEASURES TO THE DIFFERENTIAL PREDICTION OF MARKS

In a study of differential predictions of course marks in grades 12 and 13 interest measures were found to add appreciably to differential validities, particularly at the high school level. Differential validities remained quite low; however–typically in the .20's and .30's-probably because of high intercorrelations among the criteria. The relationship between intercorrelations of actual criteria and intercorrelations of predicted criteria was examined, and the implications of this relationship for differential validities were discussed.     

RELIABILITY AND STRUCTURAL VALIDITY OF THE TEACHER RATING SCALES OF EARLY ACADEMIC COMPETENCE

Currently, there are few strengths‐based preschool rating scales that sample a wide array of behaviors believed to be essential for early academic success. The purpose of this study was to assess the factor structure of a new measure of early academic competence for at‐risk preschool populations. The Teacher Rating Scales of Early Academic Competence (TRS‐EAC) includes two broad scales (Early Academic Skills and Early Academic Enablers) and was completed by 60 teachers for 440 children enrolled in Head Start and public preschool classrooms. Evidence from two exploratory factor analyses supported a five‐factor solution for the Early Academic Skills Scale (Creative Thinking, Critical Thinking Skills, Numeracy, Early Literacy, and Comprehension) and a five‐factor solution for the Early Academic Enablers Scale (Approaches to Learning, Social and Emotional Competence, Fine Motor Skills, Gross Motor Skills, and Communication). TRS‐EAC scores also demonstrated good to excellent reliability and were related to children's performance on direct measures of early academic skills.     

THE VALIDITY OF ACADEMIC PREDICTORS FOR BLACK AND WHITE STUDENTS AT A PREDOMINANTLY WHITE UNIVERSITY1

High school grades and both the verbal and mathematical scales of the Scholastic Aptitude Test (SAT-V and SAT-M, respectively) were examined as predictors of college grade point average in groups divided by race and sex. Results indicated that high school grades were not correlated as highly with college grades for black males as for the other three groups, although there were no significant differences in the correlation of either SAT-V or SAT-M with college grades. Moreover, the multiple regression equation for the black male group differed from the equations for the other groups in that SAT-V is the predictor of primary importance rather than high school grades. Weights derived on a random sample of the student body caused substantial shrinkage of the multiple R only in the black male sample. Both black males and black females were significantly overpredicted by such weights. The importance of separate prediction equations for race-sex groupings was emphasized.     

MEASURING INSTRUCTIONAL VALIDITY: A REPORT OF AN INSTRUCTIONAL VALIDITY STUDY FOR THE ALABAMA HIGH SCHOOL GRADUATION EXAMINATION

Prior to the initial administration of the Alabama High School Graduation Exam, this study was conducted to determine to what extent competencies to be measured by the exam were being taught in the public schools of Alabama.
Teachers of grades 7, 8, 9, and 10 were asked to report the proportion of students in their classes who had received instruction on each competency. The survey identifies a number of competencies that were not being extensively taught and provides insight as to possible reasons certain competencies are not uniformly taught.     

ACADEMIC STANDARDS IN THE CALIFORNIA COMMUNITY COLLEGES: A STUDY OF FACULTY PERCEPTIONS

This study assessed the opinions of faculty concerning academic standards in the classroom. It compared the responses of faculty with experience in the academic senates in 23 community colleges with faculty who had no senate experience. Differences of opinion also were examined based on the demographic variables of gender, age, ethnicity, level of education, years of teaching experience, and subject area taught. While faculty generally agreed with the importance of academic standards and with important considerations for establishing academic standards in the classroom, there were significant differences based on senate membership, gender, ethnicity, age, and subject area taught. Gender and ethnicity displayed the greatest number of differences. Campus‐wide discussions of academic standards, regional subject matter articulation conferences with nearby universities, and explicitly stated goals and objectives in all course syllabi were implications for practice emanating from this study.     

COMPARISON OF PREDICTIVE VALIDITY AND DIAGNOSTIC ACCURACY OF SCREENING MEASURES OF READING SKILLS

Assessment data must be valid for the purpose for which educators use them. Establishing evidence of validity is an ongoing process that must be shared by test developers and test users. This study examined the predictive validity and the diagnostic accuracy of universal screening measures in reading. Scores on three different universal screening tools were compared for nearly 500 second‐ and third‐grade students attending four public schools in a large urban district. Hierarchical regression and receiver operating characteristic curves were used to examine the criterion‐related validity and diagnostic accuracy of students’ oral reading fluency (ORF), Fountas and Pinnell Benchmark Assessment System (BAS) scores, and fall scores from the Measures of Academic Progress for reading (MAP). Results indicated that a combination of all three measures accounted for 65% of the variance in spring MAP scores, whereas a reduced model of ORF and MAP scores predicted 60%. ORF and BAS scores did not meet standards for diagnostic accuracy. Combining the measures improved diagnostic accuracy, depending on how criterion scores were calculated. Implications for practice and future research are discussed.     

EFFECTS OF EMPIRICAL OPTION WEIGHTING ON RELIABILITY AND VALIDITY OF AN ACADEMIC APTITUDE TEST1

Item options of shortened forms of the GRE Verbal and Quantitative tests were empirically weighted by two variants of a method originally attributed to Guttman (1941). When compared with formula scores, it was found that tests scored with the empirical weights were more reliable but less valid when correlated with undergraduate GPA. A factor analysis revealed large increases in variance accounted for by the first factor. It was suggested that the weighting procedures used tended to capitalize on omitting behavior which, although a highly reliable tendency, may be invalid.     

THE USE OF A FACTOR-ANALYTIC MODEL FOR ASSESSING THE VALIDITY OF GROUP COMPARISONS

This paper assesses the validity of a comparison of mean test scores for two groups of students, and of a longitudinal comparison of means within each group. Using LISREL, confirmatory factor analyses were carried out (a) to test the hypotheses of similar factor patterns, equal units of measurement, and equal accuracy of measurement between the two groups, and (b) to estimate the correlation between the latent traits measured by two successive test administrations in each group. The results indicate (a) that a comparison of the group means may be invalid because, although the factor pattern was the same for both groups, the factors were not measured in the same units, and (b) that longitudinal comparisons within each group are seriously complicated by evidence of structural change.     

ESTABLISHING THE VALIDITY OF A SET OF SUMMATIVE TEACHING PERFORMANCE SCALES

A policy at a New Zealand University to assess lecturers’ teaching performance as an administrative requirement provided for information to be obtained from the chairperson, colleagues, students and the individual being evaluated.

In designing the summative scales to obtain student feedback, four conditions were seen as important for achieving a system acceptable to both lecturers and students. These conditions—length of scale, time to complete the scale, face validity of the scale and the acceptability of the procedures used to distribute the scales, together with the context within which teaching takes place at the Institution, provided constraints to the design of the summative scales.

Following 98 lecturing and 43 course evaluations, the scales were subjected to a principal factor analysis. This analysis showed that the items most strongly related to overall teaching effectiveness, as assessed by students, included the lecturer's ability to communicate ideas and information and to stimulate interest in the subject.

By pooling student ratings from different teaching situations for any one lecture, a Teaching Performance Profile was prepared. This profile increased the reliability and validity of the information obtained. At the same time it provided a summary suitable for combining with the evaluative . information from the other three sources in making a judgement as to the individual's level of teaching competency.     

THE VALIDITY OF HIGHER-ORDER QUESTIONS AS A PROCESS INDICATOR OF EDUCATIONAL QUALITY

One way to assess the quality of education in post-secondary institutions is through the use of performance indicators. Studies that have compared currently popular process indicators (e.g., library size, percentage of faculty with PhD) found that after controlling for incoming student ability, these process indicators tend to be weakly associated with student outcomes (Pascarella and Terenzini, 2005). In addition, while much research has found that students increase their critical thinking skills as a result of attending college, little is known about what goes on during the college experience that contributes to this. The purpose of this research was to examine the validity of higher-order questions on tests and assignments as a process indicator by comparing it with gains in critical thinking skills among college students as an outcome indicator. The present research consisted of three studies that used different designs, samples, and instruments. Overall, it was found that frequency of higher-order questions can be a valid process indicator as it is related to gains in students’ critical thinking skills.     

A META ANALYSIS OF THE VALIDITY OF PREDICTORS OF PERFORMANCE IN LAW SCHOOL

Although there is considerable evidence that the Law School Admission Test (LSAT) and the undergraduate grade-point average (UGPA) have a useful degree of predictive validity, there is also a large variation in the magnitude of the coefficients across schools. Understanding this variation has important implications for the use and interpretation of results of a validity study conducted at an individual school. A meta analysis of the validity results and data on applicants to 154 law schools was conducted in an effort to better understand this observed variation. The standard deviation (SD) on the LSAT and the correlation between the LSAT and UGPA for accepted students at each law school accounted for 58.5% of the between-school variance in the multiple correlations of these two predictors with first-year average grade in law school. Sampling error accounted for an additional 12% of the variance. Hence, only a small fraction of the between-school variability in validities remains to be explained by other statistical artifacts of situational specificity factors. Mean validities and 90% credibility values for four adjustment procedures are reported as are the mean observed validities for different combinations of predictors.     

VALIDITY OF THE DISCRIMINATION INDEX AS A MEASURE OF ITEM QUALITY1

Numerous writers have suggested that the discrimination index may be helpful in identifying faulty test items. The purpose of this study was to investigate systematically the validity of the index for this purpose. To attain this objective, two forms of an arithmetic-reasoning test were written. In each form, the items were designed to vary in quality with respect to nine item-writing principles, and on the basis of the responses of 364 examinees, a discrimination index was computed for each item. Next, the items were rated independently for quality by three judges who used a check list of the nine item-writing principles. The average of their ratings for each item was used as the criterion for determining the validity of the indices. The results indicate that the discrimination index is a moderately valid measure of item quality. The implications of this finding are discussed.     

ACADEMIC AND SOCIAL INTEGRATION OF COMMUNITY COLLEGE STUDENTS: A CASE STUDY

The literature suggests that most studies on student retention have focused on student satisfaction at 4-year institutions where students tend to be more traditional in nature. The study discussed in this article investigated how Tinto's (Tinto, Russo, & Kadel, 1994) model of retention could be applied to 2-year institutions. It explored academic and social integration and their effects on student withdrawal rates as well as the effect of background skills on withdrawal rates. Study participants were 462 second-semester degree-seeking community college students who completed a survey regarding their satisfaction with the academic and social climate of the community college. Performance on Computer Placement Tests (CPTs) were correlated with withdrawal rates to determine the association between background skill levels and withdrawal patterns. No correlation was found between academic and social integration and withdrawal rates. However, findings did show that the poorer the CPT performance, the more likely students were to withdraw from courses.