Detecting potential hucksterism in meta-analysis using a follow-up fail-safe test

Meta-analysis is an analysis of analyses. It is a technique widely used by researchers and practitioners to aggregate and summarize statistically reported empirical educational research. In 10 years, meta-analysis appeared more than 600 times in research journals and dissertation abstracts. Although most meta-analyses were reported as significant, few of the findings determined how many unpublished “no-effect” studies, if sampled, would have invalidated significance. If significant meta-analysis results are over represented through selective sampling, hucksterism in the form of sampling bias exists. An explanation for using a follow-up test called the fail-safe N is provided with tables constructed to assist researchers and practitioners to estimate, without calculation, the relative stability of meta-analysis results. The implication is that failsafe N should routinely be used and reported in meta-analysis research.     

Curriculum-based measures of written expression for learning-disabled and nondisabled students

This study examined differences in written expression between a sample of learning-disabled and nondisabled middle school students, matched by grade and sex, using eight curriculum-based measures. All of the learning-disabled students had been identified as having written language deficits as part of their handicapping conditions. The nondisabled students showed superior written expression skills, especially on the production-independent measures. Implications of these results are discussed in relation to psycho educational assessment, the use of CBM for ongoing assessment of written expression, and future studies.     

The Effect of Functional Flow Diagrams on Apprentice Aircraft Mechanics' Technical System Understanding

Cognitive theory suggests that a key to expert performance lies in the internal organization of the expert's knowledge. The authors contend that the type of technical illustration used during instruction influences knowledge organization and greatly impacts students' understanding of the content. This paper describes an experimental study that tested the impact of one type of conceptual illustration on students' understanding of the structure, function, and behavior of complex technical systems. The results show that supplementing traditional technical instruction with functional flow diagrams can improve overall system understanding. The functional flow diagrams were also found to be an effective instructional aid for enhancing students' conceptual understanding of the causal behavior of systems. In addition, the use of the functional flow diagram was found to significantly improve the subjects' ability to construct conceptual models that were similar to those of an expert. The implications of using conceptual diagrams for technical instruction are discussed and recommendations for future research in this area are provided.     

Curriculum bias and reading achievement test performance

The overlap of words specifically taught in reading textbooks with the contents of standardized reading achievement tests may be a source of bias that is frequently overlooked in psychoeducational assessments. This study compares the standardized achievement test performance of 62 second graders receiving instruction in two different reading curricula (Open Court and Houghton-Mifflin) to determine whether either curriculum generates different quantitative estimates of reading achievement. Reading subtest scores derived from the Kaufman Test of Educational Achievement-Brief Form (K-TEA), the Wide Range Achievement Test-Revised (WRAT-R), and the Reading Recognition and Reading Comprehension subtests from the Peabody Individual Achievement Test (PIAT) were examined. Grade level equivalents and scaled scores from the California Achievement Test (CAT) were also examined. Three Curriculum × Test repeated measures ANOVAs were conducted using grade level scores (2×7), standard scores (2×4), and CAT scaled scores (2 × 5) as dependent measures. A significant Curriculum × Test interaction was identified, suggesting differences among tests in estimates of reading ability as a function of the reading program.     

IQ and achievement: 1930s to 1980s

One hundred thirty-eight fifth-grade students from three upstate New York school districts were administered a 1930s version of the Pintner General Ability Tests: Verbal Series. Scores from this test were compared and correlated with scores from current IQ and achievement tests. The Pintner means were similar to those of the current IQ tests; the correlations of the Pintner with the other tests were of the same order as those typically reported among contemporary measures; and item difficulty, as indicated by rank order, appeared similar for the current sample and the 1930s norm groups. The findings are discussed with reference to research that has shown sharp increases in IQs of Americans since the 1930s and questions concerning changes in mental ability measures.     

The retrograde motion of planets and children: Interpreting percentile rank

A common method used to interpret raw scores is to convert them into percentile rank. The broad application of percentile rank in almost all norm-referenced tests, and the exclusive use in some tests, requires the conversion of raw scores. Because percentile rank is not a linear transformation of raw scores, percentile-rank transformation in the lower and upper part of the raw-score distribution relates percentile rank and raw scores differently than in the center of the raw-score distribution. This paper describes the popularity of using percentile rank and offers an interpretation of the use of percentile rank with normal and skewed score distributions.     

Crisis intervention teams in the schools

When a crisis occurs in the school setting, school officials must be prepared to handle the situation effectively and efficiently. One of the ways school officials can handle a crisis situation is through the use of crisis intervention teams. The steps in establishing a crisis intervention team in a school setting include: determining goals, performing a needs assessment, finding model programs, developing a membership pool, developing a training program, preparing and maintaining a list of resources and plans for support services, establishing a communication network, designating a base of operations, planning team meetings, keeping records, and implementing the phases of debriefing. A crisis management scenario is presented and discussed in terms of the steps.