A Guide for Studying the Personality of a Young Child

Five methods for analyzing data arising from research involving pretests and posttests are considered. These methods include: (1) posttest analysis only; (2) analysis of raw gain scores (posttest minus pretest); (3) analysis of the data with a pretest-posttest factor included in the statistical model; (4) analysis of posttest data with pretests as a covariate, and (5) analysis of gain scores with pretests as a covariate. The characteristics of each are discussed, with a conclusion that the fifth method is superior to the others when the assumptions underlying covariance analysis are met.     

Assessing Longitudinal Change: Adjustment for Regression to the Mean Effects

Pascarella (J Coll Stud Dev 47:508–520, 2006) has called for an increase in use of longitudinal data with pretest-posttest design when studying effects on college students. However, such designs that use multiple measures to document change are vulnerable to an important threat to internal validity, regression to the mean. Herein, we discuss a brief history of regression to the mean and illustrate a straightforward procedure to make adjustments to initial pretest scores for regression to the mean effects utilizing a method developed by Roberts (in: G. Echternacht (Guest ed.) New directions for testing and measurement, 1980). Analyses are shown with both unadjusted and adjusted pretest scores, illustrating dramatic differences in conclusions about whether students change across time.

Resolution of a Regression Paradox in Pretest-Posttest Designs

Regression to the mean (RTM) of individuals is the tendency for extreme individuals to become less extreme on remeasurement; RTM of group means describes this same tendency among group means. Under the classical test model, in pretest-posttest designs where subjects are selected on the basis of extreme values at the pretest, RTM of group means will always occur for the attribute used to select extreme subjects. For other attributes, however, RTM of group means requires a positive correlation between that attribute's measurement error and the measurement error of the attribute used in the selection. Thus, while all attributes will evidence RTM of individuals, extreme groups do not always regress to the mean. RTM depends most fundamentally on the magnitude of the pretest measurement error.     

The effects of the use of activities of sapa on the oral communication skills of disadvantaged kindergarten children

The study investigated the effects of participation in the activities of Science-A Process Approach (SAPA) on the oral communication skills of disadvantaged kindergarten children. Experimental-control group, pretest-posttest research design was employed. The investigator designed the Test of Oral Communication Skills (TOCS) to measure students' oral communication skills. Two experimental groups (one morning and one afternoon, N = 52) and two control groups (one morning and one afternoon, N = 48) were randomly assigned from two kindergarten classrooms (morning and afternoon classes in each) in an inner-city school in Columbus, Ohio. The investigator provided the science instruction to all students. Experimental subjects received 22 half-hour lessons from exercises a through k of Part A, SAPA. Control subjects received 22 lessons from Springboards to Science, a teacher demonstration, class discussion type of kindergarten curricuoum used in the school system. The treatment period spanned 12 academic weeks. Analysis of variance of TOCS pretest scores showed no significant differences among any of the means of the four groups. Analysis of covariance of TOCS posttest scores revealed differences significant at .05 or beyond in the mean scores of the experimental groups and control groups for total oral and total receiving skills and for six of the eight subskills measured by TOCS. Application of the Tukey (b) test revealed that experimental group mean scores exceeded control group mean scores in every TOCS skill and subskill area and that in total oral transmitting subskills both experimental group mean scores were significantly greater at .05 level or beyond than corresponding mean scores for both control groups. Within the parameters of this study, it may be concluded that when the activities and teaching strategies of SAPA curriculum are presented to disadvantaged kindergarten children, their oral communication skills, especially the transmitting (speaking) skills, are clearly enhanced.     

The Efficacy of Retesting in Relation to Improved Test Performance of College Undergraduates

Abstract

A causal-comparative analysis was done of the test performance of undergraduate students in five different sections of educational psychology taught over a 3-year period. A pretest-posttest design was used to test the hypothesis that a testing program employing fewer retests than original tests produced significant gains in mean highest test/retest scores. Original tests and retests were of comparable mean difficulty, discrimination, and cognitive level. Correlations between the unretested means and highest test/retest means for retesters in the five sections were statistically significant. Various retesting programs were not found to be significantly different in producing gains in mean test scores among retesters. Subgroup analysis revealed that retesting was used most frequently by students at the lower end of the marking spectrum. Further subgroup analysis revealed no significant difference in the frequency of retesting among sophomores, juniors, and seniors in the five sections. Students’ response to the use of retests was overwhelmingly positive.     

Testing the effectiveness of the project success summer program for adult dyslexics

The major purpose of this study was to test the effectiveness of the Project Success Summer Program (PSSP) for adult dyslexics. Using a pretest-posttest one-group design, the experimenter examined the effects of PSSP’s intensive eight-week program on the reading and spelling test scores of subjects (n=63). Data analysis indicated subjects’ reading and spelling scores improved significantly in five areas: reading of real words, passage reading rate, reading comprehension, spelling, and phoneme deletion. A second purpose of the study was to determine the best predictors of reading comprehension from among the study’s eight pretest reading and spelling measures. Regression analysis revealed that passage reading rates and individual word reading rates were the best predictors of reading comprehension scores. Interpretation of the results of this study suggest that training in phonological skills can improve poor readers’ basic decoding skills resulting in improved reading rate, comprehension and spelling.     

Differential Improvement in Student Understanding of Mathematical Principles Following Formative Assessment Intervention

ABSTRACT

The authors describe results from a study of a middle school mathematics formative assessment strategy. They employed a randomized, controlled design to address the following question: Does using our strategy improve student performance on assessments of key mathematical ideas relative to a comparison group? Eighty-five teachers and 4,091 students were included. Students took a pretest and a transfer measure at the end of the year. Treatment students completed formative assessments. Treatment teachers had exposure to professional development and instructional resources. Results indicated students with higher pretest scores benefited more from the treatment compared to students with lower pretest scores. In addition treatment students significantly outperformed control students on distributive property items. This effect was larger as pretest scores increased. Results, limitations, and future directions are discussed.     

The Superiority of Distributed Practice in Drill in Arithmetic

Abstract

Forty-three second and third grade non-readers who were diagnosed as having visual perceptual deficiencies were randomly assigned to experimental and control groups. The experimental group received training in visual analysis, which included visual short-term memory, visual discrimination of letters and words, and remedial instruction. The control group received only remedial instruction. After seven months, there was no significant difference between the groups in visual perception or achievement of reading skills. However, there was a significant mean difference between pretest and posttest scores for both groups on all measures tested. Experimental subjects gained 8.9 months in reading achievement versus 9.2 months for controls, indicating that visual discrimination and visual memory of word training influenced the achievement of reading skills for this group to the same degree as did a remedial program without visual training.     

The Ecological Message of the Outdoor Magazines

Abstract

The purpose of this study was to examine (a) college students' attitudes and complexity of thinking about the Endangered Species Act (ESA) and (b) the effects of environment-based coursework on students' attitudes and thinking. Using self-report questionnaires in a pretest-posttest design, the authors examined attitudes in terms of their direction, extremity, ambivalence, and importance. Complexity of thinking was measured as integrative complexity. Results suggested that college students (N = 205) who had moderate and ambivalent attitudes toward the ESA wrote significantly more integratively complex essays about the issue than did students who had unambivalent attitudes. Students' integratively complex thinking was not related to the direction of their attitudes toward the ESA or its personal importance to them. Students who were enrolled in an environment-based, university-wide writing course showed a significantly greater increase in integratively complex thinking about the ESA than did students enrolled in a nonenvironment-based, university-wide writing course.     

Effects of Early Field Experience on Student Teachers’ Self-Concepts and Performance

This research was designed to determine whether (1) selecting a pre-student teaching field-based experience brought about significant differences in secondary student teachers’ self-concepts and performance; and (2) whether self-concept scores were predictors of performance ratings. Student teachers who had selected Project Interaction (PI) and those who had no early field experience (NF) were pre-and posttested on four self-concept variables, and rated on performance by cooperating teachers, supervisors, and by self-ratings. Pretest self-concept scores were correlated with performance ratings. Results indicated the PI group had significantly more conflict on the pretest, which was reversed on the posttest to produce a trend toward significantly less conflict. The PI group had significantly higher self-esteem than the NF group on the posttest. No significant differences were found on performance ratings, but cooperating teachers’ ratings did correlate with pretest self-concept scores. It was concluded that early field experience appeared to have latent positive effects on self-concept.     

The Effects of Team-Based Learning on Social Studies Knowledge Acquisition in High School

Abstract

This randomized control trial examined the efficacy of team-based learning implemented within 11th-grade social studies classes. A randomized blocked design was implemented with 26 classes randomly assigned to treatment or comparison. In the treatment classes teachers implemented team-based learning practices to support students in engaging in dialogue about course content, application of content to solve problems, and use of evidence to support responses. Significant differences in favor of the treatment group on content acquisition were noted (Hedges's g =.19). Examination of differences in response to the treatment indicated groups of students classified with high or moderate pretest scores benefitted from the treatment, whereas a group of students classified with low pretest scores did not benefit from the treatment.     

Use of Restricted Item Response Theory Models for Examining the Stability of Item Parameter Estimates Over Time

Item parameter instability can threaten the validity of inferences about changes in student achievement when using Item Response Theory- (IRT) based test scores obtained on different occasions. This article illustrates a model-testing approach for evaluating the stability of IRT item parameter estimates in a pretest-posttest design. Stability of item parameter estimates was assessed for a random sample of pretest and posttest responses to a 19-item math test. Using MULTILOG (Thissen, 1986), IRT models were estimated in which item parameter estimates were constrained to be equal across samples (reflecting stability) and item parameter estimates were free to vary across samples (reflecting instability). These competing models were then compared statistically in order to test the invariance assumption. The results indicated a moderately high degree of stability in the item parameter estimates for a group of children assessed on two different occasions.     

Validity of a Residualized Dependent Variable After Pretest Covariance Adjustments: Still the Same Variable?

The authors empirically examined whether the validity of a residualized dependent variable after covariance adjustment is comparable to that of the original variable of interest. When variance of a dependent variable is removed as a result of one or more covariates, the residual variance may not reflect the same meaning. Using the pretest–posttest design as a general framework, the authors compared the nomological validity network for the (a) original dependent variable scores and (b) residualized dependent variable scores after having covaried-out variance explainable by a pretest. Heuristic and empirical examples are provided that demonstrate potential variation in construct validity of residualized dependent variables is a function of correlations among dependent, covariate, and validity variables.     

The effectiveness of a learning disabilities inservice program

Forty-three regular classroom teachers were provided with four one-hour inservice sessions related to the field of learning disabilities. Pre- and posttest administrations of the Learning Disabilities Information Inventory were utilized to measure change resulting from these sessions. Twenty-four control subjects did not participate in these sessions but did complete the dependent measure. An analysis of covariance with pretest scores serving as the covariate indicated that the inservice treatment did result in improved performance on the dependent measure. Approximately 50% of the experimental subjects completed a follow-up questionnaire and most indicated that the sessions increased their knowledge of learning disabilities and provided them with techniques and materials useful in their classroom settings. It is concluded that such inservice sessions are a viable means of increasing classroom teachers' information concerning learning disabilities and the Learning Disabilities Information Inventory is sensitive to the effects of such inservice.     

Explaining differences between retrospective and traditional pretest self-assessments: competing theories and empirical evidence

Summarizing theory and results of empirical research, this article serves to illustrate why effects measured with retrospective pretests may be subject to bias and may not always be explained by response shift theory. It presents three contending theories to explain the difference between retrospective and traditional pretest results and considers how the evaluation environment may inform subject bias. Four recommendations are made for workforce education (WE) researchers and practitioners who employ retrospective pretest data to report programme outcomes. WE professionals should (a) consider the cognitive implications of tasking participants to recall information, (b) select a robust evaluation design to encompass the retrospective pretest, (c) provide validity evidence of retrospective pretest data, and (d) conduct additional research to evaluate how elements of the evaluation process moderate retrospective assessments.     

The effects of computer-assisted instruction and locus of control upon preservice elementary teachers' acquisition of the integrated science process skills

The purpose of this study was to determine the effects of computer-assisted instruction (CAI) versus a text mode of programmed instruction (PI), and the cognitive style of locus of control, on preservice elementary teachers' achievement of the integrated science process skills. Eighty-one preservice elementary teachers in six sections of a science methods class were classified as internally or externally controlled. The sections were randomly assigned to receive instruction in the integrated science process skills via a microcomputer or printed text. The study used a pretest-posttest control group design. Before assessing main and interaction effects, analysis of covariance was used to adjust posttest scores using the pretest scores. Statistical analysis revealed that main effects were not significant. Additionally, no interaction effects between treatments and loci of control were demonstrated. The results suggest that printed PI and tutorial CAI are equally effective modes of instruction for teaching internally and externally oriented preservice elementary teachers the integrated science process skills.     

Experimental problem solving: An instructional improvement field experiment

An instructional program based on expert-novice differences in experimental problem-solving performance was taught to grade 6 students (N = 265). Classes of students were randomly assigned to conditions in a delayed treatment design. Performance was assessed with multiple-choice and open-ended measures of specific transfer. Between group comparisons using pretest scores as a covariate showed that treatment condition students consistently outperformed controls; similar results were revealed in the within group comparisons. The achievement of the early treatment group did not decline in tests administered one month after the posttest.     

The Relationship of Cognitive Style and Instructional Style to the Learning Performance of Undergraduate Students

Abstract

The purpose of this study was to investigate the relative effect of two different instructional styles, “discovery” and “expository,” on the change in learning performance of subjects of contrasting cognitive styles, field dependent and field independent. Based on the administration of the Group Embedded Figures Test, sixty-four subjects (thirty-two field dependent and thirty-two field independent) were randomly assigned to one of nine experimental groups (three expository experimental groups, three discovery experimental groups, and three no-treatment groups). The Behavior Modification Achievement Test, an original research instrument, was administered to all subjects on a pretest-posttest basis to measure change in learning performance. ANOVA revealed no significant interaction tied to the manipulation of the independent variable.     

Measuring the Transfer of Training

This study examined the use of self-report to measure training transfer by comparing three training transfer assessment methods. Applying a framework provided by instructional alignment theory, the study tested the hypothesis that if training transfer is measured by assessment methods that vary in their degree of alignment with the post-training learning assessment, the training transfer scores would be higher as the degree of alignment increased. Instructional alignment is the extent to which stimulus conditions match across instructional components: intended outcomes, instructional processes, and assessment. Three training transfer assessments were administered to 40 telecommunications technicians approximately 60 days after they completed a training course. The moan transfer score for the job performance assessment with high alignment was significantly higher than the two self-report assessments of moderate and low alignment with effect size differences of 0.96 (p < .01) and 0.87 (p < .01), respectively. The mean scores for the two self-report assessments did not significantly differ. This study has implications for the extensive use of self-report to assess training transfer in both research and training evaluation programs.